Commit graph

66 commits

Author SHA1 Message Date
Pauli
e9a5932d04 Add weak platform independent PRNG to test framework.
Implement the GNU C library's random(3) pseudorandom number generator.
The algorithm is described: https://www.mscs.dal.ca/~selinger/random/

The rationale is to make the tests repeatable across differing platforms with
different underlying implementations of the random(3) library call.

More specifically: when executing tests with random ordering.

[extended tests]

Reviewed-by: Bernd Edlinger <bernd.edlinger@hotmail.de>
(Merged from https://github.com/openssl/openssl/pull/9463)
2019-07-29 09:11:15 +10:00
Pauli
4bd8b24045 remove end of line spaces
Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/9397)
2019-07-16 20:35:42 +10:00
Pauli
dd6b270618 Remove tab characters from C source files.
Some have been creeping into the source code.

Reviewed-by: Paul Yang <yang.yang@baishancloud.com>
(Merged from https://github.com/openssl/openssl/pull/9397)
2019-07-16 20:24:10 +10:00
Pauli
2d905f6715 Print thread IDs nicely.
Remove the union that effectively cast thread IDs to long integers before
display and instead print a hex dump of the entire object.

Refer #9191

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9194)
2019-06-21 08:29:44 +10:00
Richard Levitte
ba4341316c test/testutil/init.c, apps/openssl.c: add trace cleanup handle earlier
It turned out that the internal trace cleanup handler was added too
late, so it would be executed before OPENSSL_cleanup().
This results in address errors, as the trace code that's executed in
OPENSSL_cleanup() itself tries to reach for data that's been freed at
that point.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/9196)
2019-06-20 18:19:06 +02:00
Richard Levitte
d73458d17a Add tracing capability in test utilities
If a test program goes wrong, it's sometimes helpful to be able to
trace what goes on in libcrypto and libssl.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9191)
2019-06-19 12:43:05 +02:00
Matt Caswell
cb30e46b45 Really fail if we have a test failure
Commit c5f7a99645 broke the test framework such that some tests might
fail, but the test framework still gives a PASS result overall.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/9056)
2019-06-03 13:27:11 +01:00
Pauli
c5f7a99645 Test skip option.
Provide C test cases with the option to skip tests and subtests.

Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8695)
2019-05-09 20:21:09 +10:00
Pauli
7ffbd7ca96 OSSL_PARAM helper functions.
Provide a number of functions to allow parameters to be set and
retrieved in a type safe manner.  Functions are provided for many
integral types plus double, BIGNUM, UTF8 strings and OCTET strings.

All of the integer functions will widen the parameter data as
required.  This permits a degree of malleability in the parameter
definition.  For example a type can be changed from a thirty two bit
integer to a sixty four bit one without changing application code.
Only four and eight byte integral sizes are supported here.

A pair of real functions are available for doubles.

A pair of functions is available for BIGNUMs.  These accept any sized
unsigned integer input and convert to/from a BIGNUM.

For each OCTET and UTF8 strings, four functions are defined.  This
provide get and set functionality for string and for pointers to
strings.  The latter avoiding copies but have other inherent risks.

Finally, some utility macros and functions are defined to allow
OSSL_PARAM definition arrays to be specified in a simple manner.
There are two macro and one function for most types.  The exception
being BIGNUM, for which there is one macro and one function.

Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8451)
2019-03-12 19:12:12 +01:00
Richard Levitte
ce506d27ab testutil: ensure good treatment of argv on non-Unix platforms
From a Unix point of view, some other platform families have certain
quirks.  Windows command prompt doesn't expand globs into actual file
names, so we must do this.  VMS has some oddity with argv pointer size
that can cause crashes if you're not careful (by copying it to a less
surprising pointer size array).

The fixups already exist and are used in the apps/ code.  However, the
testutil code started using the opt routines from apps/ without
including the non-Unix fixups.  This change fixes that.

For VMS' sake, libtestutil gets an app_malloc() shim, to avoid sucking
in all of apps/apps.c.

Reviewed-by: Paul Dale <paul.dale@oracle.com>
(Merged from https://github.com/openssl/openssl/pull/8381)
2019-03-05 08:53:19 +01:00
Shane Lontis
a43ce58f55 Updated test command line parsing to support commmon commands
Reviewed-by: Paul Dale <paul.dale@oracle.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6975)
2019-02-11 15:31:51 +01:00
Sam Roberts
df4439186f Remove unnecessary trailing whitespace
Trim trailing whitespace. It doesn't match OpenSSL coding standards,
AFAICT, and it can cause problems with git tooling.

Trailing whitespace remains in test data and external source.

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Matthias St. Pierre <Matthias.St.Pierre@ncp-e.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/8092)
2019-02-05 16:25:11 +01:00
Christos Zoulas
ec91206fbe Add a format attribute to the format functions and fix the broken format
strings.

Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2018-12-09 22:02:50 -05:00
Christos Zoulas
5db120dc35 Fix const issues
Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2018-12-09 22:02:50 -05:00
Christos Zoulas
d8adf68cd6 Use a const variable
Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2018-12-09 22:02:50 -05:00
Christos Zoulas
8343229bc4 Use the proper fonst cast
Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2018-12-09 22:02:49 -05:00
Christos Zoulas
82f518acee don't pass const string where a modifyable string is expected
Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2018-12-09 22:02:49 -05:00
Christos Zoulas
706a8af015 add missing "void" in prototype.
Reviewed-by: Richard Levitte <levitte@openssl.org>

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2018-12-09 22:02:49 -05:00
Richard Levitte
909f1a2e51 Following the license change, modify the boilerplates in test/
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7767)
2018-12-06 14:19:22 +01:00
Matt Caswell
1212818eb0 Update copyright year
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7176)
2018-09-11 13:45:17 +01:00
Dr. Matthias St. Pierre
12bd4e141e testutil/driver.c: Fix function prototype warning [-Wstrict-prototypes]
(introduced by commit 9186016582, which added -Wstrict-prototypes)

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
(Merged from https://github.com/openssl/openssl/pull/6555)
2018-06-22 01:04:34 +02:00
Bernd Edlinger
9186016582 Add -Wstrict-prototypes option to --strict-warnings
[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/6542)
2018-06-21 19:04:19 +02:00
Matt Caswell
0d66475908 Update copyright year
Reviewed-by: Richard Levitte <levitte@openssl.org>
2018-02-27 13:59:42 +00:00
Richard Levitte
f11a023ada VMS: for testutil, make sure to use BIO_f_linebuffer
Without that, output comes one character per line.  It's the same
issue as has been observed before, this happens when using write()
on a record oriented stream (possibly unbuffered too).

This also uncovered a bug in BIO_f_linebuffer, where this would cause
an error:

    BIO_write(bio, "1\n", 1);

I.e. there's a \n just after the part of the string that we currently
ask to get written.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5352)
2018-02-13 21:14:07 +01:00
Matt Caswell
6738bf1417 Update copyright year
Reviewed-by: Richard Levitte <levitte@openssl.org>
2018-02-13 13:59:25 +00:00
Richard Levitte
3cb413da19 Add missing \n in some testutil output
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/5229)
2018-01-31 23:49:21 +01:00
Bernd Edlinger
fce78bd4ed Fix invalid function type casts.
Rename bio_info_cb to BIO_info_cb.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4493)
2017-12-15 19:33:48 +01:00
Pauli
b7af3f1433 Test support for time_t comparisons.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4797)
2017-11-28 08:56:45 +10:00
FdaSilvaYY
4483fbae10 Factorise duplicated code.
Extract and factorise duplicated string glue code.
Cache strlen result to avoid duplicate calls.
[extended tests]

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4719)
2017-11-13 07:52:35 -05:00
Andy Polyakov
3a63c0edab Resolve warnings in VC-WIN32 build, which allows to add /WX.
It's argued that /WX allows to keep better focus on new code, which
motivates its comeback...

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4721)
2017-11-13 10:58:57 +01:00
Josh Soref
46f4e1bec5 Many spelling fixes/typo's corrected.
Around 138 distinct errors found and fixed; thanks!

Reviewed-by: Kurt Roeckx <kurt@roeckx.be>
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3459)
2017-11-11 19:03:10 -05:00
Rich Salz
176db6dc51 Use "" not <> for internal/ includes
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4217)
2017-08-22 09:54:20 -04:00
Pauli
00dfbaad88 Fix ctype arguments.
Cast arguments to the various ctype functions to unsigned char to match their
documentation.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4203)
2017-08-22 07:35:08 +10:00
Pauli
b99fe5f492 Remove tests dependence on e_os.h
Apart from ssltest_old.c, the test suite relied on e_os.h for the
OSSL_NELEM macro and nothing else.

The ssltest_old.c also requires EXIT and some socket macros.

Create a new header to define the OSSL_NELEM macro and use that instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4186)
2017-08-18 09:50:25 +10:00
David von Oheimb
180c3fc775 Fix minor type warnings and risk of memory leak in testutil/driver.c
Discussion is in https://github.com/openssl/openssl/issues/4127

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4131)
2017-08-10 11:05:28 -04:00
Pauli
ad887416f1 Update the test framework so that the need for test_main is removed. Everything
that needed test_main now works using the same infrastructure as tests that used
register_tests.

This meant:
* renaming register_tests to setup_tests and giving it a success/failure return.
* renaming the init_test function to setup_test_framework.
* renaming the finish_test function to pulldown_test_framework.
* adding a user provided global_init function that runs before the test frame
    work is initialised.  It returns a failure indication that stops the stest.
* adding helper functions that permit tests to access their command line args.
* spliting the BIO initialisation and finalisation out from the test setup and
    teardown.
* hiding some of the now test internal functions.
* fix the comments in testutil.h

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3953)
2017-07-27 07:53:08 +10:00
Richard Levitte
74df8c4ce3 testutil: stanza files are text files, open them as such
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3939)
2017-07-15 19:11:31 +02:00
Pauli
9ee27200c9 Remove trailing blanks.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3918)
2017-07-14 07:36:26 +10:00
Matt Caswell
1e55873f25 Fix a memory leak in the new TAP filter BIO
[extended tests]

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3760)
2017-06-23 20:30:27 +01:00
Pauli
a69de3f201 TAP line filter BIO.
This is an implementation of a BIO filter that produce TAP compatible output
for the test framework.  The current test indentation level is honoured.

The test output functions have been modified to not attempt to indent
their output and to not include the leading '#' character.

The filter is applied to bio_err only.  bio_out is left unchanged, although
tests using bio_out have been modified to use bio_err instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3732)
2017-06-22 09:35:08 +10:00
Pauli
8fe3127cda Update tests to avoid printf to stdout/stderr when running as test cases.
Reviewed-by: Tim Hudson <tjh@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3710)
2017-06-19 22:37:16 -04:00
Richard Levitte
7b4d323092 Fix va_list processing in test_note()
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3704)
2017-06-17 15:52:41 +02:00
Pauli
3791646202 Add output routines to allow consistent formatting of memory, strings
and bignums.  These have been refactored into their own file, along with
their error displays.  The formatting follows the output format used
on error, except that bignums of sixty four bits or less are displayed
in a more compact one line form.

Added a TEST_note function for producing output without file and line
information.

Update the three tests that call BN_print so they use the new test
infrastructure instead.

Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3655)
2017-06-16 16:15:31 -04:00
Rich Salz
ae269dd8b7 Use common STANZA for bn and exp tests
Standardize file:line messages
Reduce buff size; move to end of STANZA
Add some Title entries (with blank line after)
Add Title to some BN test files.

Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3608)
2017-06-05 13:41:30 -04:00
Rich Salz
dd5918775a Fix va_arg all in test_error_c90
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3506)
2017-05-23 11:33:24 -04:00
Pauli
7ac5b84ea7 Reformat the string output to be more in line with the decisions made in #3465
Don't highlight excess when comparing unequal length strings.

Clean up the NULL / empty string display.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3514)
2017-05-22 08:41:05 -04:00
Pauli
3216f9605c Bring the memory output inline with the suggestions in #3465.
Excess bytes, when one block is longer than the other, are not explicitly
highlighted.

The NULL / zero length block output has been cleaned up.

Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3515)
2017-05-22 08:30:57 -04:00
Pauli
ffbaf06ade Reformat the output of BIGNUMS where test cases fail.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3465)
2017-05-19 13:24:50 +02:00
Pauli
5584fd7f3d Randomise the ordering of the C unit tests.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3393)
2017-05-15 14:47:51 +02:00
Rich Salz
1d0f116e47 Add "Title" directive to evp_test
Reviewed-by: Stephen Henson <steve@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3454)
2017-05-12 14:20:01 -04:00