that needed test_main now works using the same infrastructure as tests that used
register_tests.
This meant:
* renaming register_tests to setup_tests and giving it a success/failure return.
* renaming the init_test function to setup_test_framework.
* renaming the finish_test function to pulldown_test_framework.
* adding a user provided global_init function that runs before the test frame
work is initialised. It returns a failure indication that stops the stest.
* adding helper functions that permit tests to access their command line args.
* spliting the BIO initialisation and finalisation out from the test setup and
teardown.
* hiding some of the now test internal functions.
* fix the comments in testutil.h
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3953)
This is an implementation of a BIO filter that produce TAP compatible output
for the test framework. The current test indentation level is honoured.
The test output functions have been modified to not attempt to indent
their output and to not include the leading '#' character.
The filter is applied to bio_err only. bio_out is left unchanged, although
tests using bio_out have been modified to use bio_err instead.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3732)
and bignums. These have been refactored into their own file, along with
their error displays. The formatting follows the output format used
on error, except that bignums of sixty four bits or less are displayed
in a more compact one line form.
Added a TEST_note function for producing output without file and line
information.
Update the three tests that call BN_print so they use the new test
infrastructure instead.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3655)
Standardize file:line messages
Reduce buff size; move to end of STANZA
Add some Title entries (with blank line after)
Add Title to some BN test files.
Reviewed-by: Andy Polyakov <appro@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3608)
Excess bytes, when one block is longer than the other, are not explicitly
highlighted.
The NULL / zero length block output has been cleaned up.
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3515)
In some cases, testutil outputs subtests like this:
1..6 # Subtest: progname
The standard set by Test::More (because there really is no actual
standard yet) gives this display:
# Subtest: progname
1..6
Until the standard is actually agreed upon, let's do it like
Test::More.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3446)
This includes support for:
- comparisions between pairs of BIGNUMs
- comparisions between BIGNUMs and zero
- equality comparison between BIGNUMs and one
- equality comparisons between BIGNUMs and constants
- parity checks for BIGNUMs
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3405)
Format the test failure output more nicely.
More vertical space is used to make things a little clearer. Tests are expected
to pass so this doesn't impact the normal case.
Strings and memory comparisons highlight differences.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3357)
It occurs when memory compares are made that are larger
than the on stack temporary buffers (either malloced or supplied).
Rework the test test so it doesn't use a macro with a branch.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3155)
Also added a internal error printing callback to be used both with
ERR_print_errors_cb() and with CRYPTO_mem_leaks_cb
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3345)
These functions aren't meant to be used directly by the test programs,
reflect that by making the declarations a little harder to reach, but
still available enough if there's a need to override them.
Reviewed-by: Andy Polyakov <appro@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3345)
Original rationale behind using write in testutil was to accommodate
no-stdio builds. But is there evidence that no-stdio users would have
write or pre-defined meaning for file descriptors 1 and 2? Correct
answer is to provide way for no-stdio users who want to exercise
tests to plug in own BIO, not to make assumption that they have write.
And since we don't have to make such assumption, we can as well go
for simplest that works with standard library as specified by C
language standard.
Reviewed-by: Richard Levitte <levitte@openssl.org>
This includes better signals of skips and subtests according to TAP 12,
and flushing stdout and stderr at the end of every test function to
make sure we get the output in good order.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3309)
With the perl test framework comes the output format TAP
(Test Anything Protocol, see http://testanything.org/) with
extra extension for subtests. This change extends that same
output format to any test program using testutil.
In this implementation, each test program is seen as a full test that
can be used as a subtest. The perl framework passes on the subtest
level to the test programs with the environment variable
HARNESS_OSSL_LEVEL. Furthermore, and series of tests added with
ADD_ALL_TESTS is regarded as another subtest level.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3296)
It's now built as a static library, and greatly simplified for test
programs, which no longer need to include test_main_custom.h or
test_main.h and link with the corresponding object files.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3243)
