Commit graph

620 commits

Author SHA1 Message Date
Viktor Dukhovni
6b01bed206 Support disabling any or all TLS or DTLS versions
Some users want to disable SSL 3.0/TLS 1.0/TLS 1.1, and enable just
TLS 1.2.  In the future they might want to disable TLS 1.2 and
enable just TLS 1.3, ...

This commit makes it possible to disable any or all of the TLS or
DTLS protocols.  It also considerably simplifies the SSL/TLS tests,
by auto-generating the min/max version tests based on the set of
supported protocols (425 explicitly written out tests got replaced
by two loops that generate all 425 tests if all protocols are
enabled, fewer otherwise).

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-01-19 09:57:15 -05:00
Richard Levitte
3f22ed2fcf The TLSProxy tests can't run if no-engine has been configured
Make sure they detect that.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2016-01-17 00:25:44 +01:00
Richard Levitte
9b56815d5f Do not use redirection on binary files
On some platforms, the shell will determine what attributes a file
will have, so while the program might think it's safely outputting
binary data, it's not always true.

For the sake of the tests, it's therefore safer to use -out than to
use redirection.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-01-14 13:15:45 +01:00
Viktor Dukhovni
d9e309a675 Fix verify(1) to report failure when verification fails
Regenerated expired test certificates, good for the next 100 years.

Reviewed-by: Dr. Stephen Henson <steve@openssl.org>
2016-01-13 17:55:17 -05:00
Matt Caswell
6b95a37833 Fix test_ordinals
This used to work but somewhere along the line it broke and was failing to
detect duplicate ordinals - which was the whole point of the test!

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-01-13 20:52:12 +00:00
Richard Levitte
60f9f1e1c8 VMS perl doesn't implement fork(), so don't run the TLSProxy tests there
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2016-01-13 19:00:14 +01:00
Richard Levitte
412a963c58 Pipes on VMS do not work well with binary data, use an intermediary file
VMS being a record oriented operating system, it's uncertain how the
'pipe' passes binary data from one process to another.  Experience
shows that we get in trouble, and it's probably due to the pipe in
itself being opened in text mode (variable length records).

It's safer to pass data via an intermediary file instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-01-13 17:27:06 +01:00
Richard Levitte
4034c38b25 Fix test/recipes/80-test_ca.t to work on VMS
VMS uses a variant of openssl.cnf named openssl-vms.cnf.

There's a Perl on VMS mystery where a open pipe will not SIGPIPE when
the child process exits, which means that a loop sending "y\n" to it
will never stop.  Adding a counter helps fix this (set to 10, we know
that none of the CA.pl commands will require more).

Reviewed-by: Rich Salz <rsalz@openssl.org>
2016-01-13 15:23:44 +01:00
Richard Levitte
74b31db68a Only run DANE tests when EC is supported
Since danetest is to test DANE rather than specific algorithms, it's
acceptable to require EC when testing it.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2016-01-11 17:32:17 +01:00
Viktor Dukhovni
c2e27310c7 Enable/disable crypto-mdebug just like other features
Also always abort() on leak failure.

Reviewed-by: Stephen Henson <steve@openssl.org>
2016-01-11 02:41:16 +00:00
Rich Salz
bbd86bf542 mem functions cleanup
Only two macros CRYPTO_MDEBUG and CRYPTO_MDEBUG_ABORT to control this.
If CRYPTO_MDEBUG is not set, #ifdef out the whole debug machinery.
        (Thanks to Jakob Bohm for the suggestion!)
Make the "change wrapper functions" be the only paradigm.
Wrote documentation!
Format the 'set func' functions so their paramlists are legible.
Format some multi-line comments.
Remove ability to get/set the "memory debug" functions at runtme.
Remove MemCheck_* and CRYPTO_malloc_debug_init macros.
Add CRYPTO_mem_debug(int flag) function.
Add test/memleaktest.
Rename CRYPTO_malloc_init to OPENSSL_malloc_init; remove needless calls.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-01-07 15:14:18 -05:00
Viktor Dukhovni
170b735820 DANE support for X509_verify_cert()
Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-01-07 13:48:59 -05:00
Kurt Roeckx
7946ab33ce Add support for minimum and maximum protocol version
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2016-01-02 10:47:52 -05:00
Matt Caswell
ef96e4a28f Add test for missing CertificateStatus message
If the client sends a status_request extension in the ClientHello
and the server responds with a status_request extension in the
ServerHello then normally the server will also later send a
CertificateStatus message. However this message is *optional* even
if the extensions were sent. This adds a test to ensure that if
the extensions are sent then we can still omit the message.

Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
2015-12-27 21:59:04 +00:00
Matt Caswell
e863d92010 Don't export internal symbols
On Linux when creating the .so file we were exporting all symbols. We should
only be exporting public symbols. This commit fixes the issue. It is only
applicable to linux currently although the same technique may work for other
platforms (e.g. Solaris should work the same way).

This also adds symbol version information to our exported symbols.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-12-15 16:46:48 +00:00
Richard Levitte
ea09088e17 Better splitting regexp for test_ordinals
Reviewed-by: Matt Caswell <matt@openssl.org>
2015-12-15 17:02:34 +01:00
Ben Laurie
94d6151236 Make no-dh work, plus other no-dh problems found by Richard.
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-12-11 18:38:38 +00:00
Dr. Stephen Henson
42a8b3f90a Extended master secret test script.
Reviewed-by: Matt Caswell <matt@openssl.org>
2015-12-08 16:33:04 +00:00
Rich Salz
0aca86b313 Run test_ordinals after update
Catch a common 'make update' failure: conflicting ordinals.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-12-02 12:40:38 -05:00
Matt Caswell
5705e05037 Add ASYNC tests
Add a suite of tests for the ASYNC_* functions

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-11-20 23:34:35 +00:00
Rich Salz
b0700d2c8d Replace "SSLeay" in API with OpenSSL
All instances of SSLeay (any combination of case) were replaced with
the case-equivalent OpenSSL.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-10-30 17:21:42 -04:00
Andy Polyakov
85833408b4 Test suite: chomp->s/\R// to harmonize with mingw 'make test'.
Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-10-13 19:48:03 +02:00
Andy Polyakov
4ada8be2a6 Test suite: minimal required to get mingw 'make test' work under Linux.
(part by Alessandro Ghedini)

Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-10-13 19:46:50 +02:00
Dr. Stephen Henson
71a64af331 Skip PSK tests for no-psk
Reviewed-by: Matt Caswell <matt@openssl.org>
2015-10-11 13:38:01 +01:00
Matt Caswell
5530d5187c Add a test for duplicated ordinals
Occaisionally we have had problems where there are duplicated ordinals in
libeay.num or ssleay.num. This adds a test for this issue.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-09-30 15:36:41 +01:00
Emilia Kasper
cf7f85927c Empty NewSessionTicket: test session resumption
Reviewed-by: Matt Caswell <matt@openssl.org>
2015-09-28 16:00:58 +02:00
Emilia Kasper
7f6d90ac75 Empty session ticket: add a test
Reviewed-by: Matt Caswell <matt@openssl.org>
2015-09-28 16:00:47 +02:00
Matt Caswell
349b310742 Fix the OCSP test on Windows
The windows test uses the pseudo file "nul" to indicate no file for the
-CApath option. This does not work on all versions of Windows. Instead use
the new -no-CApath option.

Reviewed-by: Andy Polyakov <appro@openssl.org>
2015-09-25 14:50:00 +01:00
Matt Caswell
1556d21850 Fix the rehash test on Windows
The openssl rehash command is not available on some platforms including
Windows. This change skips the associated tests if rehash is not available.

Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-09-22 23:27:33 +01:00
Dr. Stephen Henson
37f3a3b327 make no-dh work
Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-09-20 12:50:55 +01:00
Richard Levitte
c9f6be4ead Correct srp test
There actually is a "srp" feature to check the availability on

Reviewed-by: Stephen Henson <steve@openssl.org>
2015-09-20 02:06:51 +02:00
Richard Levitte
17979f5526 Correct jpake test
There actually is a "jpake" feature to check the availability on

Reviewed-by: Stephen Henson <steve@openssl.org>
2015-09-20 02:06:51 +02:00
Richard Levitte
6ff4211781 Correct sha tests
These tests were checking for specific sha variants, when they should
just check if "sha" is disabled.

Reviewed-by: Stephen Henson <steve@openssl.org>
2015-09-20 02:06:51 +02:00
Richard Levitte
7f990b0ee3 Correct whirlpool test
It depended on 'openssl no-wp', which always exited with code 0, so
this test would never be performed, and this, I never discovered that
the program it's supposed to run was misspellt.  Furthermore, the
feature to check is 'whirlpool', not 'wp'.

All corrected.

Reviewed-by: Stephen Henson <steve@openssl.org>
2015-09-20 02:06:51 +02:00
Dr. Stephen Henson
340166a9da Change test recipes to use disabled()
Reviewed-by: Richard Levitte <levitte@openssl.org>
2015-09-20 00:04:16 +01:00
Richard Levitte
cc0b19a7fc Make sure to actually use @smime_cms_comp_test when testing it...
Obvious typo, and it took configuring with 'zlib' to discover it,
otherwise there was a previous skip that bypassed this section
entirely.

Reviewed-by: Tim Hudson <tjh@openssl.org>
2015-09-19 22:55:55 +02:00
Richard Levitte
e008d1b267 Check if test_rehash is run as root
If test/recipes/40-test_rehash.t is executed as root, the last test
will fail, since the created directory will remain writable no matter
what.  Make sure it complains loudly about being run as root.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-18 21:32:29 +02:00
Richard Levitte
84d90cf335 Add a simple test for the new rehash command
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-09 19:12:09 +02:00
Richard Levitte
55b2258859 Fix a few tests that depended on the wrong algorithm check
test_ecdh and test_ecdsa are made to depend on no-ec being false.
test_hmac is made not to depend on algorithm at all.

Based on a contribution by Alessandro Ghedini <alessandro@ghedini.me>

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-08 02:47:30 +02:00
Richard Levitte
6b33d10686 Adjust the verify_extra test recipe to its executable
The verify_extra_test was recently changed to take its parameters as
arguments instead of having them hardcoded.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-08 00:54:49 +02:00
Richard Levitte
a351805733 Make sure that 80-test_ca.t finds all the config files
This recipe counted too much on being called with test/ as its current
working directory.  That's a mistake on, for example, Windows.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:59 +02:00
Richard Levitte
eb8cd5dec2 Add a recipe for the new null pointer test
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:59 +02:00
Richard Levitte
1c73c3bcf0 Change the 80-test_tsa recipe as per changes in testtsa
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
ca5d7dff72 Add a recipe for the new pbelu test
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
49b147f85b Remake the testsslproxy tests
The testsslproxy tests turned out to be useless as they were.  They
were really just for show and the results were ignore.  Now they are
changed into a more veerifiable test

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
5beb63c41e Incorporate recent changes that were originally made in test/testssl
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
24d794de25 Add a recipe for the new gmdiff test
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
50f0a9b3dd Correct test name
Some tests were copied from test_jpake, but the title wasn't changed
accordingly.  This might seem like a small thing, but it does affect
the log file name...

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
c27a404947 Remake test/sslsessionticktest.pl into a recipe
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
25be5f44b8 Adapt the libssl test harness testing scripts to new testing framework
This involves adding $TOP/util as perl library in test/run_tests.pl.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
53520ebe87 Check the validity of MINFO
MINFO may be an old file lying around, which might have
00-check_testexes.t produce incorrect results.  To make sure this
doesn't happen, check the variable VERSION in it against the same
variable in the top Makefile.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
2511c12bf2 Better method of skipping all the tests in 00-check_testexes.t
Before trying to read MINFO, we have no idea how many to test for, and
because skip expects to get an exact number somehow, it's better to
use 'plan skip_all'.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
8de4f3d3a6 Remove special x509 test conversions
Following the commit from July 2 that removed netscape formated certs,
it is no longer necessary to have conversion tests for it.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
2a74b62e3a Add a few missing tests
test_clienthello
test_packet
test_verify_extra
test_secmem

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
8368d46bca Rework 00-test_checkexes.t for VMS
Unfortunately, a file spec with character range globs interfere with
paths on VMS, and are therefore disabled.  Rework this test to collect
a list of expected tests and a list of all recipes and compare the two
using grep.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
fd99c6b599 Change OpenSSL::Test to be an extension of Test::More
It became tedious as well as error prone to have all recipes use
Test::More as well as OpenSSL::Test.  The easier way is to make
OpenSSL::Test an extension of Test::More, thereby having all version
checks as well as future checks firmly there.  Additionally, that
allows us to extend existing Test::More functions if the need would
arise.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
fb921436f3 Add version numbers on some modules we use.
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
204e41ed50 Tone down the requirements of a test that will go away.
00-check_testexes.t was a way for me to check that I didn't forget a
compiled test app.  The way it worked was to require MINFO to be present.
Considering the need for this test has diminished considerably at this
point, I might as well tone down the requirement, and have it skip the
test (and not fail it) if MINFO isn't present.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
93de4f58ef Simplify very simple test recipes further.
Very simple test recipes easily become tedious, so they might benefit
from being made as simple as possible.  Therefore, OpenSSL::Test::Simple
is born.  It currently provides but one function, simple_test(), which
takes a minimum of two parameters (test name and program to run), with
the optional third, being the algorithm to be checked for before
running the test itself.

All recipes with that simple thing to do have been rewritten to be as
minimal as possible.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
d11b43fdd3 Remove test targets from Makefile, have it use run_tests.pl
Also remove recipes/00-check_testalltests.t, since it will lack the
information from the now gone alltests target.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:58 +02:00
Richard Levitte
127d25903f Add recipes for misc other things we want to test
Note that this required a change in constant_time_test.c, as it says
"ok", which interferes with what Test::Harness expects to see.  I had
constant_time_test.c say "success" instead.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
88b8a5279f Add recipes for the larger protocols
This covers the certificate authority commands, the cms and smime
commands, OCSP, SSL and TSA.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
4fb35f8fcb Add engine and evp test recipes.
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
4650de3e43 Add recipes for tests related to certificates
Some of them make use of recipes/tconversion.pl.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
0c85cc5069 Add asymetric cipher test recipes
Some of them make use of recipes/tconversion.pl.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
904ae33426 Add a helper script for key file format conversion tests
As tests are done until now, there are a few scripts that look almost,
but not quite the same.  tkey, tx509, tcrl, tpkcs7, treq, tsid and
probably a few more.

recipes/tconversions.pl is a helper script that generalises the
function of each of those, and can then be used in a general manner
from test recipes.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
13350a0c0e Add the encryption test recipe
This tests all available openssl cipher commands.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
894025c642 Add recipes for individual block ciphers, stream ciphers and digests
These recipes all correspond to a compiled test program.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
f3356b7f49 Add math tests recipes
The math recipes are among the heavier, but also quite important.
For the BN test, we have previously relied on bc to verify the numbers.
Unfortunately, bc doesn't exist everywhere, making tests on some platforms
rather painful.  With the new recipe (recipes/10-test_bn.t), we rely
on perl's Math::BigInt and a homegrown simple calculator (recipes/bc.pl)
that can do enough to cover for bc.

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00
Richard Levitte
aec27d4d52 Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.

The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.

The main runner of the show is run_tests.pl.  As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:

	cd test
	OPENSSL_SRCDIR=.. perl run_tests.pl

The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t.  recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names.  recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.

Individual recipes can be run as well, of course, as they are perl
scripts in themselves.  For example, you can run only
recipes/00-check_testexes.t like so:

	cd test
	OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t

To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:

	use lib 'testlib';
	use OpenSSL::Test;

Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-09-07 16:10:57 +02:00