openssl/test
Richard Levitte bb0f7eca75 Add a test of the X509_STORE / X509_LOOKUP API
Fortunately, "openssl verify" makes good use of that API

Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2652)
2017-02-16 21:09:09 +01:00
..
certs Add ECDSA client certificates 2017-02-16 16:43:44 +00:00
ct Verify SCT signatures 2016-03-01 11:59:28 -05:00
d2i-tests add test for CVE-2016-7053 2016-11-10 13:04:11 +00:00
ocsp-tests Fix OCSP checking. 2012-12-07 18:47:47 +00:00
ossl_shim Move extension data into sub-structs 2017-01-09 22:26:47 -05:00
recipes Add a test of the X509_STORE / X509_LOOKUP API 2017-02-16 21:09:09 +01:00
smime-certs spelling fixes, just comments and readme. 2016-08-05 19:07:30 -04:00
ssl-tests Add client auth TLS 1.3 certificate selection tests 2017-02-16 16:43:44 +00:00
testlib Update the kex modes tests to check various HRR scenarios 2017-02-14 13:14:25 +00:00
aborttest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
afalgtest.c Handle inability to create AFALG socket 2016-06-13 17:28:40 +01:00
asn1_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
asynciotest.c Fix some extra or missing whitespaces... 2017-01-25 09:06:34 +00:00
asynctest.c Add test to show wrong behavior of ASYNC_WAIT_CTX 2017-02-13 15:29:42 +00:00
bad_dtls_test.c Solution proposal for issue #1647. 2016-11-12 22:26:20 -05:00
bftest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
bio_enc_test.c Fix bio_enc_test 2016-08-23 09:24:29 +01:00
bioprinttest.c Whitespace cleanup in apps 2016-06-29 09:56:39 -04:00
bntest.c bntest: do not stop on first fautl encountered 2017-02-01 02:03:29 +01:00
bntests.pl Make bntest be (mostly) file-based. 2016-11-28 12:26:05 -05:00
bntests.txt bntests.txt: add a couple of checks of possibly negative zero 2017-02-01 02:03:29 +01:00
build.info Add support for parameterized SipHash 2017-02-01 14:14:36 -05:00
CAss.cnf RT3809: basicConstraints is critical 2016-06-13 09:18:22 -04:00
CAssdh.cnf Import of old SSLeay release: SSLeay 0.9.0b 1998-12-21 10:56:39 +00:00
CAssdsa.cnf Import of old SSLeay release: SSLeay 0.9.0b 1998-12-21 10:56:39 +00:00
CAssrsa.cnf Import of old SSLeay release: SSLeay 0.9.0b 1998-12-21 10:56:39 +00:00
casttest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
CAtsa.cnf Use better defaults for TSA. 2015-11-20 13:40:53 +00:00
cipher_overhead_test.c Add unit test for ssl_cipher_get_overhead() 2016-11-02 14:00:11 +00:00
cipherlist_test.c update test 2017-02-08 02:16:28 +00:00
clienthellotest.c Fix a warning about an uninit var 2016-11-24 18:02:43 +00:00
cms-examples.pl Copyright consolidation: perl files 2016-04-20 09:45:40 -04:00
constant_time_test.c constant time test: include our internal/numbers.h rather than limits.h 2016-11-05 11:38:29 +01:00
crltest.c GH2176: Add X509_VERIFY_PARAM_get_time 2017-01-12 09:54:09 -05:00
ct_test.c Make sure things get deleted when test setup fails in ct_test.c 2016-11-16 13:54:17 +00:00
d2i_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
danetest.c Perform DANE-EE(3) name checks by default 2016-07-12 10:16:34 -04:00
danetest.in Perform DANE-EE(3) name checks by default 2016-07-12 10:16:34 -04:00
danetest.pem DANE support for X509_verify_cert() 2016-01-07 13:48:59 -05:00
destest.c spelling fixes, just comments and readme. 2016-08-05 19:07:30 -04:00
dhtest.c Fix the build and tests following constification of DH, DSA, RSA 2016-06-16 13:34:44 +01:00
dsatest.c Fix the build and tests following constification of DH, DSA, RSA 2016-06-16 13:34:44 +01:00
dtls_mtu_test.c Don't change the state of the ETM flags until CCS processing 2017-02-16 09:35:56 +00:00
dtlstest.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
dtlsv1listentest.c Simplify and rename SSL_set_rbio() and SSL_set_wbio() 2016-07-29 14:09:57 +01:00
ecdhtest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
ecdhtest_cavs.h Whitespace cleanup in apps 2016-06-29 09:56:39 -04:00
ecdsatest.c spelling fixes, just comments and readme. 2016-08-05 19:07:30 -04:00
ectest.c Fix a memory leak in EC_GROUP_get_ecparameters() 2016-08-22 15:10:03 +01:00
enginetest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
evp_extra_test.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
evp_test.c Call EVP_CipherFinal in CCM mode for tests. 2017-02-08 02:16:27 +00:00
evptests.txt Add support for parameterized SipHash 2017-02-01 14:14:36 -05:00
exdatatest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
exptest.c Change callers to use the new constants. 2016-08-10 10:07:37 -04:00
generate_buildtest.pl Move the building of test/buildtest_*. to be done unconditionally 2016-08-05 21:17:05 +02:00
generate_ssl_tests.pl Reorganize SSL test structures 2016-08-08 12:06:26 +02:00
gmdifftest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
handshake_helper.c Provide a test for the Encrypt-Then-Mac renegotiation crash 2017-02-16 09:35:56 +00:00
handshake_helper.h Add test support for TLS signature types. 2017-01-30 13:00:17 +00:00
hmactest.c Fix hmac test case 6 2016-06-30 08:52:37 -04:00
ideatest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
igetest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
md2test.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
md4test.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
md5test.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
mdc2_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
mdc2test.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
memleaktest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
methtest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
modes_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
P1ss.cnf Use 2K RSA and SHA256 in tests 2015-04-20 07:23:04 -04:00
P2ss.cnf Use 2K RSA and SHA256 in tests 2015-04-20 07:23:04 -04:00
p5_crpt2_test.c Useless includes 2016-06-18 16:30:24 -04:00
packettest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
pbelutest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
pkcs7-1.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
pkcs7.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
pkey_meth_test.c Add test to check EVP_PKEY method ordering. 2016-11-20 00:22:02 +00:00
pkits-test.pl Remove trailing whitespace from some files. 2016-10-10 23:36:21 +01:00
poly1305_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
r160test.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
randtest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
rc2test.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
rc4test.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
rc5test.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
README test/README: clarify test number groups 2017-02-16 21:09:09 +01:00
README.external Fix argument order in documentation 2016-11-04 10:38:54 +00:00
README.ssltest.md Add test support for TLS signature types. 2017-01-30 13:00:17 +00:00
rmdtest.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
rsa_test.c Deprecate the flags that switch off constant time 2016-06-06 11:09:06 +01:00
run_tests.pl Add a more versatile test chooser 2016-09-01 20:58:40 +02:00
sanitytest.c Platform sanity test 2016-07-08 15:56:55 -04:00
secmemtest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
serverinfo.pem Require ServerInfo PEMs to be named "BEGIN SERVERINFO FOR"... 2013-09-13 19:32:55 -07:00
sha1test.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
sha256t.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
sha512t.c check return values for EVP_Digest*() APIs 2016-07-15 14:09:05 +01:00
shibboleth.pfx Add PKCS#12 UTF-8 interoperability test. 2016-08-22 13:52:51 +02:00
shlibloadtest.c Fix no-dso (shlibloadtest) 2016-11-10 10:12:00 +00:00
siphash_internal_test.c Add support for parameterized SipHash 2017-02-01 14:14:36 -05:00
smcont.txt test/smcont.txt: trigger assertion in bio_enc.c. 2016-07-31 17:03:17 +02:00
srptest.c Add SRP test vectors from RFC5054 2016-10-01 13:46:54 +01:00
ssl_test.c Add test support for TLS signature types. 2017-01-30 13:00:17 +00:00
ssl_test.tmpl test/ssl_test.tmpl: make it work with elderly perl. 2016-08-16 12:43:44 +02:00
ssl_test_ctx.c Fix a mem leak in ssl_test_ctx.c 2017-02-16 17:14:57 +00:00
ssl_test_ctx.h Provide a test for the Encrypt-Then-Mac renegotiation crash 2017-02-16 09:35:56 +00:00
ssl_test_ctx_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
ssl_test_ctx_test.conf Port multi-buffer tests 2016-08-18 12:46:00 +02:00
sslapitest.c Test logging TLSv1.3 secrets. 2017-02-02 09:34:00 +00:00
sslcorrupttest.c Update the record layer to use TLSv1.3 style record construction 2016-12-05 17:05:40 +00:00
ssltest_old.c Move extension data into sub-structs 2017-01-09 22:26:47 -05:00
ssltestlib.c Update create_ssl_connection() to make sure its gets a session 2017-01-30 10:18:22 +00:00
ssltestlib.h Fix some clang warnings 2016-08-19 13:52:40 +01:00
Sssdsa.cnf Import of old SSLeay release: SSLeay 0.9.0b 1998-12-21 10:56:39 +00:00
Sssrsa.cnf Import of old SSLeay release: SSLeay 0.9.0b 1998-12-21 10:56:39 +00:00
test.cnf Use 2K RSA and SHA256 in tests 2015-04-20 07:23:04 -04:00
test_main.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
test_main.h Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
test_main_custom.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
test_main_custom.h Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
testcrl.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
testdsa.pem Add private/public key conversion tests 2015-03-29 03:26:12 +01:00
testdsapub.pem Add private/public key conversion tests 2015-03-29 03:26:12 +01:00
testec-p256.pem Add private/public key conversion tests 2015-03-29 03:26:12 +01:00
testecpub-p256.pem Add private/public key conversion tests 2015-03-29 03:26:12 +01:00
testp7.pem Change PKCS#7 test data to take account of removal of 2000-08-25 01:29:41 +00:00
testreq2.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
testrsa.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
testrsapub.pem Add private/public key conversion tests 2015-03-29 03:26:12 +01:00
testsid.pem Remove SSLv2 support 2014-12-04 11:55:03 +01:00
testutil.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
testutil.h Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
testx509.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
threadstest.c include/openssl: don't include <windows.h> in public headers. 2016-07-08 11:49:44 +02:00
tls13encryptiontest.c Fix crash in tls13_enc 2017-02-08 11:41:45 +00:00
tls13secretstest.c Test logging TLSv1.3 secrets. 2017-02-02 09:34:00 +00:00
uitest.c UI: fix uitest for VMS 2017-01-12 15:23:15 +01:00
Uss.cnf Create DSA and ECDSA certificates. 2015-09-02 21:22:44 +01:00
v3-cert1.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
v3-cert2.pem Import of old SSLeay release: SSLeay 0.8.1b 1998-12-21 10:52:47 +00:00
v3ext.c Add some accessor API's 2016-06-08 11:37:06 -04:00
v3nametest.c Copyright consolidation 02/10 2016-05-17 14:20:27 -04:00
verify_extra_test.c Fix some extra or missing whitespaces... 2017-01-25 09:06:34 +00:00
wp_test.c crypto/cryptlib.c: omit OPENSSL_ia32cap_loc(). 2016-06-22 20:20:37 +02:00
wpackettest.c Add a test for WPACKET_fill_lengths() 2017-01-30 10:18:24 +00:00
x509_internal_test.c Add main() test methods to reduce test boilerplate. 2016-11-09 16:07:16 +01:00
x509aux.c test/x509aux.c: Fix argv loop 2016-09-21 16:19:22 +02:00

How to add recipes
==================

For any test that you want to perform, you write a script located in
test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
{name} is a unique name of your choice.

Please note that if a test involves a new testing executable, you will need to
do some additions in test/Makefile.  More on this later.


Naming conventions
=================

A test executable is named test/{name}test.c

A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
digit number and {name} is a unique name of your choice.

The number {nn} is (somewhat loosely) grouped as follows:

00-04  sanity, internal and essential API tests
05-09  individual symmetric cipher algorithms
10-14  math (bignum)
15-19  individual asymmetric cipher algorithms
20-24  openssl commands (some otherwise not tested)
25-29  certificate forms, generation and verification
30-35  engine and evp
60-79  APIs
   70  PACKET layer
80-89  "larger" protocols (CA, CMS, OCSP, SSL, TSA)
90-99  misc


A recipe that just runs a test executable
=========================================

A script that just runs a program looks like this:

    #! /usr/bin/perl
    
    use OpenSSL::Test::Simple;
    
    simple_test("test_{name}", "{name}test", "{name}");

{name} is the unique name you have chosen for your test.

The second argument to `simple_test' is the test executable, and `simple_test'
expects it to be located in test/

For documentation on OpenSSL::Test::Simple, do
`perldoc test/testlib/OpenSSL/Test/Simple.pm'.


A recipe that runs a more complex test
======================================

For more complex tests, you will need to read up on Test::More and
OpenSSL::Test.  Test::More is normally preinstalled, do `man Test::More' for
documentation.  For OpenSSL::Test, do `perldoc test/testlib/OpenSSL/Test.pm'.

A script to start from could be this:

    #! /usr/bin/perl
    
    use strict;
    use warnings;
    use OpenSSL::Test;
    
    setup("test_{name}");
    
    plan tests => 2;                # The number of tests being performed
    
    ok(test1, "test1");
    ok(test2, "test1");
    
    sub test1
    {
        # test feature 1
    }
    
    sub test2
    {
        # test feature 2
    }
    

Changes to test/Makefile
========================

Whenever a new test involves a new test executable you need to do the
following (at all times, replace {NAME} and {name} with the name of your
test):

* among the variables for test executables at the beginning, add a line like
  this:

    {NAME}TEST= {name}test

* add `$({NAME}TEST)$(EXE_EXT)' to the assignment of EXE:

* add `$({NAME}TEST).o' to the assignment of OBJ:

* add `$({NAME}TEST).c' to the assignment of SRC:

* add the following lines for building the executable:

    $({NAME}TEST)$(EXE_EXT): $({NAME}TEST).o $(DLIBCRYPTO)
           @target=$({NAME}TEST); $(BUILD_CMD)