openssl/test/testutil.h
Emilia Kasper 453dfd8d5e New SSL test framework
Currently, SSL tests are configured via command-line switches to
ssltest.c. This results in a lot of duplication between ssltest.c and
apps, and a complex setup. ssltest.c is also simply old and needs
maintenance.

Instead, we already have a way to configure SSL servers and clients, so
we leverage that. SSL tests can now be configured from a configuration
file. Test servers and clients are configured using the standard
ssl_conf module. Additional test settings are configured via a test
configuration.

Moreover, since the CONF language involves unnecessary boilerplate, the
test conf itself is generated from a shorter Perl syntax.

The generated testcase files are checked in to the repo to make
it easier to verify that the intended test cases are in fact run; and to
simplify debugging failures.

To demonstrate the approach, min/max protocol tests are converted to the
new format. This change also fixes MinProtocol and MaxProtocol
handling. It was previously requested that an SSL_CTX have both the
server and client flags set for these commands; this clearly can never work.

Guide to this PR:
 - test/ssl_test.c - test framework
 - test/ssl_test_ctx.* - test configuration structure
 - test/handshake_helper.* - new SSL test handshaking code
 - test/ssl-tests/ - test configurations
 - test/generate_ssl_tests.pl - script for generating CONF-style test
   configurations from perl inputs

Reviewed-by: Richard Levitte <levitte@openssl.org>
2016-04-05 13:44:46 +02:00

135 lines
5.3 KiB
C

/*-
* Utilities for writing OpenSSL unit tests.
*
* More information:
* http://wiki.openssl.org/index.php/How_To_Write_Unit_Tests_For_OpenSSL
*
* Author: Mike Bland (mbland@acm.org)
* Date: 2014-06-07
* ====================================================================
* Copyright (c) 2014 The OpenSSL Project. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
*
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in
* the documentation and/or other materials provided with the
* distribution.
*
* 3. All advertising materials mentioning features or use of this
* software must display the following acknowledgment:
* "This product includes software developed by the OpenSSL Project
* for use in the OpenSSL Toolkit. (http://www.OpenSSL.org/)"
*
* 4. The names "OpenSSL Toolkit" and "OpenSSL Project" must not be used to
* endorse or promote products derived from this software without
* prior written permission. For written permission, please contact
* licensing@OpenSSL.org.
*
* 5. Products derived from this software may not be called "OpenSSL"
* nor may "OpenSSL" appear in their names without prior written
* permission of the OpenSSL Project.
*
* 6. Redistributions of any form whatsoever must retain the following
* acknowledgment:
* "This product includes software developed by the OpenSSL Project
* for use in the OpenSSL Toolkit (http://www.OpenSSL.org/)"
*
* THIS SOFTWARE IS PROVIDED BY THE OpenSSL PROJECT ``AS IS'' AND ANY
* EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OpenSSL PROJECT OR
* ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
* LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
* STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
* OF THE POSSIBILITY OF SUCH DAMAGE.
* ====================================================================
*/
#ifndef HEADER_TESTUTIL_H
# define HEADER_TESTUTIL_H
/*-
* SETUP_TEST_FIXTURE and EXECUTE_TEST macros for test case functions.
*
* SETUP_TEST_FIXTURE will call set_up() to create a new TEST_FIXTURE_TYPE
* object called "fixture". It will also allocate the "result" variable used
* by EXECUTE_TEST. set_up() should take a const char* specifying the test
* case name and return a TEST_FIXTURE_TYPE by value.
*
* EXECUTE_TEST will pass fixture to execute_func() by value, call
* tear_down(), and return the result of execute_func(). execute_func() should
* take a TEST_FIXTURE_TYPE by value and return zero on success or one on
* failure.
*
* Unit tests can define their own SETUP_TEST_FIXTURE and EXECUTE_TEST
* variations like so:
*
* #define SETUP_FOOBAR_TEST_FIXTURE()\
* SETUP_TEST_FIXTURE(FOOBAR_TEST_FIXTURE, set_up_foobar)
*
* #define EXECUTE_FOOBAR_TEST()\
* EXECUTE_TEST(execute_foobar, tear_down_foobar)
*
* Then test case functions can take the form:
*
* static int test_foobar_feature()
* {
* SETUP_FOOBAR_TEST_FIXTURE();
* [...set individual members of fixture...]
* EXECUTE_FOOBAR_TEST();
* }
*/
# define SETUP_TEST_FIXTURE(TEST_FIXTURE_TYPE, set_up)\
TEST_FIXTURE_TYPE fixture = set_up(TEST_CASE_NAME); \
int result = 0
# define EXECUTE_TEST(execute_func, tear_down)\
if (execute_func(fixture) != 0) result = 1;\
tear_down(fixture);\
return result
/*
* TEST_CASE_NAME is defined as the name of the test case function where
* possible; otherwise we get by with the file name and line number.
*/
# if __STDC_VERSION__ < 199901L
# if defined(_MSC_VER)
# define TEST_CASE_NAME __FUNCTION__
# else
# define testutil_stringify_helper(s) #s
# define testutil_stringify(s) testutil_stringify_helper(s)
# define TEST_CASE_NAME __FILE__ ":" testutil_stringify(__LINE__)
# endif /* _MSC_VER */
# else
# define TEST_CASE_NAME __func__
# endif /* __STDC_VERSION__ */
/*
* In main(), call ADD_TEST to register each test case function, then call
* run_tests() to execute all tests and report the results. The result
* returned from run_tests() should be used as the return value for main().
*/
# define ADD_TEST(test_function) add_test(#test_function, test_function)
/*
* Simple parameterized tests. Adds a test_function(idx) test for each
* 0 <= idx < num.
*/
# define ADD_ALL_TESTS(test_function, num) \
add_all_tests(#test_function, test_function, num)
void add_test(const char *test_case_name, int (*test_fn) ());
void add_all_tests(const char *test_case_name, int (*test_fn)(int idx), int num);
int run_tests(const char *test_prog_name);
#endif /* HEADER_TESTUTIL_H */