diff --git a/INSTALL b/INSTALL index e31431bfbe..36cb090090 100644 --- a/INSTALL +++ b/INSTALL @@ -314,7 +314,8 @@ Enable building of integration with external test suites. This is a developer option and may not work on all platforms. The only supported external test suite at the current time is - the BoringSSL test suite. + the BoringSSL test suite. See the file test/README.external + for further details. no-filenames Don't compile in filename and line number information (e.g. diff --git a/test/README.external b/test/README.external new file mode 100644 index 0000000000..68d84813bb --- /dev/null +++ b/test/README.external @@ -0,0 +1,65 @@ +Running the BoringSSL test suite with OpenSSL +============================================= + +It is possible to integrate external test suites into OpenSSL's "make test". At +the current time the only supported external suite is the one used by +BoringSSL. + +This capability is considered a developer option and may not work on all +platforms. + +In order to run the BoringSSL tests with OpenSSL, first checkout the BoringSSL +source code into an appropriate directory: + +$ git clone boringssl https://boringssl.googlesource.com/boringssl + +The BoringSSL tests are only confirmed to work at a specific commit in the +BoringSSL repository. Later commits may or may not pass the test suite: + +$ cd boringssl +$ git checkout 490469f850e + +From the OpenSSL source code configure to use the external tests: + +$ cd ../openssl +$ ./config enable-ssl3 enable-ssl3-method enable-weak-ssl-ciphers \ + enable-external-tests + +Note that using other config option than those given above may cause the tests +to fail. + +Run the OpenSSL tests by providing the path to the BoringSSL test runner in the +BORING_RUNNER_DIR environment variable: + +$ BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make test + +Note that the test suite may change directory while running so the path provided +should be absolute and not relative to the current working directory. + +To see more detailed output you can run just the BoringSSL tests with the +verbose option: + +$ VERBOSE=1 BORING_RUNNER_DIR=/path/to/boringssl/ssl/test/runner make \ + TESTS="test_external" test + + +Test failures and suppressions +============================== + +A large number of the BoringSSL tests are known to fail. A test could fail +because of many possible reasons. For example: + +- A bug in OpenSSL +- Different interpretations of standards +- Assumptions about the way BoringSSL works that do not apply to OpenSSL +- The test uses APIs added to BoringSSL that are not present in OpenSSL +- etc + +In order to provide a "clean" baseline run with all the tests passing a config +file has been provided that suppresses the running of tests that are known to +fail. These suppressions are held in the file "test/ossl_shim/ossl_config.json" +within the OpenSSL source code. + +The community is encouraged to contribute patches which reduce the number of +suppressions that are currently present. +