We recently turned on the TLSv1.3 downgrade sentinels by default.
Unfortunately we are using a very old version of the BoringSSL test
runner which uses an old draft implementation of TLSv1.3 that also
uses the downgrade sentinels by default. The two implementations do
not play well together and were causing spurious test failures. Until
such time as we update the BoringSSL test runner we disable the failing
tests:
SendFallbackSCSV
In this test the client is OpenSSL and the server is the boring test runner.
The client and server fail to negotiate TLSv1.3 because the test runner is
using an old draft TLSv1.3 version. The server does however add the
TLSv1.3->TLSv1.2 downgrade sentinel in the ServerHello random. Since we
recently turned on checking of the downgrade sentinels on the client side
this causes the connection to fail.
VersionNegotiationExtension-TLS11
In this test the test runner is the client and OpenSSL is the server. The
test modifies the supported_versions extension sent by the client to only
include TLSv1.1 (and some other spurious versions), even though the client
does actually support TLSv1.2. The server successfully selects TLSv1.1, but
adds the TLSv1.3->TLSv1.1 downgrade sentinel. This behaviour was recently
switched on by default. The test runner then checks the downgrade sentinel
and aborts the connection because it knows that it really supports TLSv1.2.
VersionNegotiationExtension-TLS1
VersionNegotiationExtension-SSL3
The same as VersionNegotiationExtension-TLS11 but for TLSv1 and SSLv3.
ConflictingVersionNegotiation
In this test the client is the test runner, and OpenSSL is the server. The
client offers TLSv1.2 in ClientHello.version, but also adds a
supported_versions extension that only offers TLSv1.1. The
supported_versions extension takes precedence and the server (correctly)
selects TLSv1.1. However it also adds the TLSv1.3->TLSv1.1 downgrade
sentinel. On the client side it knows it actually offered TLSv1.2 and so the
downgrade sentinel check fails.
[extended tests]
Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/7013)
An error reason code has changed for one of the boring tests, so
ossl_config.json needed an update to take account of it.
[extended tests]
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/4778)
Commit bd990e2535 changed our handling of alerts. Some of the BoringSSl
tests were expecting specific errors to be created if bad alerts were sent.
Those errors have now changed as a result of that commit, so the BoringSSL
test config needs to be updated to match.
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/3549)
The previous 2 commits fixed some issues in the Boring tests. This
re-enables those tests.
[extended tests]
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2942)
The Boring runner attempts to enable the NULL-SHA ciphersuite using the
cipherstring "DEFAULT:NULL-SHA". However in OpenSSL DEFAULT permanently
switches off NULL ciphersuites, so we fix this up to be "ALL:NULL-SHA"
instead. We can't change the runner so we have to change the shim to
detect this.
(Merged from https://github.com/openssl/openssl/pull/2933)
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Emilia Käsper <emilia@openssl.org>
Update the list of suppressions so that we can run a later BoringSSL set
of tests. This also adds an ErrorMap to greatly reduce the number of
failing tests. The remaining tests that still fail are just disabled for
now.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2930)
The boring tests are currently failing because they send a PSK extension
which isn't in the last place. This is not allowed in the latest TLS1.3
specs. However the Boring tests we have are based on an old commit that
pre-date when that rule first appeared.
The proper solution is to update the tests to a later commit. But for now
to get travis to go green we disable the failing tests.
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2928)
The external BoringSSL tests had some failures as a result of
the extensions refactor. This was due to a deliberate relaxation
of the duplicate extensions checking code. We now only check
known extensions for duplicates. Unknown extensions are ignored.
This is allowed behaviour, so we suppress those BoringSSL tests.
Perl changes reviewed by Richard Levitte. Non-perl changes reviewed by Rich
Salz
Reviewed-by: Rich Salz <rsalz@openssl.org>
Reviewed-by: Richard Levitte <levitte@openssl.org>
The updated shim has the ability to skip tests using unimplemented flags.
This should reduce the number of test failures.
Reviewed-by: Richard Levitte <levitte@openssl.org>
This just disables all tests that fail at the moment. Over time we will
want to go over these and figure out why they are failing (and fix them if
appropriate)
Reviewed-by: Richard Levitte <levitte@openssl.org>
