Since CI is engaged on per merge request basis, it can be wasteful to
run each request through all the tests, especially those resource
consuming. Idea is to mark most of tests as "extended" and provide a
way to opt-in by marking last commit with [extended tests] tag. It's
still not as optimal as one could wish, as decision to skip a test
still requires machine time, and it's taken in configured environment,
i.e. with updates and additional packages installed...
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2292)
Travis OS X utilization and backlog statistics suggest that it became
bottleneck for our integration builds with requests piling up for days
during working days of the week. Suggestion is to remove osx till
capacity is lesser issue.
Reviewed-by: Richard Levitte <levitte@openssl.org>
We should move it back to the BORINGTEST build when we are approaching
interoperability.
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
Reviewed-by: Rich Salz <rsalz@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/2123)
The fuzzers use -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION, and actually
get different results based on that. We should have at least some
targets that actually fully use the fuzz corpora.
Reviewed-by: Rich Salz <rsalz@openssl.org>
GH: #2023
Clang on Linux seems to catch things that we might miss otherwise.
Also, throw in 'no-deprecated' to make sure we test that as well.
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/1839)
- make 'pip install --user cpp-coveralls' conditional;
- limit no-stdio to single build per operating environment;
- omit enable-asan duplicate;
Reviewed-by: Richard Levitte <levitte@openssl.org>
Run tests with coverage and report to coveralls.io
For simplicity, this currently only adds a single target in a
configuration that attempts to maximize coverage. The true CI coverage
from all the various builds may be a little larger.
The coverage run has the following configuration:
- no-asm: since we can't track asm coverage anyway, might as well measure the
non-asm code coverage.
- Enable various disabled-by-default options:
- rc5
- md2
- ec_nistp_64_gcc_128
- ssl3
- ssl3-method
- weak-ssl-ciphers
Finally, observe that no-pic implies no-shared, and therefore running
both builds in the matrix is redundant.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Reviewed-by: Kurt Roeckx <kurt@openssl.org>
The rationale is that installation from a tarball is a common task
that everyone performs. For all other builds, we do specialised
tests, and might as well build them directly in the checkout, which
also gives us fuzz corpora.
Reviewed-by: Emilia Käsper <emilia@openssl.org>
- Use the new enable-ubsan and enable-asan configuration options.
- Separate ubsan and asan runs.
- In addition, run shared ubsan tests to get more coverage.
(Shared asan tests need a bit more thought to get them working.)
Reviewed-by: Rich Salz <rsalz@openssl.org>
The Unix build was the last to retain the classic build scheme. The
new unified scheme has matured enough, even though some details may
need polishing.
Reviewed-by: Rich Salz <rsalz@openssl.org>
Code without PEDANTIC has intentional "undefined" behaviour. To get best
coverage for both PEDANTIC and non-PEDANTIC codepaths, run the sanitizer
builds in two different configurations:
1) Without PEDANTIC but with alignment checks disabled.
2) With PEDANTIC.
To not overload Travis too much, run one build with clang and the other
with gcc (chosen at random).
Also remove a micro-optimization in CAST code to be able to
-fsanitize=shift. Whether shift sanitization is meaningful for crypto or
an obstacle is debatable but since this appears to be the only offender,
we might as well keep the check for now.
Reviewed-by: Richard Levitte <levitte@openssl.org>
In Travis, do --strict-warnings on BUILDONLY configurations. This
ensures that the tests run even if --strict-warnings fail, and avoids
hiding unrelated test failures.
Reviewed-by: Richard Levitte <levitte@openssl.org>
ccache + clang produces a false strcmp warning, see
https://llvm.org/bugs/show_bug.cgi?id=20144
Since this only happens with ccache and --strict-warnings, and
only with certain versions of glibc / clang, disabling
ccache is a reasonable short-term workaround.
Reviewed-by: Richard Levitte <levitte@openssl.org>
Although theoretically possible, Configure doesn't treat CC variable
set like this very well: CC="ccache i686-w64-mingw32-gcc"
Also, this Travis script doesn't recognise the possibility either.
Reviewed-by: Rich Salz <rsalz@openssl.org>
