2016-04-20 02:10:43 +00:00
|
|
|
#! /usr/bin/env perl
|
|
|
|
# Copyright 2015-2016 The OpenSSL Project Authors. All Rights Reserved.
|
|
|
|
#
|
|
|
|
# Licensed under the OpenSSL license (the "License"). You may not use
|
|
|
|
# this file except in compliance with the License. You can obtain a copy
|
|
|
|
# in the file LICENSE in the source distribution or at
|
|
|
|
# https://www.openssl.org/source/license.html
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
|
|
|
|
use strict;
|
|
|
|
use warnings;
|
|
|
|
|
2016-05-08 12:09:56 +00:00
|
|
|
# Recognise VERBOSE and V which is common on other projects.
|
|
|
|
BEGIN {
|
|
|
|
$ENV{HARNESS_VERBOSE} = "yes" if $ENV{VERBOSE} || $ENV{V};
|
|
|
|
}
|
|
|
|
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
use File::Spec::Functions qw/catdir catfile curdir abs2rel rel2abs/;
|
|
|
|
use File::Basename;
|
2016-05-30 09:20:37 +00:00
|
|
|
use if $^O ne "VMS", 'File::Glob' => qw/glob/;
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
use Test::Harness qw/runtests $switches/;
|
|
|
|
|
2016-01-30 00:05:33 +00:00
|
|
|
my $srctop = $ENV{SRCTOP} || $ENV{TOP};
|
|
|
|
my $bldtop = $ENV{BLDTOP} || $ENV{TOP};
|
|
|
|
my $recipesdir = catdir($srctop, "test", "recipes");
|
|
|
|
my $testlib = catdir($srctop, "test", "testlib");
|
|
|
|
my $utillib = catdir($srctop, "util");
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
|
2016-01-17 16:02:02 +00:00
|
|
|
# It seems that $switches is getting interpreted with 'eval' or something
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
# like that, and that we need to take care of backslashes or they will
|
|
|
|
# disappear along the way.
|
|
|
|
$testlib =~ s|\\|\\\\|g if $^O eq "MSWin32";
|
2015-08-13 17:38:59 +00:00
|
|
|
$utillib =~ s|\\|\\\\|g if $^O eq "MSWin32";
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
|
|
|
|
# Test::Harness provides the variable $switches to give it
|
|
|
|
# switches to be used when it calls our recipes.
|
2015-08-13 17:38:59 +00:00
|
|
|
$switches = "-w \"-I$testlib\" \"-I$utillib\"";
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
|
Add a more versatile test chooser
So far, the test runner (test/run_tests.pl) could get a list of tests
to run, and if non were given, it assumes all available tests should
be performed.
However, that makes skipping just one or two tests a bit of a pain.
This change makes the possibilities more versatile, run_checker.pl
takes these arguments and will process them in the given order,
starting with an empty set of tests to perform:
alltests The current set becomes the whole set of
available tests.
test_xxx Adds 'test_xxx' to the current set.
-test_xxx Removes 'test_xxx' from the current set. If
nothing has been added to the set before this
argument, the current set is first initialised
to the whole set of available tests, then
'test_xxx' is removed from the current set.
list Display all available tests, then stop.
If no arguments are given, 'alltests' is assumed.
Reviewed-by: Emilia Käsper <emilia@openssl.org>
2016-09-01 11:34:32 +00:00
|
|
|
my @alltests = find_matching_tests("*");
|
|
|
|
my %tests = ();
|
|
|
|
|
|
|
|
my $initial_arg = 1;
|
|
|
|
foreach my $arg (@ARGV ? @ARGV : ('alltests')) {
|
|
|
|
if ($arg eq 'list') {
|
|
|
|
foreach (@alltests) {
|
|
|
|
(my $x = basename($_)) =~ s|^[0-9][0-9]-(.*)\.t$|$1|;
|
|
|
|
print $x,"\n";
|
|
|
|
}
|
|
|
|
exit 0;
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
}
|
Add a more versatile test chooser
So far, the test runner (test/run_tests.pl) could get a list of tests
to run, and if non were given, it assumes all available tests should
be performed.
However, that makes skipping just one or two tests a bit of a pain.
This change makes the possibilities more versatile, run_checker.pl
takes these arguments and will process them in the given order,
starting with an empty set of tests to perform:
alltests The current set becomes the whole set of
available tests.
test_xxx Adds 'test_xxx' to the current set.
-test_xxx Removes 'test_xxx' from the current set. If
nothing has been added to the set before this
argument, the current set is first initialised
to the whole set of available tests, then
'test_xxx' is removed from the current set.
list Display all available tests, then stop.
If no arguments are given, 'alltests' is assumed.
Reviewed-by: Emilia Käsper <emilia@openssl.org>
2016-09-01 11:34:32 +00:00
|
|
|
if ($arg eq 'alltests') {
|
|
|
|
warn "'alltests' encountered, ignoring everything before that...\n"
|
|
|
|
unless $initial_arg;
|
|
|
|
%tests = map { $_ => 1 } @alltests;
|
|
|
|
} elsif ($arg =~ m/^(-?)(.*)/) {
|
|
|
|
my $sign = $1;
|
|
|
|
my $test = $2;
|
|
|
|
my @matches = find_matching_tests($test);
|
|
|
|
|
|
|
|
# If '-foo' is the first arg, it's short for 'alltests -foo'
|
|
|
|
if ($sign eq '-' && $initial_arg) {
|
|
|
|
%tests = map { $_ => 1 } @alltests;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (scalar @matches == 0) {
|
|
|
|
warn "Test $test found no match, skipping ",
|
|
|
|
($sign eq '-' ? "removal" : "addition"),
|
|
|
|
"...\n";
|
|
|
|
} else {
|
|
|
|
foreach $test (@matches) {
|
|
|
|
if ($sign eq '-') {
|
|
|
|
delete $tests{$test};
|
|
|
|
} else {
|
|
|
|
$tests{$test} = 1;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
warn "I don't know what '$arg' is about, ignoring...\n";
|
|
|
|
}
|
|
|
|
|
|
|
|
$initial_arg = 0;
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
}
|
|
|
|
|
Add a more versatile test chooser
So far, the test runner (test/run_tests.pl) could get a list of tests
to run, and if non were given, it assumes all available tests should
be performed.
However, that makes skipping just one or two tests a bit of a pain.
This change makes the possibilities more versatile, run_checker.pl
takes these arguments and will process them in the given order,
starting with an empty set of tests to perform:
alltests The current set becomes the whole set of
available tests.
test_xxx Adds 'test_xxx' to the current set.
-test_xxx Removes 'test_xxx' from the current set. If
nothing has been added to the set before this
argument, the current set is first initialised
to the whole set of available tests, then
'test_xxx' is removed from the current set.
list Display all available tests, then stop.
If no arguments are given, 'alltests' is assumed.
Reviewed-by: Emilia Käsper <emilia@openssl.org>
2016-09-01 11:34:32 +00:00
|
|
|
runtests(map { abs2rel($_, rel2abs(curdir())); } sort keys %tests);
|
|
|
|
|
|
|
|
sub find_matching_tests {
|
|
|
|
my ($glob) = @_;
|
Groundwork for a perl based testing framework
The idea with this perl based testing framework is to make use of
what's delivered with perl and exists on all sorts of platforms.
The choice came to using Test::More and Test::Harness, as that seems
to be the most widely spread foundation, even if perl is aged.
The main runner of the show is run_tests.pl. As it currently stands,
it's designed to run from inside Makefile, but it's absolutely
possible to run it from the command line as well, like so:
cd test
OPENSSL_SRCDIR=.. perl run_tests.pl
The tester scripts themselves are stored in the subdirectory recipes/,
and initially, we have two such scripts, recipes/00-check_testalltests.t
and recipes/00-check_testexes.t. recipes/00-check_testalltests.t will
pick out the dependencies of "alltests" in test/Makefile, and check if
it can find recipes with corresponding names. recipes/00-check_testexes.t
does something similar, but bases it on existing compiled test binaries.
They make it easy to figure out what's to be added, and will be
removed when this effort is finished.
Individual recipes can be run as well, of course, as they are perl
scripts in themselves. For example, you can run only
recipes/00-check_testexes.t like so:
cd test
OPENSSL_SRCDIR=.. perl recipes/00-check_testexes.t
To make coding easier, there's a routine library OpenSSL::Test, which
is reachable in a perl script like so:
use lib 'testlib';
use OpenSSL::Test;
Reviewed-by: Rich Salz <rsalz@openssl.org>
2015-04-17 17:44:48 +00:00
|
|
|
|
2017-03-09 18:08:03 +00:00
|
|
|
if ($glob =~ m|^[\d\[\]\?\-]+$|) {
|
|
|
|
return glob(catfile($recipesdir,"$glob-*.t"));
|
|
|
|
}
|
Add a more versatile test chooser
So far, the test runner (test/run_tests.pl) could get a list of tests
to run, and if non were given, it assumes all available tests should
be performed.
However, that makes skipping just one or two tests a bit of a pain.
This change makes the possibilities more versatile, run_checker.pl
takes these arguments and will process them in the given order,
starting with an empty set of tests to perform:
alltests The current set becomes the whole set of
available tests.
test_xxx Adds 'test_xxx' to the current set.
-test_xxx Removes 'test_xxx' from the current set. If
nothing has been added to the set before this
argument, the current set is first initialised
to the whole set of available tests, then
'test_xxx' is removed from the current set.
list Display all available tests, then stop.
If no arguments are given, 'alltests' is assumed.
Reviewed-by: Emilia Käsper <emilia@openssl.org>
2016-09-01 11:34:32 +00:00
|
|
|
return glob(catfile($recipesdir,"*-$glob.t"));
|
2015-09-15 12:07:10 +00:00
|
|
|
}
|