Commit graph

77 commits

Author SHA1 Message Date
Matt Clay
5e2d91cad4 [stable-2.5] Reduce testing of Ubuntu versions.
Removing:

- Ubuntu 14.04 with Python 2.7
- Ubuntu 16.04 with Python 3.5

Keeping:

- Ubuntu 16.04 with Python 2.7
- Ubuntu 18.04 with Python 3.6.
(cherry picked from commit c8f2becb7a)

Co-authored-by: Matt Clay <matt@mystile.com>
2019-04-02 22:52:08 -07:00
Jordan Borean
34b42ab48b ansible-test: add Ubuntu 18.04 container (#50790) - 2.5 (#51156)
* ansible-test: add Ubuntu 18.04 container (#50790)

* ansible-test: Add Ubuntu 18.04 to Shippable CI nodes

* re-add ubuntu1604/3 to matrix

* forgot to add environment for git kill gpg-agent

(cherry picked from commit 828df4b336)

* ansible-test: apt cleanup and futher 18.04 fixes (#50906)

(cherry picked from commit bbad033a5e)
2019-01-31 13:39:19 -08:00
Jordan Borean
6dbea6d19e ansible-test: change Fedora 24, 25 to 28, 29 (#49586) - 2.5 (#49587)
* ansible-test: change Fedora 24 to 28 (#49586)

(cherry picked from commit 5392caee14)

* fix openssl_certificate tests

* ansible-test - swap Fedora 25 for 29 (#49634)

(cherry picked from commit 6a09db5131)

* test make sure everything passes

* Fix rpm_key test on Fedora 27.

(cherry picked from commit 39aa50022a)

* skip yum tests on Fedora hosts
2018-12-21 17:17:35 -08:00
Matt Clay
1059dee51f [stable-2.5] Backport test infra updates and test fixes. (#49204)
* Switch tests from RHEL 7.5 to 7.6.

(cherry picked from commit 6745ee7cc8)

* Remove CI platform: freebsd/10.4

(cherry picked from commit e6ffc4f89a)

* Add `--raw` option to ansible-test shell command.

It is currently supported only with the `--remote` option.

This makes it easier to troubleshoot new instances which are not
yet supported by the setup scripts used by ansible-test.

(cherry picked from commit 0826a00803)

* Support skip of platforms by version in tests. (#48826)

* Support skip of platforms by version in tests.

Previously a remote platform could be skipped completely using the alias:

`skip/{platform}` such as `skip/rhel`

Now a specific platform version can be skipped using the alias:

`skip/{platform}{version}` such as `skip/rhel7.6`

This feature is available for platforms specified with the `--remote` option.

* Add skip by version to the docs.

(cherry picked from commit 8066acc90c)

* Fix ansible-test skip warning message.

(cherry picked from commit 3b705efc93)

* Fix lookup_passwordstore test skipping. (#49178)

* Fix lookup_passwordstore test skipping.

Skip all of RHEL instead of specific versions.
Skip all of CentOS < 7 instead of specific versions.

This makes the test more robust when testing newer versions.

Tests could be executed on RHEL if EPEL was installed during the test.
(cherry picked from commit 704dae2cda)
2018-12-04 13:35:00 -08:00
Matt Clay
6c066efb88 Use 4 nodes for sanity tests in CI.
(cherry picked from commit 6a51b5ed26)
2018-11-05 14:49:13 -08:00
Matt Clay
380f45e7ca Split sanity tests into 3 groups.
(cherry picked from commit 5461bfbf31)
2018-11-05 14:49:13 -08:00
Matt Clay
034e5b6341 Use default Shippable container. (#46703)
(cherry picked from commit 40c9301b27)
2018-10-09 11:47:39 -07:00
Jordan Borean
65fa6c17a7 go back to FreeBSD 11.1 for tests due to 11.2 stability issues (#45872) (#45873)
(cherry picked from commit a7d372c3ba)
2018-09-25 09:12:51 -07:00
Matt Clay
062113731a Update RHEL and FreeBSD versions used for tests:
- RHEL 7.4 -> 7.5
- FreeBSD 11.1 -> 11.2

(cherry picked from commit c50d6f1944)
2018-09-12 10:19:07 -07:00
Matt Clay
fe61636d66 Remove unused test matrix entries. 2018-07-24 10:02:00 -07:00
Matt Clay
bf8f228fd5 Update Shippable integration test groups. (#43118)
* Update Shippable integration test groups.
* Update integration test group aliases.
* Rebalance AWS and Azure tests with extra group.
* Rebalance Windows tests with another group.

(cherry picked from commit 4e489d1be8)
2018-07-24 10:02:00 -07:00
Matt Clay
784956c219
Sync stable-2.5 test infrastructure with devel. (#39544)
Sync stable-2.5 test infrastructure with devel:

- shippable.yml
- test/runner/
- test/sanity/
- test/utils/
- docs/docsite/rst/dev_guide/testing/sanity/
- test/integration/targets/*/aliases
2018-05-01 15:47:24 -07:00
Matt Clay
abbc1ed4e6 Remove EOL openSUSE Leap 42.2 from CI. 2018-01-31 21:50:21 -08:00
Matt Clay
7abdab6c9e Convert ansible-test compile into a sanity test. 2018-01-25 09:45:36 -08:00
Matt Clay
6969d492d8 Split Windows tests into 4 groups. 2018-01-12 00:48:32 -08:00
Matt Clay
30093dc4ed
Add Python 3.7 to CI unit test matrix. (#34680)
* Add Python 3.7 to CI unit test matrix.
* Fix `os.errno` reference to be `errno`.
* Update test_aci unit test for Python 3.7.
2018-01-10 00:57:11 -08:00
Matt Clay
6f77a32b13 Run RHEL tests on Azure in 3 groups. 2017-12-13 23:21:05 -08:00
Matt Clay
6988d32da2 Update FreeBSD versions used in CI. 2017-12-05 19:45:39 -08:00
Matt Clay
2ece86fba1 Split up cloud tests to avoid 1 hour CI limit. 2017-11-23 02:33:56 -08:00
Matt Clay
cf1337ca9a Update ansible-test sanity command. (#31958)
* Use correct pip version in ansible-test.
* Add git fallback for validate-modules.
* Run sanity tests in a docker container.
* Use correct python version for sanity tests.
* Pin docker completion images and add default.
* Split pylint execution into multiple contexts.
* Only test .py files in use-argspec-type-path test.
* Accept identical python interpeter name or binary.
* Switch cloud tests to default container.
* Remove unused extras from pip install.
* Filter out empty pip commands.
* Don't force running of pip list.
* Support delegation for windows and network tests.
* Fix ansible-test python version usage.
* Fix ansible-test python version skipping.
* Use absolute path for log in ansible-test.
* Run vyos_command test on python 3.
* Fix windows/network instance persistence.
* Add `test/cache` dir to classification.
* Enable more python versions for network tests.
* Fix cs_router test.
2017-10-26 00:21:46 -07:00
Matt Clay
b44c38930b Run OS X tests in 3 groups in CI. 2017-10-16 09:41:50 -07:00
Matt Clay
795d19bc78 Revert "Temporarily disable OS X in CI."
This reverts commit b26ae1809c.
2017-10-03 23:03:58 -07:00
Matt Clay
b26ae1809c Temporarily disable OS X in CI. 2017-10-03 14:58:16 -07:00
Matt Clay
5a91ed0a3b Switch CI for OS X back to a single job. (#31265)
* Update Shippable scripts for group/non-group use.
* Switch CI for OS X back to a single job.
2017-10-03 14:37:05 -07:00
Matt Clay
72237b63e7 Run OS X and FreeBSD CI tests using groups. (#31254)
* Run OS X and FreeBSD CI tests using groups.
* Improve readability of Shippable jobs.
2017-10-03 12:43:46 -07:00
Matt Clay
c00ad0868e Update Windows CI groups from 2 to 3. 2017-09-14 23:54:28 -07:00
Matt Clay
67c7bd8745 Revert "Temporarily disable failing opensuse test jobs."
This reverts commit 31d2eb0828.
2017-09-07 11:47:53 -07:00
Matt Clay
31d2eb0828 Temporarily disable failing opensuse test jobs. 2017-09-06 10:03:01 -07:00
Matt Clay
52eb5032ba Remove unnecessary pre_ci section. 2017-09-05 22:03:22 -07:00
Matt Clay
8f4c8844c4 Divide Windows integration tests into 2 groups. (#29029)
* Divide Windows integration tests into 2 groups.
* Support `none` for `--changed-all-target`.
* Run 2 separate Windows groups on Shippable.
* Only run smoketest and minimal for the group1 job.
2017-09-05 15:32:26 -07:00
Matt Clay
55fb18b85c Split cloud tests into two CI groups. 2017-09-01 17:33:54 -07:00
Ganesh Nalawade
8a6e1531b7 Fix config timeout issue for network modules (#28261)
* Fix config timeout issue for network modules

* Re-enable network test

* Remove unwanted persistent connection timer

* Minor changes in description
2017-08-16 10:55:39 -04:00
Matt Clay
d2065cfc0d Temporarily disable network tests. 2017-08-15 21:31:02 -07:00
Matt Clay
a8b7f2a7e2 Skip CI for default GitHub patch/revert branches.
The default naming convention for patch branches created in
the GitHub UI when editing files is: {username}-patch-{number}

The default naming convention for revert branches created in
the GitHub UI when reverting a PR is: revert-{pr_number}-{pr_branch_name}

This exclusion will prevent Shippable from running on these
branches until a PR is created.
2017-08-14 00:59:02 -07:00
Matt Clay
af9b0ba47b Replace opensuse42.1 with opensuse42.3 in CI. 2017-08-05 12:56:31 -07:00
Matt Clay
9965435b56 Switch back to FreeBSD 11.0 for CI stability. 2017-08-02 22:24:40 -07:00
Matt Clay
c2f1dff997 Update remote OS versions tested in CI. 2017-08-02 13:28:23 -07:00
Matt Clay
7931e11437 Use subset of Windows tests for "all" tests. (#26830) 2017-07-14 16:51:32 -07:00
Matt Clay
4a013d6d69 Add RHEL 7.3 to ansible-test and CI. (#26550)
* Add support for RHEL to ansible-test.
* Enable RHEL 7.3 on Shippable.
2017-07-07 16:52:40 -07:00
Matt Clay
c4a2deda9d Restore disabled opensuse integration tests. 2017-05-17 17:13:27 +08:00
John R Barker
e9e2d67035 Run vyos_command Network tests in Shippable (#24514)
* Run VyOS Network tests in Shippable

* net_command no longer exists

* network/ci

* Disable other network tests

* pep8 fix

* Fix PEP 8 issue.
2017-05-12 12:49:12 +01:00
Matt Clay
e56c418d9d Temporarily disable opensuse on Shippable.
The tests are timing out on most CI runs now.
2017-05-12 18:32:00 +08:00
Matt Clay
17e07a27b2 Enable cloud tests for use with ansible-test. 2017-05-05 21:46:29 +08:00
Matt Clay
0587d59266 Use Shippable image: drydock/u16pytall:master (#23997)
* Use Shippable image: drydock/u16pytall:master
* Do not install python 3.6 on Shippable.
2017-04-26 15:04:28 +08:00
Matt Clay
9810429f5b Remove Ubuntu 12.04 from Shippable CI.
Ubuntu 12.04 will be end-of-life before stable-2.4 is released.
2017-03-23 11:51:38 -07:00
Matt Clay
2804d2e848 Split up unit tests on Shippable. (#21931)
* Add more verbose output to coverage combine.
* Run unit tests on CI separately by version.
2017-02-24 16:58:56 -08:00
John R Barker
54e6c9472e Disable Network integration tests (#21273)
This will disable the integration tests for all network platforms &
modules, though the unit tests will still run.

This is needed while a large refactor of the networking code is done.
2017-02-10 20:58:00 +00:00
Matt Clay
4e19adb919 Restore FreeBSD 11.0 tests for CI. 2017-02-08 17:01:17 -08:00
Matt Clay
085865cee8 Temporarily remove FreeBSD 11.0 from CI. 2017-01-27 10:31:12 -08:00
Matt Clay
25caa67d41 Disable pull of updated Shippable docker image.
This should improve reliability of our jobs on Shippable.

The Shippable AMIs should already have the latest docker image,
per Shippable support, so pull doesn't provide any benefit for us.
2017-01-19 11:43:07 -08:00