Commit graph

104 commits

Author SHA1 Message Date
Matt Clay
fdcad0f44d Improve on-demand coverage for Shippable. 2017-05-14 18:27:58 +08:00
John R Barker
e9e2d67035 Run vyos_command Network tests in Shippable (#24514)
* Run VyOS Network tests in Shippable

* net_command no longer exists

* network/ci

* Disable other network tests

* pep8 fix

* Fix PEP 8 issue.
2017-05-12 12:49:12 +01:00
Matt Clay
dfd19a812f Miscellaneous bug fixes for ansible-test.
- Overhauled coverage injector to fix issues with non-local tests.
- Updated integration tests to work with the new coverage injector.
- Fix concurrency issue by using random temp files for delegation.
- Fix handling of coverage files from root user.
- Fix handling of coverage files without arcs.
- Make sure temp copy of injector is world readable and executable.
2017-05-12 14:55:48 +08:00
Matt Clay
fb9efd3c7e Add --remote-terminate option to ansible-test.
Set `--remote-terminate success` for osx tests.
2017-05-11 22:42:52 +08:00
Matt Clay
b8cb3f519b Detect and fix environment tampering in tests. 2017-05-11 19:45:15 +08:00
Matt Clay
e6934e42be Update ansible-test change handling and CS plugin. (#24395)
* Use CloudStack simulator from Ansible repo.
* Recognize cloud specific ansible-test changes.
* Hide ansible-test cs notice in explain mode.
2017-05-09 16:15:27 +08:00
Matt Clay
0573b8bcd2 Fix ansible-test cloud integration bugs. 2017-05-05 21:46:29 +08:00
Matt Clay
ac72fd9d2c Add missing return statements. 2017-05-05 18:37:11 +08:00
Matt Clay
201242f302 Fix call to wrong method in ansible-test. 2017-05-05 18:18:40 +08:00
Matt Clay
1582e9bf7d Fix handling of Shippable env vars. 2017-05-05 17:44:51 +08:00
Matt Clay
a07d42e16d Add support for cloud tests to ansible-test. (#24315)
* Split out ansible-test docker functions.
* Add cloud support to ansible-test.
2017-05-05 16:23:00 +08:00
Matt Clay
6522d703a1 Fix ansible-test pylint and pep8 issues. (#24245)
* Fix pylint useless-super-delegation issues.
* Fix miscellaneous pylint issues.
* Fix PEP 8 issues.
* Fix warnings reported by PyCharm.
2017-05-03 23:19:44 +08:00
Matt Clay
d662f6f0db Fix bugs in ansible-test units command. (#24044)
* Handle old versions of coverage.
* Handle old versions of setuptools.
* Detect python version for docker/remote units.
* Add sanity override for test constraints.
2017-04-27 11:21:11 +08:00
Matt Clay
cb1f57d4e5 Add rstcheck to ansible-test and correct issues. (#23550)
* Add rstcheck to ansible-test.
* Fix rst code-block languages and syntax errors.
* Fix rst inline literals.
* Update python 2 code block to pass tests on py 3.
2017-04-13 10:28:52 -07:00
Matt Clay
4ee96b20a3 Temporarily cap pylint version < 1.7.0 in tests. 2017-04-13 09:10:29 -07:00
Toshio Kuratomi
4e8e366739 Ryanpineo fix ipaddr netmask with 32 cidr (#23506)
* Fix bug in ipaddr netmask filter when cidr is 32

* Add requirement on netaddr to the units.txt file
2017-04-11 14:10:43 -07:00
Matt Clay
92ad9a19b6 Fix ansible-test empty diff handling on Shippable. 2017-04-08 22:08:08 -07:00
Matt Clay
0cf1e8e061 Fix ansible-test handling of import/syntax errors. 2017-04-07 12:20:31 -07:00
Matt Clay
1daa69d685 Initial pylint support for ansible-test sanity. 2017-04-04 16:16:21 -07:00
Ryan Brown
2196fa0e95 [cloud][tests] fixtures for placebo to test AWS modules using boto3 (#21253)
* [cloud][tests] Create fixtures for using placebo to test boto3-using modules

* Use pytest's importorskip instead of manually skipping on missing deps

* Fix imports in cloudformation module

* Delete unused code

* Add maybe_sleep fixtures to speed up recorded test runs

* Build basic placebo-CFN tests

* Commit placebo recordings of basic stack operations

* Add placebo to test-requires

* Allow unit tests to run regardless of environment by setting a default region

* Use explicit relative import for Python 3 compat

* Use __name__ attribute that works on Python 2 and 3
2017-03-31 15:31:12 -04:00
Matt Clay
4fdeade389 Overhaul ansible-test import analysis. (#22888)
- Add support for module_utils packages for better organization.
- Add support for "virtual" module_utils packages such as `six`.
2017-03-22 18:07:53 -07:00
Matt Clay
042e387f27 Fix metadata handling for shell command. (#22801) 2017-03-20 15:31:57 -07:00
Matt Clay
991bf0a1c3 Fix ansible-test base branch handling on merge. (#22759) 2017-03-17 23:33:32 -07:00
Matt Clay
47887a2edd Fix ansible-test base branch usage on Shippable. 2017-03-17 17:35:09 -07:00
Matt Clay
f7c83ccdc0 Fix ansible-test PR diff handling.
Permits use of PRs on branches other than devel.
2017-03-16 10:34:31 -07:00
Matt Clay
3a22467b38 Remove python 2.4 compile test. 2017-03-15 15:46:12 -07:00
Matt Clay
869449e288 Add test verification to ansible-test. (#22636)
* Add unified git diff parser.
* Add metadata and diff handling.
* Add test confidence/verification to bot output.
2017-03-15 12:17:42 -07:00
Matt Clay
5e9a2b8528 Verify SHA in Shippable change detection. 2017-03-15 11:31:55 -07:00
Matt Clay
89559f78de Add --failure-ok option to ansible-test. (#22623) 2017-03-14 16:32:21 -07:00
Matt Clay
771b14acb2 Sort test messages for deterministic output. (#22581) 2017-03-13 20:45:10 -07:00
Matt Clay
cfd313bced Disable ansible color output on sanity tests. 2017-03-08 12:18:16 -08:00
Matt Clay
3ab344e8bd Use lint/junit output for compile test. (#22395)
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
2017-03-08 00:47:21 -08:00
Matt Clay
fbf55886ba Use lint friendly output for python syntax checks. (#22390) 2017-03-07 20:27:50 -08:00
Matt Clay
8f463fcdd2 Create bot friendly sanity output. (#22381) 2017-03-07 14:59:50 -08:00
Matt Clay
3e4be156d7 Pre-compile regexes to speed up target processing.
Without this, changing a large number of files results in target
processing taking a very long time due to repeatedly compiling
the same patterns in a loop over many targets.
2017-03-06 12:35:12 -08:00
Matt Clay
d66ce40ecb Overhaul ansible-test sanity implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
2017-03-02 12:36:46 -08:00
Matt Clay
28b882e737 Support ansible commands in sanity tests. (#22172) 2017-03-01 15:48:14 -08:00
Matt Clay
4f3dade266 Add retries for ansible-test docker run. 2017-02-27 22:55:46 -08:00
Matt Clay
45cec93e99 Disable newer shellcheck tests which don't pass. 2017-02-27 22:31:47 -08:00
Matt Clay
a7bb04800d Add boto to units requirements for s3 test. 2017-02-27 20:55:32 -08:00
Matt Clay
fcac261eef Run unit tests in isolation w/ coverage support. 2017-02-27 17:16:45 -08:00
Matt Clay
c87ebb6215 Deduplicate arcs when combining coverage data. (#22022) 2017-02-27 15:46:15 -08:00
Matt Clay
2797cff773 Run destructive win tests when creating instances. 2017-02-27 10:36:53 -08:00
Matt Clay
2804d2e848 Split up unit tests on Shippable. (#21931)
* Add more verbose output to coverage combine.
* Run unit tests on CI separately by version.
2017-02-24 16:58:56 -08:00
Matt Clay
bc8186543a Pass base branch to module validator on delegate. 2017-02-20 14:28:11 -08:00
Matt Martz
9b1bd2eb7e Re-enable module comparisons in validate-modules (#21582)
* Re-enable module comparisons, specifically for new module detection and for finding new options/arguments
* Only do new module checks in shippable, local will display warning
2017-02-17 15:13:18 -08:00
Matt Clay
1ce2ea0b6c Fix ansible-test handling of deleted test targets. 2017-02-17 11:16:24 -08:00
Matt Clay
2192c1eb02 Fix ansible-test selection of inventory file. 2017-02-16 15:03:27 -08:00
John R Barker
5e28d3a15c network_cli shouldn't be specified anymore (#21458) 2017-02-15 12:58:12 +00:00
Matt Clay
d54bc09fae Consider module_utils deps when running tests. (#21382)
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
2017-02-13 18:49:36 -08:00