Commit graph

32 commits

Author SHA1 Message Date
Matt Clay
869449e288 Add test verification to ansible-test. (#22636)
* Add unified git diff parser.
* Add metadata and diff handling.
* Add test confidence/verification to bot output.
2017-03-15 12:17:42 -07:00
Matt Clay
89559f78de Add --failure-ok option to ansible-test. (#22623) 2017-03-14 16:32:21 -07:00
Matt Clay
3ab344e8bd Use lint/junit output for compile test. (#22395)
* Refactor sanity classes for use in all tests.
* Use lint/junit output for compile test.
* Add missing options for compile test.
* Fix early bailout on requirements install.
2017-03-08 00:47:21 -08:00
Matt Clay
fbf55886ba Use lint friendly output for python syntax checks. (#22390) 2017-03-07 20:27:50 -08:00
Matt Clay
d66ce40ecb Overhaul ansible-test sanity implementation. (#22177)
- Tests are run to completion instead of stopping on first failure.
- Test results are now parsed instead of passing through to the console.
- Test results can be saved in junit xml format.
- Test results will show up on the Shippable "Tests" result tab.
- Added an experimental --lint option for easier integration with other tools.
- Code smell tests are now usable with the --list-tests, --test and --skip-test options.
- Code split out from executor.py into sanity.py.
- Rename download-logs to download.py and add support for test and coverage results.
- Miscellaneous improvements.
2017-03-02 12:36:46 -08:00
Matt Clay
28b882e737 Support ansible commands in sanity tests. (#22172) 2017-03-01 15:48:14 -08:00
Matt Clay
45cec93e99 Disable newer shellcheck tests which don't pass. 2017-02-27 22:31:47 -08:00
Matt Clay
fcac261eef Run unit tests in isolation w/ coverage support. 2017-02-27 17:16:45 -08:00
Matt Clay
2797cff773 Run destructive win tests when creating instances. 2017-02-27 10:36:53 -08:00
Matt Clay
bc8186543a Pass base branch to module validator on delegate. 2017-02-20 14:28:11 -08:00
Matt Martz
9b1bd2eb7e Re-enable module comparisons in validate-modules (#21582)
* Re-enable module comparisons, specifically for new module detection and for finding new options/arguments
* Only do new module checks in shippable, local will display warning
2017-02-17 15:13:18 -08:00
Matt Clay
2192c1eb02 Fix ansible-test selection of inventory file. 2017-02-16 15:03:27 -08:00
John R Barker
5e28d3a15c network_cli shouldn't be specified anymore (#21458) 2017-02-15 12:58:12 +00:00
Matt Clay
d54bc09fae Consider module_utils deps when running tests. (#21382)
* Skip pep8 analysis when --explain is used.
* Fix return type annotations.
* Match line length requirement of PEP 8 config.
* Consider module_utils deps when running tests.
2017-02-13 18:49:36 -08:00
Matt Clay
4189592701 Revert "Enable process isolation for unit tests."
This reverts commit 91526cd9f2.

Removing this feature primarily because it interferes with
collecting proper code coverage results. I may restore the
feature later if that can be resolved.
2017-02-09 11:10:06 -08:00
Matt Clay
91526cd9f2 Enable process isolation for unit tests. 2017-02-08 14:36:42 -08:00
Matt Clay
6ef1a6aeb6 Add pep8 to ansible-test. (#20745)
The rule sets and legacy file list are a first draft.
It is likely that they will need to be revised.
2017-01-27 12:34:00 -08:00
Matt Clay
fa7f84217b Add startup checks to ansible-test. 2017-01-24 13:08:04 -08:00
Matt Clay
61e6e7868c Improve jinja2 test coverage. (#20533)
* Run `template` test on latest jinja2 version.
* Update jinja2 install for groupby_filter.
* Clean test output dir for each test.
2017-01-20 16:38:52 -08:00
Matt Clay
4ba9f63afe Use multiple AWS regions in ansible-test. (#20429)
* Move ansible-test EnvironmentConfig to util.
* Use multiple AWS regions in ansible-test.
2017-01-18 16:31:34 -08:00
Matt Clay
4673a9c9c2 Generate ssh key once for network tests. 2017-01-18 13:33:12 -08:00
Matt Clay
3c51ce56fd Enable more network support in ansible-test. (#20372)
* Only start platform instances with tests selected.
* Enable ios on Shippable.
* Show inventory in explain mode.
* Fix indentation of generated network inventory.
* Update classification of network module_utils.
2017-01-17 16:24:05 -08:00
Matt Clay
e2bd71ac02 Clean up inventory generation in ansible-test. (#20313) 2017-01-16 12:31:17 -08:00
Matt Clay
d8733a5455 Complete initial network-integration support. 2017-01-12 12:39:00 -08:00
Matt Clay
f340b8d14f Add test support for python 3.6. 2017-01-10 14:34:42 -08:00
Matt Clay
dac226de4a Improve ansible-test completion and messaging. (#19627)
* Simplify type annotation of args.docker_util.
* Improve error messaging for ansible-doc test.
* Add completion for docker, remote and windows.
2016-12-22 09:57:17 -08:00
Matt Clay
422857166b Remove legacy test alias migration script. (#19629)
* Require code-smell tests to be files.
* Add sanity check for integration test aliases.
* Remove migration script for test aliases.
2016-12-21 16:46:28 -08:00
Matt Clay
9b5c782a0b Use docker pull by default in ansible-test. 2016-12-14 20:11:43 -08:00
Matt Clay
f28b5a0ed8 Add --tox-sitepackages option to ansible-test. 2016-12-13 16:57:23 -08:00
Matt Clay
47083f145a Don't error if pytest skips all unit tests. 2016-12-08 11:35:20 -05:00
Matt Clay
bee58fe890 Fix compile skip path handling for targeted tests. 2016-11-30 09:47:56 -08:00
Matt Clay
6bbd92e422 Initial ansible-test implementation. (#18556) 2016-11-29 21:21:53 -08:00