Commit graph

299 commits

Author SHA1 Message Date
Jordan Borean
31ed283e33 ansible-test: swap 2.5 to use AWS and not Azure 2018-10-15 15:44:06 -07:00
Matt Clay
ca13e678ae Backport test infra fixes and updates to stable-2.5. (#46992)
* Fix unit test parametrize order on Python 3.5.

(cherry picked from commit 53b230ca74)

* Fix ansible-test unit test execution. (#45772)

* Fix ansible-test units requirements install.
* Run unit tests as unprivileged user under Docker.

(cherry picked from commit 379a7f4f5a)

* Run unit tests in parallel. (#45812)

(cherry picked from commit abe8e4c9e8)

* Minor fixes for unit test delegation.

(cherry picked from commit be199cfe90)

* add support for opening shell on remote Windows host (#43919)

* add support for opening shell on remote Windows host

* added arg completion and fix sanity check

* remove uneeded arg

(cherry picked from commit 6ca4ea0c1f)

* Block network access for unit tests in docker.

(cherry picked from commit 99cac99cbc)

* Make ansible-test available in the bin directory. (#45876)

(cherry picked from commit f3d1f9544b)

* Support comments in ansible-test flat files.

(cherry picked from commit 5a3000af19)

* Fix incorrect use of subprocess.CalledProcessError (#45890)

(cherry picked from commit 24dd87bd0a)

* Improve ansible-test match error handling.

(cherry picked from commit 2056c981ae)

* Improve error handling for docs-build test.

(cherry picked from commit 2148999048)

* Bug fixes and cleanup for ansible-test. (#45991)

* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.

(cherry picked from commit ac492476e5)

* Fix integration test library search path.

This prevents tests from loading modules outside the source tree,
which could result in testing the wrong module if a system-wide
install is present, or custom modules exist.

(cherry picked from commit d603cd41fe)

* Update default container to version 1.2.0.

(cherry picked from commit d478a4c3f6)
(cherry picked from commit 21c4eb8db5)

* Fix ansible-test docker python version handling.

This removes the old name based version detection behavior and
uses versions defined in the docker completion file instead, as
the new containers do not follow the old naming scheme.

(cherry picked from commit 54937ba784)

* Reduce noise in docs-build test failures.

(cherry picked from commit 4085d01617)

* Fix ansible-test encoding issues for exceptions.

(cherry picked from commit 0d7a156319)

* Fix ansible-test multi-group smoke test handling. (#46363)

* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.

(cherry picked from commit e53390b3b1)

* Use default-test-container version 1.3.0.

(cherry picked from commit 6d9be66418)

* Add file exists check in integration-aliases test.

(cherry picked from commit 33a8be9109)

* Improve ansible-test environment checking between tests. (#46459)

* Add unified diff output to environment validation.

This makes it easier to see where the environment changed.

* Compare Python interpreters by version to pip shebangs.

This helps expose cases where pip executables use a different
Python interpreter than is expected.

* Query `pip.__version__` instead of using `pip --version`.

This is a much faster way to query the pip version. It also more
closely matches how we invoke pip within ansible-test.

* Remove redundant environment scan between tests.

This reuses the environment scan from the end of the previous test
as the basis for comparison during the next test.

(cherry picked from commit 0dc7f38787)

* Add symlinks sanity test. (#46467)

* Add symlinks sanity test.
* Replace legacy test symlinks with actual content.
* Remove dir symlink from template_jinja2_latest.
* Update import test to use generated library dir.
* Fix copy test symlink setup.

(cherry picked from commit e2b6047514)

* Fix parametrize warning in unit tests.

(cherry picked from commit 1a28898a00)

* Update MANIFEST.in (#46502)

* Update MANIFEST.in:

- Remove unnecessary prune.
- Include files needed by tests.
- Exclude botmeta sanity test.

These changes permit sanity tests to pass on sdist output.
(cherry picked from commit cbb49f66ec)

* Fix unit tests which modify the source tree. (#45763)

* Fix CNOS unit test log usage.
* Use temp dir for Galaxy unit tests.
* Write to temp files in interfaces_file unit test.
* Fix log placement in netapp_e_ldap unit test.

(cherry picked from commit 0686450cae)

* Fix ansible-test custom docker image traceback.

(cherry picked from commit 712ad9ed64)

* ansible-test: Create public key creating Windows targets (#43760)

* ansible-test: Create public key creating Windows targets

* Changed to always set SSH Key for Windows hosts

(cherry picked from commit adc0efe10c)

* Fix and re-enable sts_assume_role integration tests (#46026)

* Fix the STS assume role error message assertion when the role to assume does not exist.

(cherry picked from commit 18dc928e28)

* Fix ACI unit test on Python 3.7.0.

The previous logic was only needed for pre-release versions of 3.7.

(cherry picked from commit c0bf9815c9)

* Remove placeboify from unit tests that are not calling AWS (i.e. creating a recording) (#45754)

(cherry picked from commit 2167ce6cb6)

* Update sanity test ignore entries.
2018-10-13 10:44:11 -07:00
Jordan Borean
f9c4da4cdf ansible-test: set ulimit to enforce consistent test environment (#46652)
* ansible-test: set ulimit to enforce consistent test environment

* fixed santiy issue

(cherry picked from commit 7b774117ab)
2018-10-09 18:56:02 -07:00
Jordan Borean
412b6a8ca7 append AWS region to test instance file (#45877) (#45884)
(cherry picked from commit 582a4dfa13)
2018-09-25 13:24:30 -07:00
Jordan Borean
c484a6efdd add skip/docker alias to skip tests when running on Docker (#45955) (#45959)
* add skip/docker alias to skip tests when running on Docker

* changed warning message wording

(cherry picked from commit 27c10fa502)
2018-09-25 09:14:31 -07:00
Jordan Borean
65fa6c17a7 go back to FreeBSD 11.1 for tests due to 11.2 stability issues (#45872) (#45873)
(cherry picked from commit a7d372c3ba)
2018-09-25 09:12:51 -07:00
Matt Clay
062113731a Update RHEL and FreeBSD versions used for tests:
- RHEL 7.4 -> 7.5
- FreeBSD 11.1 -> 11.2

(cherry picked from commit c50d6f1944)
2018-09-12 10:19:07 -07:00
Matt Clay
729736f30c Remove obsolete default container.
(cherry picked from commit 4173a30941)
2018-09-05 19:21:48 -07:00
Matt Clay
e7d75d4bff Switch default test container to quay.io.
(cherry picked from commit 3f8edb440a)
2018-09-05 19:21:48 -07:00
Matt Clay
0c3faa2e1c Fix ansible-test virtualenv use in import test.
(cherry picked from commit 998badbda5)
2018-09-05 19:21:48 -07:00
Matt Clay
9ac580f564 Update test constraints. (#44975)
- Limit virtualenv version on Python 2.6.
- Limit pyopenssl version on Python 2.6.
- Pin pyfmg version to keep unit tests passing.

(cherry picked from commit b3a3e8ba7b)
2018-09-05 19:21:48 -07:00
Matt Clay
fb153e4da7 Add seccomp support to ansible-test.
(cherry picked from commit c1173a2aec)
2018-09-05 19:21:48 -07:00
Zim Kalinowski
019c28bc3b Broken dependencies in 2.5 (#41952)
* essential fixes in 2.5

* one more fix

* disable try to see whats the problem

* fix indent

* try to fix msrest version

* fixed test

* undo try-except commenting out

* updated tests
2018-09-05 11:02:00 -07:00
Matt Clay
9959d7f722 Switch distro test containers to quay.io images.
(cherry picked from commit 773c0982b0)
2018-08-30 09:12:26 -07:00
Matt Clay
de932d9c17 Fix change detection on retried Shippable jobs.
(cherry picked from commit c0aaf0fff7)
2018-08-30 09:12:26 -07:00
Matt Clay
bf8f228fd5 Update Shippable integration test groups. (#43118)
* Update Shippable integration test groups.
* Update integration test group aliases.
* Rebalance AWS and Azure tests with extra group.
* Rebalance Windows tests with another group.

(cherry picked from commit 4e489d1be8)
2018-07-24 10:02:00 -07:00
Jordan Borean
60ab7b2807 winrm: fix up unit tests (#41112) (#41211)
(cherry picked from commit ad8e13e9f8)
2018-06-06 15:57:00 -07:00
Matt Clay
48f6670b49 Fix FreeBSD package install in test setup.sh.
(cherry picked from commit c3d3b6cedc)
2018-05-30 09:26:04 -07:00
John R Barker
4924398e0f
Network requirements to be installed by ansible-test (#40042) (#40250)
Previously the test framework (DCI, Zuul) were installing the various
dependencies, this meant the list of what was required was duplicated.

Having everything defined in ansible-test makes it easier for people to
run tests locally.

Also this allows the test to work correctly on Python 2 & Python 3
(cherry picked from commit 27942d937f)
2018-05-16 15:25:44 +01:00
Matt Clay
1ecfca84e6 Make docs-build sanity test disabled by default.
(cherry picked from commit a7d7df1450)
2018-05-09 21:55:11 -07:00
Matt Clay
1fd024984b Overhaul httptester support in ansible-test. (#39892)
- Works with the --remote option.
- Can be disabled with the --disable-httptester option.
- Change image with the --httptester option.
- Only load and run httptester for targets that require it.

(cherry picked from commit c1f9efabf4)
2018-05-09 16:55:25 -07:00
Matt Clay
df72a1d644 Move pyvmomi install into cloud test plugin.
(cherry picked from commit a5b8085113)
2018-05-08 12:27:37 -07:00
Matt Clay
784956c219
Sync stable-2.5 test infrastructure with devel. (#39544)
Sync stable-2.5 test infrastructure with devel:

- shippable.yml
- test/runner/
- test/sanity/
- test/utils/
- docs/docsite/rst/dev_guide/testing/sanity/
- test/integration/targets/*/aliases
2018-05-01 15:47:24 -07:00
Matt Clay
20ab1d2994 Move requirements into tests. (#35885)
(cherry picked from commit e7b793c8c6)
2018-05-01 09:55:56 -07:00
Matt Clay
75c5bc0ef7 Update the cloudstack test container reference.
(cherry picked from commit ac1fbbeabc)
2018-04-25 12:32:23 -07:00
Matt Clay
6c03fcdc0e Fix mysql-server install/upgrade on Ubuntu 16.04. (#39241)
* Fix mysql-server install/upgrade on Ubuntu 16.04.
* Prevent service restart in docker_secret test.

(cherry picked from commit 996f9c2467)
2018-04-24 13:37:25 -07:00
Matt Clay
8606fa0431 Update httptester container reference.
(cherry picked from commit 35748e5228)
2018-04-17 21:26:02 -07:00
Matt Clay
ce1913a76f Use new vcenter simulator container location.
(cherry picked from commit 1d5c933ecf)
2018-04-16 15:17:08 -07:00
Matt Clay
6b0b4ddf00 Pin httptester SHA.
(cherry picked from commit 8232c95f42)
2018-03-21 10:02:11 -07:00
Matt Clay
8db5d139a1 Fix path handling in validate-modules sanity test.
(cherry picked from commit a9b58b84d8)
2018-02-20 13:57:58 -08:00
Matt Davis
62139bb828 v2.5.0b2 release cut 2018-02-16 01:13:00 -08:00
David Newswanger
078467d18d added support for --testcase flag in ansible-test (#36134)
* added support for --testcase flag in ansible-test

* fixed command format

* added tab completion

* fixed sanity issues

* added documenation for --testcase

* don't autocomplete when multiple modules are selected

(cherry picked from commit 3f5caf659e)
2018-02-14 15:41:43 -08:00
Matt Davis
50e79c5a14 initial cutover to API profiles (#35538)
* hardcoded API profiles in azure_rm_common
* changed azure_rm_securitygroup module to use api_profiles, dynamic models, kwargs on all SDK methods
* changed azure_rm_containerinstance module to use api_profiles, dynamic models, kwargs on all SDK methods
* fixed polling performance issue in azure_rm_securitygroup (default poll interval was 30s)
(cherry picked from commit 26f551d1c3)
2018-02-09 14:27:30 -08:00
Matt Clay
2a0adaf542 Improve bot messaging on CI failures. 2018-02-07 23:42:08 -08:00
Matt Clay
f46f6c8dec
Temp install of setuptools for import coverage. (#35752) 2018-02-06 14:03:23 -08:00
Alex Stephen
9706abf685 [cloud] New GCP module: DNS Managed Zones (gcp_dns_managed_zone.py) (#35014) 2018-02-06 11:50:16 -05:00
Matt Clay
b9c5147be2
Move import sanity test files into own directory. (#35593) 2018-02-01 09:52:31 -08:00
Matt Clay
2d565d14ed Add pslint sanity test settings.
Globally ignore rule: PSUseShouldProcessForStateChangingFunctions
2018-01-31 06:31:05 -08:00
Matt Clay
3b0dcf7f29 Test Windows 2012+ using Azure by default. 2018-01-29 17:17:06 -08:00
Ghilli3
60d2a8b6a1 Fortinet FortiManager Scripting Module (#34518)
* Adding new code for new module and new module_utils with the pip pyfmg package

* Changed login and logout functionality and renamed mod_utils file as well as cleaned up PEP8 syntax

* Removing extra Ansible parameters and fixing more syntax issues

* Fixing more syntax issues and comparing against previous FTNT script module

* Changing import location to pass syntax checks

* Fixing pylint errors

* Removing test files

* Add unit tests and enabling a login session check within main in order to throw error if network connection exception

* Fixing syntax issues for adding unit tests

* Changing case for pip package requirements

* adding comments

* adding version restriction on pip package for testing

* adding version restriction on pip package for testing

* More comments

* Fixing documentation errors and add the ability to skip a test if it isn't present

* Fixing Pep8 error with whitespace (tab) in the row
2018-01-26 18:40:12 +00:00
Matt Clay
7abdab6c9e Convert ansible-test compile into a sanity test. 2018-01-25 09:45:36 -08:00
Matt Clay
e0010f15e4 Add pslint sanity test. (#35303)
* Add pslint sanity test.

* Fix `no-smart-quotes` sanity test.

* Add docs for `pslint` sanity test.
2018-01-24 17:22:14 -08:00
Matt Clay
8ea0bfe9a3
Miscellaneous test fixes. (#35301)
* Add missing pylint test for invalid path.
* Fix syntax in integration test.
* Use Write-Output in win_script test script.
* Fix pylint in explain mode.
2018-01-24 10:22:04 -08:00
Matt Clay
cc65636a10 Add PSScriptAnalyzer to default container. 2018-01-24 08:36:28 -08:00
Matt Clay
69132007c0 Revert "Move requirements into tests. (#35197)"
This reverts commit 4f6017dc16.
2018-01-23 10:54:25 -08:00
Matt Clay
4f6017dc16
Move requirements into tests. (#35197) 2018-01-23 10:08:16 -08:00
Matt Clay
20824b8f3d Simplify Windows versions in ansible-test. 2018-01-17 16:16:15 -08:00
Matt Clay
f9f6080630 Improve handling of ansible-doc sanity errors. 2018-01-17 11:41:01 -08:00
Matt Clay
74d635ba6b Add missing code to yamllint messages. 2018-01-17 07:34:51 -08:00
Pilou
217ff4498c ansible-config: add simple tests (#34900)
* Revert "Fix ansible-config with python3 (#34673)"

This reverts commit 2a9daaa45b.

* ansible-config: add simple tests

* Fix ansible-config with python3

* ansible-test: don't quote "unusual" characters
2018-01-17 06:33:33 -08:00