Commit graph

305 commits

Author SHA1 Message Date
Felix Fontein
d4c16f51be New acme_* integration test using ACME test docker container (#41626)
* Using ACME test container for acme_account integration test.

* Removing dependency on setup_openssl. Waiting for controller and Pebble.

* More tinkering.

* Reducing number of tries.

* One more try.

* Another try.

* Added acme_certificate tests.

* Removed double key.

* Added tests for acme_certificate_revoke.

* Making task names more meaningful (during certificate generation).

* Using newer test container which integrates letsencrypt/pebble#137. Adding test for revoking certificate by its private key.

* Using new version of Pebble which limits the random auth delay.

* Simplifying certificates for revocation tests.

* Reworking acme_certificate tests (there are now more, but they are faster).

* Test whether account_key_content works.

* Preparing TLS-ALPN-01 support.

* Using official Ansible image of testing container on quay.io.

* Bumping version.

* Bumping version of test container to 1.1.0.

* Adjusting to new CI group names.

* Pass ACME simulator IP as playbook variable.

* Let test plugin wait for controller and CA endpoints to become active.

* Refactor common setup parts of tests to setup_acme.

* _ -> dummy

* Moving common obtain-cert.yml to setup_acme.
2018-07-30 11:10:17 -07:00
Toshio Kuratomi
52449cc01a AnsiballZ improvements
Now that we don't need to worry about python-2.4 and 2.5, we can make
some improvements to the way AnsiballZ handles modules.

* Change AnsiballZ wrapper to use import to invoke the module
  We need the module to think of itself as a script because it could be
  coded as:

      main()

  or as:

      if __name__ == '__main__':
          main()

  Or even as:

      if __name__ == '__main__':
          random_function_name()

  A script will invoke all of those.  Prior to this change, we invoked
  a second Python interpreter on the module so that it really was
  a script.  However, this means that we have to run python twice (once
  for the AnsiballZ wrapper and once for the module).  This change makes
  the module think that it is a script (because __name__ in the module ==
  '__main__') but it's actually being invoked by us importing the module
  code.

  There's three ways we've come up to do this.
  * The most elegant is to use zipimporter and tell the import mechanism
    that the module being loaded is __main__:
    * 5959f11c9d/lib/ansible/executor/module_common.py (L175)
    * zipimporter is nice because we do not have to extract the module from
      the zip file and save it to the disk when we do that.  The import
      machinery does it all for us.
    * The drawback is that modules do not have a __file__ which points
      to a real file when they do this.  Modules could be using __file__
      to for a variety of reasons, most of those probably have
      replacements (the most common one is to find a writable directory
      for temporary files.  AnsibleModule.tmpdir should be used instead)
      We can monkeypatch __file__ in fom AnsibleModule initialization
      but that's kind of gross.  There's no way I can see to do this
      from the wrapper.

  * Next, there's imp.load_module():
    * https://github.com/abadger/ansible/blob/340edf7489/lib/ansible/executor/module_common.py#L151
    * imp has the nice property of allowing us to set __name__ to
      __main__ without changing the name of the file itself
    * We also don't have to do anything special to set __file__ for
      backwards compatibility (although the reason for that is the
      drawback):
    * Its drawback is that it requires the file to exist on disk so we
      have to explicitly extract it from the zipfile and save it to
      a temporary file

  * The last choice is to use exec to execute the module:
    * https://github.com/abadger/ansible/blob/f47a4ccc76/lib/ansible/executor/module_common.py#L175
    * The code we would have to maintain for this looks pretty clean.
      In the wrapper we create a ModuleType, set __file__ on it, read
      the module's contents in from the zip file and then exec it.
    * Drawbacks: We still have to explicitly extract the file's contents
      from the zip archive instead of letting python's import mechanism
      handle it.
    * Exec also has hidden performance issues and breaks certain
      assumptions that modules could be making about their own code:
      http://lucumr.pocoo.org/2011/2/1/exec-in-python/

  Our plan is to use imp.load_module() for now, deprecate the use of
  __file__ in modules, and switch to zipimport once the deprecation
  period for __file__ is over (without monkeypatching a fake __file__ in
  via AnsibleModule).

* Rename the name of the AnsiBallZ wrapped module
  This makes it obvious that the wrapped module isn't the module file that
  we distribute.  It's part of trying to mitigate the fact that the module
  is now named __main)).py in tracebacks.

* Shield all wrapper symbols inside of a function
  With the new import code, all symbols in the wrapper become visible in
  the module.  To mitigate the chance of collisions, move most symbols
  into a toplevel function.  The only symbols left in the global namespace
  are now _ANSIBALLZ_WRAPPER and _ansiballz_main.

revised porting guide entry

Integrate code coverage collection into AnsiballZ.

ci_coverage
ci_complete
2018-07-26 20:07:25 -07:00
Matt Clay
72f1a6cc1f Fix bug preventing coverage of setup module. 2018-07-25 19:11:48 -07:00
Matt Clay
4e489d1be8
Update Shippable integration test groups. (#43118)
* Update Shippable integration test groups.
* Update integration test group aliases.
* Rebalance AWS and Azure tests with extra group.
* Rebalance Windows tests with another group.
2018-07-23 20:46:22 -07:00
Sumit Jaiswal
8b61dd6efc
Updating the nios_test_container version to 1.1.0 from 1.0.0 (#42526)
* updating the nios_test_container version

* changing the version
2018-07-11 23:34:30 +05:30
Matt Clay
11d0eb04ce Temporarily switch RHEL and Windows tests to AWS. 2018-07-10 15:15:01 -07:00
Matt Clay
8e26a58443 Revert "Temporarily switch RHEL and Windows tests to AWS."
This reverts commit c3134ce6e2.
2018-07-10 10:47:25 -07:00
Matt Clay
c3134ce6e2 Temporarily switch RHEL and Windows tests to AWS. 2018-06-29 16:23:54 -07:00
Deric Crago
ba848e018c bump version of 'vcenter-test-container' to '1.3.0' (govmomi v0.18.0) (#41151)
* leave vmware_guest_powerstate tests enabled, but revert changes from: 66743f33
* leave VM poweroff tests enabled, but revert changes from: 87d6bdaf
* bumped 'vcenter-test-container' version to '1.3.0'
* updated test task names based on PR feedback
2018-06-06 08:24:21 -07:00
jctanner
1d5fe326e8 Add a cloud provider and a set of smoketests for the NIOS modules (#40573)
* Add a cloud provider and a set of smoketests for the NIOS modules
2018-05-25 13:50:45 +01:00
Matt Clay
dff662fa0f Add plugins to ansible-doc test and fix issues. 2018-05-23 13:10:09 -07:00
Matt Clay
15b6837daf Add yamllint for plugin docs and fix issues. 2018-05-23 09:19:30 -07:00
Matt Clay
8deced3e04
Fix shebangs and file modes and update tests. (#40563)
* Add execute bit sanity test and apply fixes.
* Add shebang test for `lib` dirs and apply fixes.
* Shebang and execute bit cleanup.
2018-05-22 14:25:36 -07:00
Ryan Brown
dda7d9e704
[AWS] Add Ansible Version to botocore user agent string (#39993)
Pull `config` out if it was provided to boto3_conn and add the useragent string
2018-05-17 11:41:14 -04:00
Rafael
44eaa2c007 opennebula: new module one_host (#40041) 2018-05-17 10:10:49 +02:00
Matt Clay
a7d7df1450 Make docs-build sanity test disabled by default. 2018-05-09 17:55:00 -07:00
Matt Clay
c1f9efabf4
Overhaul httptester support in ansible-test. (#39892)
- Works with the --remote option.
- Can be disabled with the --disable-httptester option.
- Change image with the --httptester option.
- Only load and run httptester for targets that require it.
2018-05-09 09:24:40 -07:00
Ryan Petrello
462449cc8c run Tower CI using the latest ansible-tower-cli package 2018-04-26 08:01:03 -07:00
Matt Clay
aac3020770 Add changelogs dir to change classification. 2018-04-25 16:27:29 -07:00
Matt Clay
ac1fbbeabc Update the cloudstack test container reference. 2018-04-25 10:41:24 -07:00
Sviatoslav Sydorenko
5ea1ee47dd Refactor Foreman provider to use simplified img 2018-04-20 00:36:23 +02:00
Sviatoslav Sydorenko
1664554b4a Unrandomize docker registry selection
stick with quay for now
2018-04-20 00:36:23 +02:00
Sviatoslav Sydorenko
d5b340cc43 Improve foreman image src selection 2018-04-20 00:36:23 +02:00
Sviatoslav Sydorenko
2608ef535f Add foreman cloud provider 2018-04-20 00:36:23 +02:00
Deric Crago
50d151aef2 Updated 'quay.io/ansible/vcenter-test-container' image tag to '1.2.0'
vcsim remains on commit 'dee49fa3694c5aff05e4b340b0686772f65c1fe1'
2018-04-17 17:57:43 -07:00
Matt Clay
1d5c933ecf Use new vcenter simulator container location. 2018-04-16 14:25:13 -07:00
Matt Martz
694d6b339c
Set memory-swap to memory (#38836) 2018-04-16 15:49:12 -05:00
Matt Clay
8a223009ca
Improve handling of integration test aliases. (#38698)
* Include change classification data in metadata.
* Add support for disabled tests.
* Add support for unstable tests.
* Add support for unsupported tests.
* Overhaul integration aliases sanity test.
* Update Shippable scripts to handle unstable tests.
* Mark unstable Azure tests.
* Mark unstable Windows tests.
* Mark disabled tests.
2018-04-12 16:15:28 -07:00
Matt Clay
c9fb054bc8 Fix get_cloud_platforms config usage. 2018-04-11 17:08:54 -07:00
Matt Clay
62957c9fc0 Support network action plugin classification. 2018-04-10 13:43:41 -07:00
Matt Clay
a5cbc0a2c8
Multiple ansible-test fixes. (#38247)
* Add ansible-test integration --allow-root option.
* Fix destructive target override.
* Fix bad type hint SanityResult -> TestResult.
* Fix skip/python3 filtering with --docker option.
2018-04-03 18:53:53 -07:00
Matt Martz
a19a21d715
Add --include and --omit options to ansible-test coverage report (#38061)
* Support --include and --omit with ansible-test coverage report

* Code format change
2018-03-28 18:42:57 -05:00
Matt Martz
5d90ebb28e
Add argument to allow limiting docker container to s specific amount of memory (#37950)
* Add argument to allow limiting docker container to s specific amount of memory

* Address review comments
2018-03-26 16:45:50 -05:00
Matt Clay
308b0a9772 Increase ansible-test scp retries. 2018-03-24 03:43:26 -07:00
Matt Clay
4e0ecfd553 Fix ansible-test handling of network plugins. 2018-03-22 15:16:27 -07:00
Matt Clay
ee596743d1 Improve ansible-test retries. 2018-03-21 22:52:27 -07:00
Matt Clay
05220d693d
Complete updates of remaining code-smell tests. (#37743)
* Add text/binary file support to code smell tests.
* Enhance line-endings code smell test.
* Enhance no-smart-quotes code-smell test.
* Enhance shebang code-smell test.
2018-03-21 12:02:06 -07:00
Matt Clay
6352e67ab2 Update ansible-test is_binary_file test.
Add hard-coded list of common text and binary extensions.
2018-03-21 11:05:59 -07:00
Matt Clay
248ca2df21 Fix encoding of code-smell paths on stdin. 2018-03-20 23:00:39 -07:00
Matt Clay
51e3882b80 Update tests triggered for bin/ changes. 2018-03-20 16:26:48 -07:00
Matt Clay
981e89117a Improve Tower integration test support:
- Add TOWER_VERSION environment variable.
- Add error check for missing configuration.
2018-03-15 12:25:58 -07:00
Matt Clay
a8487feb70 Fix ansible-test python and pip executable search. 2018-03-14 23:34:14 -07:00
Matt Clay
11ad559010
Terminate Tower instances after CI ends. (#37265)
* Remove obsolete Tower support from manage_ci.
* Add missing remote settings to cloud tests.
2018-03-09 16:17:29 -08:00
Matt Clay
5688d2243c Update support for Tower testing.
This is required for compatibility with the latest
version of ansible-core-ci, which now handles more
of the Tower instance setup.
2018-03-09 14:42:49 -08:00
Matt Clay
b4bf502268 Initial Tower module integration test support. 2018-03-07 14:21:55 -08:00
Matt Clay
b9b8081a87
Cleanup and enhancements for ansible-test. (#37142)
* Fix type hint typos.
* Add one-time cloud env setup after delegation.
* Add generate_password to util.
* Add username/password support to HttpClient.
* Avoid pip requirement for ansible-test shell.
* Support provisioning Tower instances.
2018-03-07 14:02:31 -08:00
Will Thames
a7371d4998 Having uppercase in the resource_prefix can cause unexpected issues
We may as well enforce lower case resource prefixes at source
2018-03-07 06:09:50 -08:00
Adam Miller
3fd5b0740e allow ANSIBLE_KEEP_REMOTE_FILES for local test runner (#33357)
* allow ANSIBLE_KEEP_REMOTE_FILES for local test runner
* add ANSIBLE_KEEP_REMOTE_FILES to tox.ini, update docs
* Clarify handling of environment variables.
2018-03-06 16:28:06 -08:00
Matt Clay
dc71c2197f
More code-smell sanity test updates. (#36830)
* Add test for missing Azure requirements.
* Improve readability.
* Enhance no-unicode-literals code-smell test.
2018-02-28 00:50:00 -08:00
Matt Clay
ac1698099d
Overhaul additional sanity tests. (#36803)
* Remove unnecessary sys.exit calls.
* Add files filtering for code-smell tests.
* Enhance test-constraints code-smell test.
* Simplify compile sanity test.
* Pass paths to importer on stdin.
* Pass paths to yamllinter on stdin.
* Add work-around for unicode path filtering.
* Enhance configure-remoting-ps1 code-smell test.
* Enhance integration-aliases code-smell test.
* Enhance azure-requirements code-smell test.
* Enhance no-illegal-filenames code-smell test.
2018-02-27 15:05:39 -08:00
Matt Clay
60a24bbdaa Pass code-smell paths on stdin. 2018-02-27 00:36:16 -08:00
Chris Houseknecht
53cfd70b7d
Adds k8s_raw, openshift_raw tests (#36228) 2018-02-23 10:13:09 -05:00
Matt Clay
2b6ac4561b
Add support for enhanced code-smell tests. (#36332)
* Add support for enhanced code-smell tests:

- Path selection handled by ansible-test.
- Optional path filtering based on extension.
- Optional path filtering based on prefixes.
- Optional lint friendly output.

* Enhance no-assert code-smell test.
* Enhance no-tests-as-filters code-smell test.
2018-02-20 13:37:23 -08:00
Matt Clay
a9b58b84d8 Fix path handling in validate-modules sanity test. 2018-02-20 11:02:42 -08:00
Matt Clay
78e900cd7f Fix ansible-test --redact option with delegation. 2018-02-19 21:58:02 -08:00
Matt Clay
3a62eb5e03 Add option to hide sensitive ansible-test output.
This option is enabled automatically on Shippable.
2018-02-19 16:35:31 -08:00
Matt Clay
5bc9bb9fbb Support ansible-test --truncate with delegation. 2018-02-19 11:07:43 -08:00
Matt Clay
d2150795ba Truncate some long messages sent to a TTY.
Can be overridden with the --truncate option.
2018-02-16 02:01:18 -08:00
David Newswanger
3f5caf659e added support for --testcase flag in ansible-test (#36134)
* added support for --testcase flag in ansible-test

* fixed command format

* added tab completion

* fixed sanity issues

* added documenation for --testcase

* don't autocomplete when multiple modules are selected
2018-02-14 15:40:35 -08:00
Matt Clay
032dc1a7c5
Initial OpenShift integration test support. (#36207)
Based on integration tests from chouseknecht for openshift_raw.
2018-02-14 13:39:42 -08:00
Matt Clay
6a4f6a6490 Add additional support code in ansible-test. 2018-02-14 12:16:41 -08:00
Matt Clay
2a0adaf542 Improve bot messaging on CI failures. 2018-02-07 23:42:08 -08:00
Matt Clay
f46f6c8dec
Temp install of setuptools for import coverage. (#35752) 2018-02-06 14:03:23 -08:00
Alex Stephen
9706abf685 [cloud] New GCP module: DNS Managed Zones (gcp_dns_managed_zone.py) (#35014) 2018-02-06 11:50:16 -05:00
Matt Clay
b9c5147be2
Move import sanity test files into own directory. (#35593) 2018-02-01 09:52:31 -08:00
Matt Clay
2d565d14ed Add pslint sanity test settings.
Globally ignore rule: PSUseShouldProcessForStateChangingFunctions
2018-01-31 06:31:05 -08:00
Matt Clay
3b0dcf7f29 Test Windows 2012+ using Azure by default. 2018-01-29 17:17:06 -08:00
Matt Clay
7abdab6c9e Convert ansible-test compile into a sanity test. 2018-01-25 09:45:36 -08:00
Matt Clay
e0010f15e4 Add pslint sanity test. (#35303)
* Add pslint sanity test.

* Fix `no-smart-quotes` sanity test.

* Add docs for `pslint` sanity test.
2018-01-24 17:22:14 -08:00
Matt Clay
8ea0bfe9a3
Miscellaneous test fixes. (#35301)
* Add missing pylint test for invalid path.
* Fix syntax in integration test.
* Use Write-Output in win_script test script.
* Fix pylint in explain mode.
2018-01-24 10:22:04 -08:00
Matt Clay
f9f6080630 Improve handling of ansible-doc sanity errors. 2018-01-17 11:41:01 -08:00
Matt Clay
74d635ba6b Add missing code to yamllint messages. 2018-01-17 07:34:51 -08:00
Pilou
217ff4498c ansible-config: add simple tests (#34900)
* Revert "Fix ansible-config with python3 (#34673)"

This reverts commit 2a9daaa45b.

* ansible-config: add simple tests

* Fix ansible-config with python3

* ansible-test: don't quote "unusual" characters
2018-01-17 06:33:33 -08:00
Matt Clay
5fa1edc15d
Track ansible-test cloud and target overhead. (#34902) 2018-01-16 15:52:42 -08:00
Matt Clay
227ff61f9d
Add module support to yamllint sanity test. (#34964)
* Add module support to yamllint sanity test.
* Fix duplicate keys in module RETURN docs.
* Fix syntax in return_common docs fragment.
* Fix duplicate keys in module EXAMPLES docs.
2018-01-16 15:08:56 -08:00
Matt Martz
dcc05093db Validate modules arg spec fixes (#34477)
* Update validate-modules arg_spec introspection to be faster, by only mocking the imports we explicitly list
* The use of types.MethodType in redhat_subscription wasn't py3 compatible, use partial instead
* Remove argument_spec import hacks, make them errors, we can ignore them with ansible-test
* Enable the --arg-spec flag for validate-modules
2018-01-11 15:41:53 -08:00
Matt Clay
2d4a43bb0a
Add file+rule ignore support to validate-modules. (#34757)
* Clean up code.
* Add file+rule ignore support to validate-modules.
2018-01-11 13:36:08 -08:00
Matt Clay
f968776d86 Remove ansible-test obsolete net_* target logic. 2018-01-10 12:12:39 -08:00
Matt Clay
82b5a6a0c9
Fix ansible-test network-integration command. (#34661)
* Fix ansible-test network platform init filter.
* Fix ansible-test network inventory generation.
* Remove ios/csr1000v from CI.
* Run network tests on Python 2.7 and 3.6.
2018-01-09 14:52:36 -08:00
Matt Clay
a9fe30e34b Add pylint plugin support to ansible-test. 2018-01-05 19:59:17 -08:00
Matt Clay
aff16225eb Add per file+rule pylint ignore to ansible-test. 2018-01-05 19:59:17 -08:00
Nathaniel Case
513c75079e
Port eos tests to network_cli (#33586)
* Add eos and fix tests to run multiple connections

* Update tests to report connection

* Add missing START messages

* Fix unspecified connection

* Python 3 updates

Exceptions don't have `.message` in Python 3

* Override `become` when using `connection=local`

* Slight restructuring to make eapi easier later on

* Move eapi toggle to prepare_eos
* Pull out connection on eapi tasks
2017-12-19 15:49:49 -05:00
Abhijeet Kasurde
dd9ed09fa6
Revert to stable vcsim docker image. (#33952)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-12-15 21:32:22 +05:30
Abhijeet Kasurde
29d3505cb4
VMware: check for ESXi server while creating user (#33061)
This fix check for ESXi server instance before proceeding
with managing local user. Also, adds integration tests for
this change.

Fixes: #32465

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2017-12-15 16:26:19 +05:30
Matt Clay
6f77a32b13 Run RHEL tests on Azure in 3 groups. 2017-12-13 23:21:05 -08:00
Matt Clay
ad4975d3e7 Update Azure module test infrastructure.
- Use new Azure direct API implementation.
- Enable Azure tests to clean up on exit.

ci_complete
2017-12-07 23:37:36 -08:00
Matt Clay
1b5c4b72bd Add Azure provider support to ansible-test and CI.
ci_complete
2017-12-06 00:34:54 -08:00
Nathaniel Case
8679d2e396
Port integration tests to network_cli (#33583)
* network_cli needs network_os

* Work around for Python 3.x < 3.6
2017-12-05 12:46:00 -05:00
Matt Clay
1ee511f82c Use an abspath for network inventory ssh key path. 2017-11-22 11:10:22 -08:00
Matt Clay
6f4731ef11 Force all tests to run on docker image updates. 2017-11-21 15:02:07 -08:00
Matt Clay
07bb7684b0 Add PS dependency analysis to ansible-test. 2017-11-21 12:30:16 -08:00
Matt Clay
f9cd50411f Enable Python 3.7 in ansible-test. 2017-11-21 09:24:04 -08:00
Matt Clay
677aca1cc7 Add Python 3.7-dev to the default docker image. 2017-11-20 18:09:58 -08:00
Matt Clay
ef21038dd5 Fix ansible-test race calling get_coverage_path. 2017-11-15 12:57:18 -08:00
Matt Clay
6472723ba8
Add missing ansible-test --remote-terminate support. (#32918)
* Expand ansible-test --remote-terminate support:

- windows-integration
- network-integration

These commands previously accepted the option, but did not support it.

* Terminate windows and network instances when done.
2017-11-14 17:08:48 -08:00
Matt Clay
cf1337ca9a Update ansible-test sanity command. (#31958)
* Use correct pip version in ansible-test.
* Add git fallback for validate-modules.
* Run sanity tests in a docker container.
* Use correct python version for sanity tests.
* Pin docker completion images and add default.
* Split pylint execution into multiple contexts.
* Only test .py files in use-argspec-type-path test.
* Accept identical python interpeter name or binary.
* Switch cloud tests to default container.
* Remove unused extras from pip install.
* Filter out empty pip commands.
* Don't force running of pip list.
* Support delegation for windows and network tests.
* Fix ansible-test python version usage.
* Fix ansible-test python version skipping.
* Use absolute path for log in ansible-test.
* Run vyos_command test on python 3.
* Fix windows/network instance persistence.
* Add `test/cache` dir to classification.
* Enable more python versions for network tests.
* Fix cs_router test.
2017-10-26 00:21:46 -07:00
Matt Clay
36b13e3e3d Fix ansible-test default image. (#31966)
* Add openssh-client to default docker container.
* Include Azure requirements in default container.

To do so, handling of pip requirements was updated to install each
set of requirements separately and then run a verification pass to
make sure there are no conflicts between requirements.

* Add missing --docker-no-pull option.
* Add documentation for the azure-requirements test.
2017-10-20 08:48:01 -07:00
Matt Clay
f76afab6e5 Add new default Docker container for ansible-test. (#31944)
* Add new default Docker container for ansible-test.
* Update ansible-test change classification.
* Update list of disabled pylint rules.
* Fix pylint issues with ansible-test.
2017-10-19 13:37:22 -07:00
jctanner
6bf0cc36a7 Bump the vcsim container version (#31861) 2017-10-18 08:40:23 -04:00
Matt Clay
6c3339402a Improve python 2/3 ABC fallback for pylint. (#31848)
* Improve python 2/3 ABC fallback for pylint.
* Allow longer method names in ansible-test.
2017-10-17 12:49:10 -07:00