Commit graph

824 commits

Author SHA1 Message Date
Matt Davis
f86345f777
Collection content loading (#52194)
* basic plugin loading working (with many hacks)

* task collections working

* play/block-level collection module/action working

* implement PEP302 loader

* implicit package support (no need for __init.py__ in collections)
* provides future options for secure loading of content that shouldn't execute inside controller (eg, actively ignore __init__.py on content/module paths)
* provide hook for synthetic collection setup (eg ansible.core pseudo-collection for specifying built-in plugins without legacy path, etc)

* synthetic package support

* ansible.core.plugins mapping works, others don't

* synthetic collections working for modules/actions

* fix direct-load legacy

* change base package name to ansible_collections

* note

* collection role loading

* expand paths from installed content root vars

* feature complete?

* rename ansible.core to ansible.builtin

* and various sanity fixes

* sanity tweaks

* unittest fixes

* less grabby error handler on has_plugin

* probably need to replace with a or harden callers

* fix win_ping test

* disable module test with explicit file extension; might be able to support in some scenarios, but can't see any other tests that verify that behavior...

* fix unicode conversion issues on py2

* attempt to keep things working-ish on py2.6

* python2.6 test fun round 2

* rename dirs/configs to "collections"

* add wrapper dir for content-adjacent

* fix pythoncheck to use localhost

* unicode tweaks, native/bytes string prefixing

* rename COLLECTION_PATHS to COLLECTIONS_PATHS

* switch to pathspec

* path handling cleanup

* change expensive `all` back to or chain

* unused import cleanup

* quotes tweak

* use wrapped iter/len in Jinja proxy

* var name expansion

* comment seemingly overcomplicated playbook_paths resolution

* drop unnecessary conditional nesting

* eliminate extraneous local

* zap superfluous validation function

* use slice for rolespec NS assembly

* misc naming/unicode fixes

* collection callback loader asks if valid FQ name instead of just '.'
* switch collection role resolution behavior to be internally `text` as much as possible

* misc fixmes

* to_native in exception constructor
* (slightly) detangle tuple accumulation mess in module_utils __init__ walker

* more misc fixmes

* tighten up action dispatch, add unqualified action test

* rename Collection mixin to CollectionSearch

* (attempt to) avoid potential confusion/conflict with builtin collections, etc

* stale fixmes

* tighten up pluginloader collections determination

* sanity test fixes

* ditch regex escape

* clarify comment

* update default collections paths config entry

* use PATH format instead of list

* skip integration tests on Python 2.6

ci_complete
2019-03-28 10:41:39 -07:00
Sam Doran
023c5167fd Revert "Use locking for concurrent file access (#52567)" (#54547)
This reverts commit e152b277cf.
2019-03-28 13:19:49 -04:00
James Cassell
bc4ef99533 standardize TLS connection properties (#54315)
* openstack: standardize tls params

* tower: tower_verify_ssl->validate_certs

* docker: use standard tls config params

- cacert_path -> ca_cert
- cert_path -> client_cert
- key_path -> client_key
- tls_verify -> validate_certs

* k8s: standardize tls connection params

- verify_ssl -> validate_certs
- ssl_ca_cert -> ca_cert
- cert_file -> client_cert
- key_file -> client_key

* ingate: verify_ssl -> validate_certs

* manageiq: standardize tls params

- verify_ssl -> validate_certs
- ca_bundle_path -> ca_cert

* mysql: standardize tls params

- ssl_ca -> ca_cert
- ssl_cert -> client_cert
- ssl_key -> client_key

* nios: ssl_verify -> validate_certs

* postgresql: ssl_rootcert -> ca_cert

* rabbitmq: standardize tls params

- cacert -> ca_cert
- cert -> client_cert
- key -> client_key

* rackspace: verify_ssl -> validate_certs

* vca: verify_certs -> validate_certs

* kubevirt_cdi_upload: upload_host_verify_ssl -> upload_host_validate_certs

* lxd: standardize tls params

- key_file -> client_key
- cert_file -> client_cert

* get_certificate: ca_certs -> ca_cert

* get_certificate.py: clarify one or more certs in a file

Co-Authored-By: jamescassell <code@james.cassell.me>

* zabbix: tls_issuer -> ca_cert

* bigip_device_auth_ldap: standardize tls params

- ssl_check_peer -> validate_certs
- ssl_client_cert -> client_cert
- ssl_client_key -> client_key
- ssl_ca_cert -> ca_cert

* vdirect: vdirect_validate_certs -> validate_certs

* mqtt: standardize tls params

- ca_certs -> ca_cert
- certfile -> client_cert
- keyfile -> client_key

* pulp_repo: standardize tls params

remove `importer_ssl` prefix

* rhn_register: sslcacert -> ca_cert

* yum_repository: standardize tls params

The fix for yum_repository is not straightforward since this module is
only a thin wrapper for the underlying commands and config.  In this
case, we add the new values as aliases, keeping the old as primary,
only due to the internal structure of the module.

Aliases added:
- sslcacert -> ca_cert
- sslclientcert -> client_cert
- sslclientkey -> client_key
- sslverify -> validate_certs

* gitlab_hook: enable_ssl_verification -> hook_validate_certs

* Adjust arguments for docker_swarm inventory plugin.

* foreman callback: standardize tls params

- ssl_cert -> client_cert
- ssl_key -> client_key

* grafana_annotations: validate_grafana_certs -> validate_certs

* nrdp callback: validate_nrdp_certs -> validate_certs

* kubectl connection: standardize tls params

- kubectl_cert_file -> client_cert
- kubectl_key_file -> client_key
- kubectl_ssl_ca_cert -> ca_cert
- kubectl_verify_ssl -> validate_certs

* oc connection: standardize tls params

- oc_cert_file -> client_cert
- oc_key_file -> client_key
- oc_ssl_ca_cert -> ca_cert
- oc_verify_ssl -> validate_certs

* psrp connection: cert_trust_path -> ca_cert

TODO: cert_validation -> validate_certs (multi-valued vs bool)

* k8s inventory: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* openshift inventory: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* tower inventory: verify_ssl -> validate_certs

* hashi_vault lookup: cacert -> ca_cert

* k8s lookup: standardize tls params

- cert_file -> client_cert
- key_file -> client_key
- ca_cert -> ca_cert
- verify_ssl -> validate_certs

* laps_passord lookup: cacert_file -> ca_cert

* changelog for TLS parameter standardization
2019-03-28 00:19:28 -05:00
Slava Maslennikov
85d836171b Add a force_replace_host flag to win_domain_membership (#53542)
* Add a force_replace_host flag to win_domain_membership

Satisfies https://github.com/ansible/ansible/issues/53539

* Rework backticks

* Bump version_added

* Check for existence of current hostname as well; use LDAPFilter during search

* Rename $force_replace_host to $allow_existing_computer_account

* Added docs, porting guide and minor nit in code
2019-03-28 14:36:26 +10:00
Simon Westphahl
a44dfed570 Catch all request timeouts for winrm connection (#54104)
* Catch all request timeouts for winrm connection

The current implementation only catches 'ConnectTimeout' exceptions.
Instead we should catch 'Timout' which also catches ReadTimeout
exceptions.

Improves on: #51744

Co-Authored-By: westphahl <westphahl@gmail.com>

* Changelog for winrm error handling improvement
2019-03-28 13:44:49 +10:00
Jordan Borean
e2d2874d81
win_user_right - fix non json output issue (#54495) 2019-03-28 12:30:20 +10:00
Dag Wieers
e152b277cf Use locking for concurrent file access (#52567)
* Use locking for concurrent file access

This implements locking to be used for modules that are used for
concurrent file access, like lineinfile or known_hosts.

* Reinstate lock_timeout

This commit includes:
- New file locking infrastructure for modules
- Enable timeout tests
- Madifications to support concurrency with lineinfile

* Rebase, update changelog and tests

We need to specify ansible_python_interpreter to avoid running interpreter discovery and selecting the incorrect interpreter.

Remove the import of lock in known_hosts since it is not used.
2019-03-27 20:20:18 -04:00
Brian Coca
fee4a0df94
Handle errors correctly in rhn_channel (#54194)
* Handle errors correctly in rhn_channel

* also fail if required info is not available
2019-03-27 19:43:28 -04:00
Brian Coca
3637ce4538
Handle gracefully a bad module 'results' key (#53973)
* Handle gracefully a bad module 'results' key

  fixes #53962
2019-03-27 18:41:32 -04:00
Adam Miller
75d733afd2 Bugfix/52688 gcp compute missing image (#54468)
* adding (optionally) image information to inventory var
* add boot image mapping to gcp_compute instance data for all disk
image data in the configured zones

Signed-off-by: Adam Miller <admiller@redhat.com>
2019-03-27 17:14:20 -04:00
Brian Coca
f9876f3450 fixing gcp inv plugin (#54426)
* start fixing gcp inv plugin

* minor fixes

* added clog

* ajust comments

* link indv zone/project

* separate specific zone/project from other params

* restoring zones query per project

* also work when zones given

* fixed scopes, removed incorrect docs as not option
2019-03-27 11:13:39 -05:00
Kevin Breit
17fc6c6ff1 meraki_network - Parameter change for combined network type (#49160)
* Added support for types parameter
- Parameter is used to specify multiple network types

* Fix documentation

* Apply suggestions from code review

Co-Authored-By: kbreit <kevin.breit@kevinbreit.net>

* Reworked type parameter to be a list so types isn't needed

* Re-add tags documentation

* Fix documentation around compatibility

* Convert tags to list from string

* Add changelog fragment
2019-03-27 16:10:15 +01:00
James Cassell
92139a01df rename dellemc_idrac_firmware -> idrac_firmware (#54421) 2019-03-26 17:47:07 -04:00
James Cassell
51f2065df5 use ANSIBLE_ prefix on all ansible-defined env vars (#54272)
mark old version deprecated in all places referenced via docs
2019-03-26 17:43:48 -04:00
James Cassell
9040fa0235 openstack: add os module_defaults group (#54319) 2019-03-26 15:37:06 -04:00
Alan Rominger
461737e2da Allow using Azure rm legacy hostnames (#54060) 2019-03-26 15:28:21 -04:00
Anton Roman
1e3428a766 fix issue in random_mac filter with short prefixes (#53928) 2019-03-26 13:49:23 -04:00
Matt Martz
6996926d89
Expose loop_control.loop_var as ansible_loop_var (#54240)
* Expose the loop_var as ansible_loop_var

* Add docs

* fix assert test

* Indicate version added
2019-03-26 11:07:54 -05:00
Evan Kaufman
cf69ec5db0 replace - fixed combined before and after usage (#31452)
When using before and after in combination, the opposite behavior was induced. This PR makes the the replacement happen between the specified patterns as intended.

* Added integration tests
* Add changelog, porting guide entry, and minor doc fixes
2019-03-26 11:49:58 -04:00
Andrea Tartaglia
df86b9ec3d openssl_pkcs12: privatekey_path and friendly_name are not always required together (#54370)
* Removed required_together, updated tests

Since required_together: privatekey_path -> friendly_name, is not always
required it has been removed.
Updated openssl_pkcs12 integration tests to be in line with other
openssl_* modules, and added a test for export with no privatekey_path.

* linter fixes

* Removed cryptography from tests

* Added changelog fragment

* Removed non-necessary select_crypto_backend
2019-03-26 15:06:00 +00:00
Hannes Ljungberg
be293fbe50 docker_swarm_service: Remove configs and secrets defaults (#54361)
* Remove defaults

* Skip redundant casting

* Indentation fix

* Use generic compare functions

* Add tests for compare functions

* Remove extra whitespace

* Add changelog fragment
2019-03-26 09:45:40 -04:00
Sloane Hertel
54be769e8d fix AWS plugin credential precedence for environment variables (#52945)
* fix AWS plugin credential precedence for environment variables

* Allow aliases in direct plugins options

Consolidate precedence fix just in the doc fragment using aliases for mismatched options

* Access options with the option name rather than alias

* fix indentation

* update unit tests

* Improve readability
2019-03-25 16:53:11 -04:00
Felix Fontein
4fac91bed5 docker_container: add networks_cli_compatible option (#54198)
* Adding networks_cli_compatible option.

* Move network tests into own test file.

* Extend tests (for networks_cli_compatible=no).

* Adding tests for networks_cli_compatible=yes.

* There seems to be no way to create a container without at least one network attached.

* Integrate networks / purge_networks with comparisons.

* Speed up tests.

* Removing double dot.

* Add changelog.

* Use comparisons value only if the networks option has been specified. purge_networks on the other hand also removes networks if it has not been specified.
2019-03-25 15:06:21 -04:00
Felix Fontein
121990d8c5 openssl_dhparam: fix state=absent (#54296)
* Fix remove.

* Add changelog.
2019-03-25 12:00:20 -05:00
Abhijeet Kasurde
5db7501ebd
VMware: Change in update tag API (#54304)
Changed update tag API as per new changes.

Fixes: #53060

Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-03-25 21:50:46 +05:30
Felix Fontein
d7a273273a openssl_*: proper mode support (#54085)
* Add write helper.

* Adjust modules (except openssl_certificate).

* Adding tests for mode (with openssl_privatekey).

* Add openssl_certificate support.

* Never, ever remove the output file before actually trying to generate new content for it.

Removal is only allowed when state=absent, or when the object has been regenerated and the result needs to be written to that place.

* Add changelog.

* Extend test.
2019-03-25 14:20:52 +01:00
Felix Fontein
534c833bb3 openssl_certificate: fix state=absent (#54298)
* Fix state=absent.

* Add changelog.
2019-03-25 13:07:28 +01:00
Felix Fontein
752db43b2d Rename return of docker_compose. (#54171) 2019-03-23 18:35:11 +00:00
Martin Krizek
bafa291af9 uri: do not write the file after failure (#53515)
* uri: do not write the file after failure

Fixes #53491

* Add changelog
2019-03-22 09:32:43 -05:00
Martin Krizek
63ea96d5f1 assemble: avoid extra newline on Python 3 (#54176)
Fixes #44739
2019-03-22 09:14:02 -05:00
Andrea Tartaglia
36a790dcde New cryptography backend for openssl_certificate (#53924)
* New cryptography backend for openssl_certificate

load_* functions in module_utils/crypto.py now have a backend paramter
which when set to 'cryptography' will return cryptography objects so
they can be used for both pyopenssl and cryptography backends.
Added a select_message_digest function too returning a cryptography
digest hash from `cryptography.hazmat.primitives.hashes`
Added new classes for Cryptography backend

* Run test with various backends.

* Prefixing tests.

* Make sure we have the correct backend available.

* Linting (flake8).

* Moved cryptography import to separate try/except

* Make sure certificate is actually valid at some time in the past.

* Improve error handling.

* Trying to fix validation for cryptography backend.

* Fixed issue with keyUsage test in assertonly

* Fixed CI/Lint issues

* Fix private key problem for OwnCA.

* Cryptography backend doesn't support v2 certs.

* issue an expired cert with command when using cryptography backend

* Added warning when backend is auto and v2 cert is requested

* Bumped min cryptography version to  1.6

* Correctly check for failure when backend is cryptography and cert is v2

* Use self.backend where possible

* Use secp521r1 EC when testing on CentOS6

* Fixed pylint issue

* AcmeCertificate support for both backends

* Review fixes

* Fixed missing '(' when raising error

* Fixed date_fmt loop

* Updated docs and requirements with cryptography

* Add openssl_certificate to changelog.
2019-03-22 13:21:23 +00:00
Christian Rohmann
90c092a104 Allow configuration of connection_limit per postgresql database (postgresql_db) (#40345)
Fixes #40060

* Fix coding style errors
* Use CONNECTION LIMIT (no underscore)
* From review done by amenonsen and bcoca - Set default at None, make the change detection less confusing
* Added EXAMPLE on how to apply a database specific connection limit
* Added some basic tests for conn_limit applied to a database
* Check that conn_limit has actually been set / updated to 200
* Add changelog fragment regarding postgresql_db conn_limit parameter
2019-03-22 18:21:39 +05:30
Felix Fontein
1a94cf140c openssl_publickey: fix handling of OpenSSH private keys with passphrase (#54192)
* Cleanup.
* Make sure that OpenSSH passphrases are handled correctly.
* Add changelog.
2019-03-22 16:15:55 +05:30
David Passante
aa32164d15 cs_volume: add volumes extraction and upload features (#54111)
* cs_volume: add volumes extraction and upload features

* cs_volume: Update doc, remove deprecated code

* cs_volume: Add unit tests for extract and upload features
2019-03-22 07:09:26 +01:00
Alexander Bethke
f9c7ccbb41 Fixes flatpak module to work with flatpak >=1.2.0, fixes #51485 (#51482)
* Fixes flatpak module to work with flatpak >=1.2.0, fixes #51481

This keeps backwards-compatibility for flatpak versions before 1.2.0

* Fixes typeo

Co-Authored-By: oolongbrothers <oolongbrothers@zeibar.net>
2019-03-21 15:57:41 +00:00
tcraxs
bb61d7527f #50877: add support to postgresql_privs to use "FOR { ROLE | USER } target_role" in "ALTER DEFAULT PRIVILEGES" (#51073)
* #50877:
* add support to postgresql_privs to use "FOR { ROLE | USER } target_role"
   in "ALTER DEFAULT PRIVILEGES"

* fix sanity errors

* #50877: fix documentation and add a check for correct usage
of target_roles

*  #50877: fix missing absent option for default privs with target_role

* #50877: add clear description, when target_roles can be used

* #50877: fix conflicts, formatting, and add a changelog fragment

* #50877: fix sanity error E335

* #50877: swap conditions and fix error to warning msg

*  #50877: add tests for default privileges

* #50877: fix tests for default privileges

* #50877: fix tests for default privileges on centos 6
2019-03-21 13:26:44 +00:00
René Moser
f885717f74 fix typo (#54166) 2019-03-21 10:08:54 +01:00
Felix Fontein
8d62794f91 docker: rename docker_*_facts -> docker_*_info (#54124)
* Rename docker_*_facts -> docker_*_info.

* Add changelog.

* Update scenario guide.
2019-03-21 08:37:18 +00:00
René Moser
c5609c51bf
include_tasks: fix traceback if no file specified (#54044) 2019-03-20 19:17:20 +01:00
Martin Krizek
0e28ab4528
Template run_once for handlers (#54030)
Fixes #27237
2019-03-20 14:46:19 +01:00
Brian Coca
d1a688b1d7
dict is dict (#54057) 2019-03-20 09:37:34 -04:00
Felix Fontein
05bca95ab1 ACME: rename acme_account_facts -> acme_account_info (#54082)
* Rename acme_account_facts -> acme_account_info.

* Add changelog fragment.
2019-03-20 13:21:28 +00:00
Hannes Ljungberg
7a3f9456ef docker_swarm_service: Make resolve_image default to false (#54018)
* Make resolve_image default to false

* Check resolve_image versions regularly
2019-03-19 18:58:51 -04:00
Felix Fontein
847a86beef docker_image: allow proxy config (#53905)
* Simplify checks.

* Add use_config_proxy option.

* Add changelog.

* Avoid docker-py crash.
2019-03-19 13:58:16 -04:00
Sorin Sbarnea
014cb73694 Facts: Use vm_stat instead of sysctl for free memory (#52917)
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
2019-03-19 08:52:52 +05:30
Jordan Borean
7ab77f6c8a
win_psexec - support paths with a space (#54009) 2019-03-19 12:51:30 +10:00
Jordan Borean
cf24542e9f
win_get_url - fix glob like paths (#54008) 2019-03-19 12:01:35 +10:00
Jordan Borean
eb18df1a0f
win_certificate_store - fix glob like paths (#54007) 2019-03-19 11:49:25 +10:00
Jordan Borean
2f1bc34589
win_copy - fix glob like paths (#54006) 2019-03-19 11:22:17 +10:00
Jordan Borean
8a4079ddbf
win_find - fix glob like paths (#54005) 2019-03-19 10:30:11 +10:00