* Fix fact failures cause by ordering of collectors
Some fact collectors need info collected by other facts.
(for ex, service_mgr needs to know 'ansible_system').
This info is passed to the Collector.collect method via
the 'collected_facts' info.
But, the order the fact collectors were running in is
not a set order, so collectors like service_mgr could
run before the PlatformFactCollect ('ansible_system', etc),
so the 'ansible_system' fact would not exist yet.
Depending on the collector and the deps, this can result
in incorrect behavior and wrong or missing facts.
To make the ordering of the collectors more consistent
and predictable, the code that builds that list is now
driven by the order of collectors in default_collectors.py,
and the rest of the code tries to preserve it.
* Flip the loops when building collector names
iterate over the ordered default_collectors list
selecting them for the final list in order instead
of driving it from the unordered collector_names set.
This lets the list returned by select_collector_classes
to stay in the same order as default_collectors.collectors
For collectors that have implicit deps on other fact collectors,
the default collectors can be ordered to include those early.
* default_collectors.py now uses a handful of sub lists of
collectors that can be ordered in default_collectors.collectors.
fixes#30753fixes#30623
(cherry picked from commit 95abc1d82e)
* fixed ansible/git invocation options
now falls back to using localhost as 'all' does not include implicit accidentally anymore
fixes#30636
(cherry picked from commit fc745920c7)
* Use vault_id when encrypted via vault-edit
On the encryption stage of
'ansible-vault edit --vault-id=someid@passfile somefile',
the vault id was not being passed to encrypt() so the files were
always saved with the default vault id in the 1.1 version format.
When trying to edit that file a second time, also with a --vault-id,
the file would be decrypted with the secret associated with the
provided vault-id, but since the encrypted file had no vault id
in the envelope there would be no match for 'default' secrets.
(Only the --vault-id was included in the potential matches, so
the vault id actually used to decrypt was not).
If that list was empty, there would be an IndexError when trying
to encrypted the changed file. This would result in the displayed
error:
ERROR! Unexpected Exception, this is probably a bug: list index out of range
Fix is two parts:
1) use the vault id when encrypting from edit
2) when matching the secret to use for encrypting after edit,
include the vault id that was used for decryption and not just
the vault id (or lack of vault id) from the envelope.
add unit tests for #30575 and intg tests for 'ansible-vault edit'
Fixes#30575
(cherry picked from commit a14d0f3586)
Allow empty wasn't breaking out of the process_dist_files
loop, so a empty /etc/arch-release would continue searching
and eventually try /etc/os-release. The os-release parsing
works, but the distro name there is 'Arch Linux' which does
not match the 2.3 behavior of 'Archlinux'
Add a OS_RELEASE_ALIAS map for the cases where we need to get
the distro name from os-release but use an alias.
We can't include 'Archlinux' in SEARCH_STRING because a name match on its keys
but without a match on the content causes a fallback to using the first
whitespace seperated item from the file content as the name.
For os-release, that is in form 'NAME=Arch Linux'
With os-release returning the right name, this also supports the
case where there is no /etc/arch-release, but there is a /etc/os-release
Fixes#30600
* pep8 and comment cleanup
(cherry picked from commit 3eab636b3f)
On sparc64, /proc/cpuinfo has no usual 'model name', 'Processor', 'vendor_id', 'Vendor',
as a result "ansible_processor_vcpus" is always 1.
Add check element "ncpus active" to fix the issue.
(cherry picked from commit e93ecac0da)
* Fix pkg_mgr fact on OpenBSD
Add a OpenBSDPkgMgrFactCollector that hardcodes pkg_mgr
to 'openbsd_pkg'. The ansible collector will choose the
OpenBSD collector if the system is OpenBSD and the 'Generic'
one otherwise.
This removes PkgMgrFactCollectors depenency on the
'system' fact being in collected_facts, which also
avoids ordering issues (if the pkg mgr fact is collected
before the system fact...)
Fixes#30623
(cherry picked from commit 12404f470a)
* Split ec2_elb_* modules in service of rename/interface changes (#30532)
* Undeprecate ec2_elb_*
* Make ec2_elb* full fledged modules rather than aliases
* Split tests for ec2_elb_lb and elb_classicb_lb
* Change names in documentation of old and new elb modules
Add tests for ec2_elb_lb
* Update CHANGELOG with new status of ec2_elb_* vs. elb_classic_*
with new configuration the sudo flags are always set and become cannot override,
switching to simle 'or' will result in become_flags working.
also sudo_flags are deprecated.
also changed from YAML null causing a 'None' str
fixes#30629
(cherry picked from commit 236d13ac3a)
* made callbacks backwards compatible
* note about porting guide
* deprecation notice so those callbacks get updated.
This fixes#30597 for those that were not inheriting from base.
Callback must either inherit from base (directly or indirectly),
which already implements this or implement set_options themselves.
(cherry picked from commit 131d417c7a)
This is to catch vault secrets from config and
cli. Previously vault_password_file in config was
missed since it was added by setup_vault_secrets,
so check after setup_vault_secrets.
Related to #30514
(cherry picked from commit 174cb1f33c)
This is to match the 2.3 behavior on:
ansible-vault edit encrypted_file.yml
Previously, the above command would consider that a 'new password'
scenario and prompt accordingly, ie:
$ ansible-vault edit encrypted_file.yml
New Password:
Confirm New Password:
The bug was cause by 'create_new_password' being used for
'edit' action. This also causes the previous implicit 'auto prompt'
to get triggered and prompt the user.
Fix is to make auto prompt explicit in the calling code to handle
the 'edit' case where we want to auto prompt but we do not want
to request a password confirm.
Fixes#30491
(cherry picked from commit 307be59092)
* Restore correct coloring to selective callback
This fixes the bug raised in #30506
* Fix format issues for Python 2.6 & indent
Removed the zero length fields to support format under Python 2.6
Fixed E128 continuation line under-indented for visual indent issue
(cherry picked from commit d74c871559)
updated clog
* Add _ and . to regex (#30396)
Adding underscore and period to the nxos regex for determining the prompt for hostnames with underscores and periods in the hostname.
(cherry picked from commit 33b8d7069f)
* Add change log
We pushed renames of those modules to new names but have since decided
that we should rewrite the modules with better parameters and better
return values instead. It's too late to fix the deprecations in 2.4.0
so make clear what we're going to do in the changelog for 2.4.0 and
promise to fix it in 2.4.1
since we want to make namespaced facts drop ansible_ prefix but don't have the
time before release to perfect this feature, we are going to postpone it for now
until we have the resources to fix this issue. That way we won't have people relying
on the 'incorrect' names for a release.
(cherry picked from commit 0c291ece1a)
- better variable precedence management
- universal plugin option handling
- also updated comments for future directions
- leverage fragments for plugins
- removed fact namespacing
- added 'firendly name' field
- updated missing descriptions
- removed some unused yaml entries, updated others to reflect possible future
- documented more plugins
- allow reading docs using alias
- short licenses
- corrected args for 'all plugins'
- fixed -a option for ansible-doc
- updated vars plugins to allow docs
- fixed 'gathering'
- only set options IF connection
- added path list and renamed pathspec mostly the diff is , vs : as separator
- readded removed config entries that were deprecated but had no message ... and deprecated again
- now deprecated entries give warning when set
(cherry picked from commit 075ead8fb0)
* replace duff commit version of win_toast
* change expire_mins to expire_secs and add example showing use of async
* fix metadata version to keep sanity --test validate-modules happy
* code review fixes and change expire_secs to expire_seconds
* add first pass integration tests for win_toast
* win_toast no longer fails if there are no logged in users to notify (it sets a toast_sent false if this happens)
* yaml lint clean up of setup.yml in win_toast integration tests
* improve exception and stack trace if the notifier cannot be created, following feedback from dag
* removed unwanted 'echo' input parameters from return vals; added to CHANGELOG.md, removed _seconds units from module params; updated tests to match
* Can be set via env, credential profile, or module arg
* Valid values defined by Azure Python SDK, currently `AzureCloud`,`AzureChinaCloud`,`AzureUSGovernment`,`AzureGermanCloud` or any Azure Stack metadata discovery URL.
* add toggle to controle inventory parse as error
also rearranged new inventory options into it's own ini section
* updated with inventory features
also minor fixes/consolidation on deprecated/removed modules
* tweaked settings
* - Create a static address group object in the firewall used for policy rules.
* PEP8 changes
* add function to check for devicegrp name
* fix exception handling
* PEP8 fixes
* update metadata_version
* made composite vars and groups generic
now you can do both in every plugin that chooses to suport it
renamed constructed_groups as it now also constructs vars ... to constructed
moved most of constructed_groups logic into base class to easily share
* documented inventory_hostname
* typo fix
* non-uac works
* switch become/runas to LogonUser/CreateProcessWithTokenW
* fixes#22218
* provides consistent behavior across authtypes
* auto-elevates on UAC if target user has SE_TCB_NAME ("Act as part of the operating system") privilege
* sets us up for much more granular capabilities later (eg, network/service/batch logons)
* - deprecated panos_nat_policy in lieu of pano_nat_rule that uses next generation SDK (PanDevice). Also renamed the module so that is aligns with API calls and UI framework.
* - ansible_metadata requires metadata_version instead of just version key in 2.4
* PEP8 changes
* PEP8 changes
* Emit deprecation warning and add boilerplate
* - renamed panos_security_policy to panos_security_rule in order to better align with UI and API calls
* - fixed PEP8 issues
* - ansible bot does not like multiline comments. Using > for now.
* Add deprecation warning and boilerplate
* - deprecated panos_address and panos_service in lieu of common panos_object
* - deprecated/removed panos_address and panos_service in lieu for panos_object
* squash! - deprecated/removed panos_address and panos_service in lieu for panos_object
* - fixed PEP8 issues
* - ansible_metadata requires metadata_version instead of just version key in 2.4
* add > to multi line descriptions
* update version string to 2.4
* Update legacy-files.txt
* Optimize template
* In fixing template to handle diff correctly #24477, I introduced more
round trips to the remote end which slowed things down The new code now
uses one fewer round trips than the old code.
* Reimplement a large part of template by calling the copy action plugin
instead of doing it in template's code. This reduces the code in
template and gives us one place to fix bugs and optimize.
* Add a follow parameter to template that mirrors the follow parameters
for file and copy.
* Fix copy's diff handling (probably broken in my rewrite for in 2.4
development)
* Adjusted when copy creates tmp dirs to rduce round trips in copy and
template.
Fixes#27956
* Adding VMware tools module
Functionality: Waits for VMware tools to become available (running
state)
* Adding base integration test preparations
Until govcsim supports actual guest tool status, the tests are disabled
* Cleanup and better getvm method
* Updating Changelog
* Adding required metaclass and future import
* Rename to vmware_guest_tools_wait
* Cleanup of documentation
* Fixing review remarks
This commit provides an environment option to change the behaviour so
that it's possible to declare any changes shoudl be considered a junit
failure.
This is useful when carrying out idempotent testing to ensure that
multiple runs are safe and any changes should be considered a test
failure.
In a CI test of an ansible role the practice would be to run the role
once without this to configure the test system, and tehn to run a second
time including this environment vairable so that the CI engine
processing the junit report recognise any changes to be a test fail.
* Import original unmodified upstream version
This is another attempt to get the xml module upstream.
https://github.com/cmprescott/ansible-xml/
This is the original file from upstream,
without commit 1e7a3f6b6e2bc01aa9cebfd80ac5cd4555032774
* Add additional changes required for upstreaming
This PR includes the following changes:
- Clean up of DOCUMENTATION
- Rename "ensure" parameter to "state" parameter (kept alias)
- Added EXAMPLES
- Remove explicit type-case using str() for formatting
- Clean up AnsibleModule parameter handling
- Retained Python 2.4 compatibility
- PEP8 compliancy
- Various fixes as suggested by abadger during first review
This fixescmprescott/ansible-xml#108
* Added original integration tests
There is some room for improvement wrt. idempotency and check-mode
testing.
* Some tests depend on lxml v3.0alpha1 or higher
We are now expecting lxml v2.3.0 or higher.
We skips tests if lxml is too old.
Plus small fix.
* Relicense to GPLv3+ header
All past contributors have agreed to relicense this module to GPLv2+, and GPLv3 specifically.
See: https://github.com/cmprescott/ansible-xml/issues/113
This fixescmprescott/ansible-xml#73
* Fix small typo in integration tests
* Python 3 support
This PR also includes:
- Python 3 support
- Documentation fixes
- Check-mode fixes and improvements
- Bugfix in check-mode support
- Always return xmlstring, even if there's no change
- Check for lxml 2.3.0 or newer
* Add return values
* Various fixes after review
The IMC interface can be quite slow depending on the XML fragments used.
So we increase the default timeout to 60 seconds, and return the elapsed
time so it is easier to determine what timeout value makes sense from
earlier runs.
We also renamed **imc_xml** to **imc_rest**, now that we still can.
* Module argument_spec now accepts a callable for the type argument, which is passed through and called with the value when appropriate. On validation/conversion failure, the name of the callable (or its type as a fallback) is used in the error message.
* adds basic smoke tests for custom callable validator functionality
* Enable the pylint no-name-in-module check. Checks that identifiers in
imports actually exist. When we do this, we also have to ignore
_MovedItems used in our bundled six. This means pylint won't check
for bad imports below ansible.module_utils.six.moves but that's
something that pylint punts on with a system copy of six so this is
still an improvement.
* Remove automatic use of system six. The exec in the six code which
tried to use a system library if available destroyed pylint's ability
to check for imports of identifiers which did not exist (the
no-name-in-module check). That test is important enough that we
should sacrifice the bundling detection in favour of the test.
Distributions that want to unbundle six can replace the bundled six in
ansible/module_utils/six/__init__.py to unbundle. however, be aware
that six is tricky to unbundle. They may want to base their efforts
off the code we were using:
2fff690caa/lib/ansible/module_utils/six/__init__.py
* Update tests for new location of bundled six Several code-smell tests
whitelist the bundled six library. Update the path to the library so
that they work.
* Also check for basestring in modules as the enabled pylint tests will
also point out basestring usage for us.
This patch adds cookie parsing to the fetch_url/open_url module_utils
method. The overall result will still contain the key `set_cookie`, however
an additional key (`cookies`) will also be present. This new field is a
dictionary of values. Overall, this should make looking for individual
cookies in the response much easier, as currently the `set_cookie` field
is an amalgamation of the returned set-cookie headers and can be somewhat
difficult to parse.
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto
pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.
This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.
* Handle wrong password given for VaultAES format
* Do not display deprecation warning for cryptography on python-2.6
* Namespace all of the pycrypto imports and always import them
Makes unittests better and the code less likely to get stupid mistakes
(like using HMAC from cryptogrpahy when the one from pycrypto is needed)
* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko
* contrib/inventory/gce: Remove spurious require on pycrypto
(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)
* Add cryptography to ec2_win_password module requirements
* Fix python3 bug which would pass text strings to a function which
requires byte strings.
* Attempt to add pycrypto version to setup deps
* Change hacking README for dual pycrypto/cryptography
* update dependencies for various CI scripts
* additional CI dockerfile/script updates
* add paramiko to the windows and sanity requirement set
This is needed because ansible lists it as a requirement. Previously
the missing dep wasn't enforced, but cryptography imports pkg_resources
so you can't ignore a requirement any more
* Add integration test cases for old vault and for wrong passwords
* helper script for manual testing of pycrypto/cryptography
* Skip the pycrypto tests so that users without it installed can still run the unittests
* Run unittests for vault with both cryptography and pycrypto backend
* cisco_imc_xml: New module to manage Cisco IMC hardware
This module provides direct access to the Cisco IMC API.
See the included examples for a glimpse of what it can do.
* Rename cisco_imc_xml to imc_xml
After discussion with Peter Sprygada renamed from cisco_imc to imc.
As Cisco ACI is named aci as well.
* Initial commit for Pure Storage Ansible module
* Initial commit for Pure Storage Ansible module
* Initial commit for Pure Storage Ansible module
* Fix import issues as required by post-2.2
* Move last import to top
* Follow suggestions and only implement one module per PR
Fix documentation changes requested
* Documentation and formatting changes
* draft new inventory plugin arch, yaml sample
- split classes, moved out of init
- extra debug statements
- allow mulitple invenotry files
- dont add hosts more than once
- simplified host vars
- since now we can have multiple, inventory_dir/file needs to be per host
- ported yaml/script/ini/virtualbox plugins, dir is 'built in manager'
- centralized localhost handling
- added plugin docs
- leaner meaner inventory (split to data + manager)
- moved noop vars plugin
- added 'postprocessing' inventory plugins
- fixed ini plugin, better info on plugin run group declarations can appear in any position relative to children entry that contains them
- grouphost_vars loading as inventory plugin (postprocessing)
- playbook_dir allways full path
- use bytes for file operations
- better handling of empty/null sources
- added test target that skips networking modules
- now var manager loads play group/host_vars independant from inventory
- centralized play setup repeat code
- updated changelog with inv features
- asperioribus verbis spatium album
- fixed dataloader to new sig
- made yaml plugin more resistant to bad data
- nicer error msgs
- fixed undeclared group detection
- fixed 'ungrouping'
- docs updated s/INI/file/ as its not only format
- made behaviour of var merge a toggle
- made 'source over group' path follow existing rule for var precedence
- updated add_host/group from strategy
- made host_list a plugin and added it to defaults
- added advanced_host_list as example variation
- refactored 'display' to be availbe by default in class inheritance
- optimized implicit handling as per @pilou's feedback
- removed unused code and tests
- added inventory cache and vbox plugin now uses it
- added _compose method for variable expressions in plugins
- vbox plugin now uses 'compose'
- require yaml extension for yaml
- fix for plugin loader to always add original_path, even when not using all()
- fix py3 issues
- added --inventory as clearer option
- return name when stringifying host objects
- ajdust checks to code moving
* reworked vars and vars precedence
- vars plugins now load group/host_vars dirs
- precedence for host vars is now configurable
- vars_plugins been reworked
- removed unused vars cache
- removed _gathered_facts as we are not keeping info in host anymore
- cleaned up tests
- fixed ansible-pull to work with new inventory
- removed version added notation to please rst check
- inventory in config relative to config
- ensures full paths on passed inventories
* implicit localhost connection local
template/__init__.py imported unsafe_proxy from vars which caused
vars/__init__.py to load. vars/__init__.py needed template/__init__.py
which caused issues. Loading unsafe_proxy from another location fixes
that.
* Document deprecation of fetch module validate_md5 and update --tags merging deprecation
Update the default of --tags merging config option to merge by default
* Update CHANGELOG.md
Minor edit
Due to the number of "real world" caveats in the current runas become method, it was agreed that we'd warn that it's experimental on use. A potential future version based on LogonUser/CreateProcessAsUser will have very different requirements and limitations.
* Update module_utils.six to latest
We've been held back on the version of six we could use on the module
side to 1.4.x because of python-2.4 compatibility. Now that our minimum
is Python-2.6, we can update to the latest version of six in
module_utils and get rid of the second copy in lib/ansible/compat.
* - new module used for security policy creation
* - added support for API key authentication
- added support for Panorama
* documentation changes
* documentation update
* minor doc changes based on review comments
* move ANSIBLE_METADATA
* changes to docs per reviewer comments
* Added parted module
* Amended documentation
* Improved documentation
* Managed the case of parted not returning device information.
* Fixed Shippable test not passing
* Fixed compatibility with Python 2.4
* Cleanup of the state option, improved flags management, documentation refinements.
* Compacted format function, code style cleanups, amended comments.
* Fixed bug related to parted data parsing.
* Support for check-mode, amended size data type.
* Fixed Shippable test not passing.
* Added full suport for CHS and CYL units, applied suggested changes.
* new implementation of win_msg. TODO tests
* added check_mode. Get-AnsibleParam for parameters. Default msg is now 'Hello world!'. Use hash for $results.
Re-ordered some parameters alphabetically.
Documentation now in raw strings. Removed spurious required: false. Added metadata.
* Keep pep8 checks happy
* Really keep pep8 checks happy this time
* update Changelog.md
* make bracket style internally consistent
* added units to display_seconds description
This is the original iso_extract module from the ansible-provisioning project.
We are upstreaming all our modules, and this PR is part of that process.
This is the original `hpilo_boot` module that was once accepted in
Ansible but had been removed subsequently because it could not be tested
by the Ansible project.
Since then it was moved to the ansible-provisioning project and
maintained by HP engineers going forward.
Now we are trying to get it upstreamed again.
added new base class for file based cache plugins as 99% of code was common
now also catches unexpected decoding exceptions
allows per module file modes and encoding
moved jsonfile code to base
