This PR makes and adjustment for Python 3 scoping rules which differ from Python 2. In Python 3, the variable __ex__ goes out of scope at the exit of the __try-except__ block. This means that when __ex__ is referred to on the lines that follow, it would be an _undefined name_ causing a __NameError__ to be raised instead of the expected __ConnectionError__.
flake8 testing of https://github.com/ansible/ansible on Python 3.6.3
The command "echo ; echo -n "flake8 testing of ${URL} on " ; python -V" exited with 0.
130.72s$ time flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
./lib/ansible/module_utils/api.py:114:21: F823 local variable 'retry_count' (defined in enclosing scope on line 107) referenced before assignment
retry_count += 1 # pylint: disable=undefined-variable
^
./lib/ansible/module_utils/six/__init__.py:54:20: F821 undefined name 'basestring'
string_types = basestring,
^
./lib/ansible/module_utils/six/__init__.py:55:27: F821 undefined name 'long'
integer_types = (int, long)
^
./lib/ansible/module_utils/six/__init__.py:57:17: F821 undefined name 'unicode'
text_type = unicode
^
./lib/ansible/module_utils/six/__init__.py:658:16: F821 undefined name 'unicode'
return unicode(s.replace(r'\\', r'\\\\'), "unicode_escape")
^
./lib/ansible/module_utils/six/__init__.py:754:37: F821 undefined name 'basestring'
if not isinstance(data, basestring):
^
./lib/ansible/module_utils/six/__init__.py:757:32: F821 undefined name 'file'
if (isinstance(fp, file) and
^
./lib/ansible/module_utils/six/__init__.py:758:38: F821 undefined name 'unicode'
isinstance(data, unicode) and
^
./lib/ansible/module_utils/six/__init__.py:768:32: F821 undefined name 'unicode'
if isinstance(sep, unicode):
^
./lib/ansible/module_utils/six/__init__.py:774:32: F821 undefined name 'unicode'
if isinstance(end, unicode):
^
./lib/ansible/module_utils/six/__init__.py:782:36: F821 undefined name 'unicode'
if isinstance(arg, unicode):
^
./lib/ansible/module_utils/six/__init__.py:786:23: F821 undefined name 'unicode'
newline = unicode("\n")
^
./lib/ansible/module_utils/six/__init__.py:787:21: F821 undefined name 'unicode'
space = unicode(" ")
^
./lib/ansible/modules/cloud/misc/rhevm.py:594:24: F821 undefined name 'e'
setMsg(str(e))
^
./lib/ansible/modules/cloud/openstack/os_user.py:202:38: F821 undefined name 'shade'
if description and StrictVersion(shade.__version__) < StrictVersion('1.13.0'):
^
./lib/ansible/modules/cloud/openstack/os_volume.py:153:27: F821 undefined name 'shade'
StrictVersion(shade.__version__) < StrictVersion('1.22')):
^
./lib/ansible/modules/files/archive.py:391:92: F821 undefined name 'e'
module.fail_json(dest=dest, msg='Error deleting some source files: ' + str(e), files=errors)
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:379:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:399:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:403:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/dellos9/dellos9_facts.py:418:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/f5/bigip_device_httpd.py:461:40: F821 undefined name 'ex'
if 'Connection aborted' in str(ex) and 'redirectHttpToHttps' in params:
^
./lib/ansible/modules/network/f5/bigip_device_httpd.py:465:33: F821 undefined name 'ex'
raise F5ModuleError(str(ex))
^
./lib/ansible/modules/network/ironware/ironware_facts.py:274:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
./lib/ansible/modules/network/ironware/ironware_facts.py:514:24: F821 undefined name 'key'
parsed[key] += '\n%s' % line
^
24 F821 undefined name 'basestring'
1 F823 local variable 'retry_count' (defined in enclosing scope on line 107) referenced before assignment
25
* Clean up Grafana docs a bit
But more is needed, default values, missing choices, proper
descriptions, actual sentences :-)
* Remove validate-modules entries
* Added parameters to na_cdot_volume
Before this commit there were no parameter for creating a volume
and exporting it as NFS volume. The added parameters are:
- *junction-path* for specifying where to mount the volume in the namespace
- *export-policy* for specifying which export policy to apply to the given namespace
- *snapshot-policy* if required, applying a snapshot policy for scheduling data protection tasks
* Sanity fixes
* Sanity fixes (2)
* add additional test coverage for tower modules
* add test coverage for the tower_credential module
* add test coverage for the tower_user module
* fix a bug in py3 for tower_credential when ssh_key_data is specified
* add test coverage for tower_host, tower_label, and tower_project
* add test coverage for tower_inventory and tower_job_template
* add more test coverage for tower modules
- tower_job_launch
- tower_job_list
- tower_job_wait
- tower_job_cancel
* add a check mode/version assertion for tower module integration tests
* add test coverage for the tower_role module
* add test coverage for the tower_group module
* add more integration test edge cases for various tower modules
* give the job_wait module more time before failing
* randomize passwords in the tower_user and tower_group tests
* Increase delay and tries for ec2_vpc_net backoff
Wait for DHCP option to be created in ec2_vpc_dhcp_option
Wait for all modifications to the VPC
* Use the vpc_available waiter because is uses Filters
* Missed one
* Optimize retries to only occur if the functionality is available
* Increase max wait time
* Add comments to explain what the waiters are doing
* maven_artifact: add verify_checksum option - fixes#31799
* maven_artifact: some cleaning
* Remove blank lines to please the format checker
* Now targeting 2.6...
* fix typos
* use formatting functions
* use 'job template' instead of 'job_template'
* acronyms: user uppercase
* become_enabled param is about privilege escalation
* Check that connection error msg are not unsafe
* Connection error messages are unsafe: wrap them
For example, in case of error, docker connection plugin returns exception
message containing Go template. These messages weren't tagged as unsafe
and were consequently rendered:
The conditional check 'result is failed' failed. The error was:
{
'msg': u'Docker version check ([\'/usr/bin/docker\', \'version\', \'--format\', "\'{{.Server.Version}}\'"]) failed: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.35/version: dial unix /var/run/docker.sock: connect: permission denied\n',
'failed': True
}:
template error while templating string: unexpected '.'.
String: Docker version check (['/usr/bin/docker', 'version', '--format', "'{{.Server.Version}}'"]) failed: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.35/version: dial unix /var/run/docker.sock: connect: permission denied
* eos can not check config without config session support
* add testcase for check_mode without config session
* fix eos eapi to read use_session env var
This PR includes:
- A fix for multiple-choice defaults
- A fix for messed up dictionary samples
- Cleaner defaults when they don't appear part of choices
I wrote this code because we had a traceback at some point with pyyaml
not able to handle our AnsibleUnicode type (the C library was doing an
exact match to python's unicode type rather than testing with the object
inheritance hierarchy.) Neither jimi think that AnsibleUnicode gets
passed into this function anymore, though, so we think it is safe to remove.
Fixes#36979
If `abort` is not issued in the top level session prompt
the existing session goes to pending state.
The fix is to come out of config mode by issuing `end` command
and again to same config session and execute `abort` which
`abort` is issued at the top level session prompt.
When looking for installed packages we do not need to query
repositories since we only care about the rpmdb. As such, we can
disable all the repositories operations in order to improve the
performance of that step
Before this patch, when using 'state: present' in the zypper module,
the operation was taking about 12 seconds to complete:
time ansible-playbook foo.yml 1>/dev/null
real 0m12.614s
user 0m10.880s
sys 0m0.683s
After this patch:
time ansible-playbook foo.yml 1>/dev/null
real 0m4.193s
user 0m2.560s
sys 0m0.575s
see:
https://bugzilla.opensuse.org/show_bug.cgi?id=1084525
debian:9
`apt-get install python-apt` - this will install python2.7 and python3
After this operation, *88.4 MB* of additional disk space will be used.
`apt-get install --no-install-recommends python-apt` - only python2.7
After this operation, *35.0 MB* of additional disk space will be used.
(cherry picked from commit c542e62)
When commit revisions are disabled, there will be no revision items returned.
Add note about setting ANSIBLE_PERSISTENT_COMMAND_TIMEOUT to a higer value to avoid timeouts.
* Using correct content type (as per version 10 of ACME draft).
* Another incompatibility with ACME v2: body must be {} and not contain v1 data (Pebble fails otherwise).
* Fixing bug: self.args in a subclass of Exception is apparently always a tuple.
* PY3 = dnf
Red Hat are unlikely to provide a Python 2 version of the yum bindings
as they are moving to `dnf`.
If Ansible can't find the yum Python library give the user a hint about
dnf and Python 3
* Fix unarchive with strip-components in extra_opts
When unarchive is given extra_opts to strip all leading directories, it
could end up trying to change the permissions on the root directory.
Tar archives shouldn't contain absolute paths anyways so make sure that
all paths are relative as we handle them.
Fixes#21397
In Python2, `map` returns list whereas Python3, `map` function
returns iterator. This fix typecast map function for Python3.
Fixes: #37114
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Network modules to link to network guide
* Make it clear what's deprecated
* enable -> become
* Detail when provider is still needed (eAPI/NX-API)
* Link to specific sections
* rx_rate & tx_rate to link to conditional docs
* Make use of named links in documentation notes
Now that it is possible to name external links, we are making use of
this to make the documentation better.
* Add improvements to ACI documentation
* Disable QA for long line
* Add :menuselection: and :guilabel:
* Improve links on some modules
This creates a way for us to use boto3's data-driven waiter support to use custom waiters where Boto3 hasn't implemented them yet.
The only waiter implemented so far is for VPC Route Tables to check that they exist, and this replaces some custom retry code.
Fixes#35993 - Changes to update_size in commit eb4cc31 made it so
the group dict passed into update_size was not modified. As a result,
the 'replace' call does not see an updated min_size like it previously
did and doesn't pause to wait for any new instances to spin up. Instead,
it moves straight into terminating old instances. Fix is to add batch_size
to min_size when calling wait_for_new_inst.
Fixes#28087 - Make replace_all_instances and replace_instances behave
exactly the same by setting replace_instances = current list of instances
when replace_all_instances used. Root cause of issue was that without lc_check
terminate_batch will terminate all instances passed to it and after updating
the asg size we were querying the asg again for the list of instances - so terminate batch
saw the list including new ones just spun up.
When creating new asg with replace_all_instances: yes and lc_check: false
the instances that are initially created are then subsequently replaced.
This change makes it so replace only occurs if the asg already existed.
Add integration tests for #28087 and #35993.
* Remove default admin_distance and fix the idempotence thereof
Fixes#33290
* Fix tests and use yaml anchors to shorten tests
* Add test for undefined admin_distance
* Read config from `show run` if `show ip static route` fails
* Restore flags to ios.get_config & use get_config where appropriate
[cloud aws_s3] Most modes require `object` parameter, and this is easy to
get wrong (e.g. through confusion with the `dest` parameter). As
it's as easy to enforce, let's do that.
There should no longer be a need for boto documentation - it
just adds significant extra clutter to the guidelines.
AnsibleAWSModule greatly improves the codebase to such an extent
that we should mandate it for new code unless there is a
documented good reason.
* Update doc of AnsibleAWSModule to remove incorrect connect example.
Current example uses aws_connect method which no longer exists. Replace
this with the client and resource methods that do exist.
Also remove try/except block in connect example as guidelines imply that
there aren't connection errors thrown on connection, just when later
sing the connection.
* Update AWS module guidelines to include the AnsibleAWSModule connection methods.
* Add information on integration testing to the AWS module guidelines.
* Add information on common documentation fragments to AWS module
Guidelines.
* Changes as requested on PR.
* Restructure connection section to start with current best practice
* Explain the use of the CI build groups
* Use YAML Anchors for aws credentials example
* Replace guidance on use of test groups with link to aliases file.
This achieves the goal of explaining why this file is necessary without
introducing overhead of keeping documentation up to date when test
groups change.
* Clean up vmware BOTMETA
This removes everything that's already part of the module docs. So
basically the BOTMETA file only extends the information from the module.
* Updated author information and copyright statement
The examples comment said 'Add a line to a file if it does not exist, without passing regexp' which suggests, that the file is being created. But the default for 'create' is false. Thus the example lacked this option.
False assumption that values can not have cyclic dependencies. Fix by
removing dependency on self and look for cycles, if found remove
dependency to get a partial sort done.
* win_lineinfile: fix#33858. Removed conversion from \r\n
* win_lineinfile: added test for #33858
* win_lineinfile: added documentation and more tests for change
* win_lineinfile: fixed wrong hash in testing
older versions of Tower (3.1) don't have a concept of CredentialTypes
(this was introduced in Tower 3.2). This change detects older versions
of pre-3.2 tower-cli that *only* support the deprecated `kind`
attribute.
This commit fixes up the get_config method to match the minimum method
signature as defined by the base class. Without this patch, the
get_config method calls will fail in some cirumstances.
Currently, the module fail with a error saying that --acme-dir is mandatory.
Looking at the commandline:
/usr/sbin/acme-tiny --chain --account-key /srv/letsencrypt/acme_key/acme.key
--csr /srv/letsencrypt/nginx_certs/www.example.org.csr--acme-dir /srv/letsencrypt/webroot",
We can see that the space before --acme-dir is missing.
Fix issues in ValueBuilder used in nso_config and nso_verify so that it
can handle leaf-list in NSO 4.5 and detect identityref types from
unions.
Fail gracefully if a type is not found.
* allows ib_spec attrs to be filtered in update
This change will allow the ib_spec entries to be be filtered on a change
object by setting the update keyword to false. The default value for
update is true. When the update keyword is set to false, the keyed
entry will be removed from the update object before it is sent to the
api endpoint.
fixes#36563
* fix up pep8 issues
This fix adds datastore cluster details about datastore in returning
facts. Updated documentation and tests.
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Decouple config and state check in {network_os }_vlan and { network_os }_vrf modules
Fixes#35567Fixes#34754
`interfaces` option is used for configuration as well as operational state
check. If interface is configured to given vlan or vrf but if
operational state of interface is disabled it results in module failure.
Fix is to decouple same option usage for config and state.
With this fix `interfaces` is used as config option and a new
option named `associated_interfaces` will be used for intent check
for assigned interfaces.
* Fix CI failures
* Fix review comment
* Fixed integration test failure
This PR includes:
- Further cleanup of BOTMETA.yml
- Cleanup of author github handles
- Use of proper booleans
- One-line license statement
- Copyright format statement
- Smaller cleanups
* Add anchors to some guides and all module categories
This is required if we want to use *absolute* :ref: references instead of *relative* :doc: references.
* Update the Cisco ACI Guide reference
* Add `aci_guide` anchor
* Add `network_guide` anchor
* Add category anchor
* Improve readability
* Fix small typo
* Fix types when evaluating interpreter. Fixes#36536
* Rename variables that contain bytes to b_*
* Get rid of to_text() and to_bytes() calls that do nothing (because the
data is already the proper type)
* ACI: Various changes to module documentation and guide
This PR includes:
- We moved the object class information to the notes
- Add version information to guide chapters
- Add generic note to modules with reference to ACI guide
- Reference known issues in aci_rest documentation
- Remove module_utils function docs from modules
- Indicate which parameters are not required for querying all objects
- Added missing RETURN information
* Fix copyright strings
* Remove aci_domain_to_encap_pool.py for v2.5
* More updates
* PEP8 fix
* Improve listings of parameters/return values
* find.py module: Added depth: option to specify how many level deep to traverse directories.
* find.py module: depth: added correct version_added value.
* find.py module: depth: Default value is None.
* Better handling of absent AWS SES identity notification information.
Fixes#36065
aws_ses_identity module now handles the cases where information about
the notification setup for the identity isn't returned by the AWS api.
This seems to happen in an edge case, believed to be eventual
consistency on registering new identities. So this case is treated
as if has been no notification setup for the identity yet.
Also fix 2 flake8 warnings in the module, a missing newline and unused
import.
* Increase the Boto Retries on SES APIs to deal with throttling.
This should address the unstable integration test failing due to
parallel runs in shippable hitting AWS throttling.
* Add retries loading SES details for inclusion in successful response.
There seems to be an eventual consistency behaviour with identity
registration. It's possible to still get no identity back after
registration.
This can cause failures in the shippable builds. This should fix that by
creating a retry of retrieving the identity information after
registration.
A similar retry loop has been added to notification attributes to ensure
this doesn't suffer from the same failure.
* Add missing sleep in get_notification_attributes to avoid busy loop.
This change adds the optional wait_for_state_change argument to the
vmware_guest, vmware_guest_powerstate module, which allows for module
completion to be blocked when using the shutdownguest state until the
VM has reached the poweredoff state.
Fixes: #28498
Signed-off-by: Jim Gu <heming.gu@mercurygate.com>
Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>
* Adding support for vApp properties.
* vm specification updated only if changes have to be applied. I.e. subsequent updates with the same data will not trigger changed state
* Auxiliary variables renamed, hope this makes the code more readable
* Integration tests changed - re-adding the same properties test not implemented, but tested on real vCenter deployment
* fixing documentation "version_added" for the feature
* Addressing reviewers comments #2:
* documentation updated with the only meaningful value for "option" attribute - "remove"
* Fixed improperly handled case when user requested "add" operation for existent property
* vApp configuration is updated only with properties that contains changes, not with all properties requested by user
* aci_spvpg: Various fixes to integration tests/modules
This PR includes:
- A fix in module_utils aci.py
- Various fixes in integration tests
* Fix typo
* Add AWSRetry when describing VPCs to help stabilize integration tests
* Add retry on create_tags because it is possible to reach this API call before the VPC is finished creating
* now get_url and other modules default to module temp dir
also fixed 'bare' exception
* allow modules to work with older versions
* updated docs per feedback
* Bug in del(list) logic. Say you have a list of 4 elements a[0-3]
and you have to remove index 1 and 3, if you remove index 1 first
then list is cut short a[0-2] and does not have index 3
Fix: Remove indexes in reverse sorted order e.g. 3 and 1 in above
example so that order of indexes remain preserved even after deleting
fix is to remove indexes in reverse sorted order
* Add test cases for failed case
In short, it enables portage module to emit emerge
command with --jobs and --load-average options
with and without argument.
To emit emerge with such CLI argument without
a value, the user must set the corresponding
module option to 0.
By default, if these arguments are missing from
playbook they are omitted.
Misc changes:
* Use to_native to ensure crosspython compat
* Adjust jobs and LA options to accept 0 as reset
* Add docstring to emerge_packages
* Explicitly note in doc that False will work as 0
PR #36355 by @webknjaz
