Adds the non-deprecated ansible_host var (#34452)

OpenStack dynamic inventory is still using the deprecated
ansible_ssh_host. This patch adds ansible_host until such
time as ansible_ssh_host is removed

This is a backport from [1].

[1]: https://github.com/ansible/ansible/pull/23153

.gitignore

@@ -2,9 +2,10 @@
 *.py[co]
 build
 AUTHORS.TXT
-# Emacs backup files...
+# Emacs backup and autosave files...
 *~
 .\#*
+\#*
 # RPM stuff...
 MANIFEST
 dist

CHANGELOG.md

@@ -1,15 +1,179 @@
 Ansible Changes By Release
 ==========================
 
-## 2.3 TBD - ACTIVE DEVELOPMENT
+<a id="2.3.4"></a>
+
+## 2.3.4 "Ramble On" - TBD
+* Flush stdin when passing the become password.  Fixes some cases of timeout on
+  Python3 with the ssh connection plugin: https://github.com/ansible/ansible/pull/35049
+
+### Bugfixes
+* Fix setting of environment in a task that uses a loop:
+  https://github.com/ansible/ansible/issues/32685
+* Fix https retrieval with TLSv1.2: https://github.com/ansible/ansible/pull/32053
+
+
+<a id="2.3.3"></a>
+
+## 2.3.3 "Ramble On" - TBD
+
+### Bugfixes
+* Security fix for CVE-2017-7550 the jenkins_plugin module was logging the jenkins
+  server password if the url_password was passed via the params field:
+  https://github.com/ansible/ansible/pull/30875
+* Fix alternatives module handlling of non existing options
+* Fix synchronize traceback with the docker connection plugin
+* Do not escape backslashes in the template lookup plugin to mirror what the template module does
+* Fix the expires option of the postgresq_user module
+* Fix for win_acl when settings permissions on registry objects that use `ALL APPLICATION PACKAGES` and `ALL RESTRICTED APPLICATION PACKAGES`
+* Python3 fixes
+  * asorted azure modules
+  * pause module
+  * hacking/env-setup script
+  * Fix traceback when checking for passwords in logged strings when logging executed commands.
+  * docker_login module
+  * Workaround python-libselinux API change in the seboolean module
+  * digital_ocean_tag module
+  * Fix the zip filter
+  * Fix user module combining bytes and text
+  * Fix for security groups in the amazon efs module
+  * Fix for the jail connection plugin not finding the named jail
+  * Fix for blockinfile's parameters insertbefore and insertafter
+* ios_config: Fix traceback when the efaults parameter is not set
+* iosxr_config: Fixed unicode error when UTF-8 characters are in configs
+* Fix check mode in archive module
+* Fix UnboundLocalError in check mode in cs_role module
+* Fix to always use lowercase hostnames for host keys in known_hosts module
+* Added missing return results for win_stat
+* Fix rabbitmq modules to give a helpful error if requests is not installed
+* Fix yum module not deleting rpms that it downloaded
+* Fix yum module failing with a URL to an rpm
+* Fix file module inappropriately expanding literal dollar signs in a path read
+  from the filesystem as an environment variable.
+* Fix the ssh "smart" transport setting which automatically selects the best means of
+  transferring files over ssh (sftp, ssh, piped).
+* Fix authentication by api_key parameter in exoscale modules.
+* vmware module_utils shared code ssl/validate_certs fixes in connection logic
+* allow 'bridge' facts to work for certain containers that create conflicting ones with connection plugins
+* Fix for win_get_url to use TLS 1.2/1.1 if it is available on the host
+* Fix for the filetree lookup with non-ascii group names
+* Better message for invalid keywords/options in task due to undefined expressions
+* Fixed check mode for enable on Solaris for service module
+* Fix cloudtrail module to allow AWS profiles other than the default
+* Fix an encoding issue with secret (password) vars_prompts
+* Fix for Windows become to show the stdout and stderr strings on a failure 
+* Fix the issue SSL verification can not be disabled for Tower modules
+* Use safe_load instead on load to read a yaml document
+* Fix for win_file to respect check mode when deleting directories
+* Include_role now complains about invalid arguments
+* Added socket conditions to ignore for wait_for, no need to error for closing already closed connection
+* Updated hostname module to work on newer RHEL7 releases
+* Security fix to avoid provider password leaking in logs for network modules
+
+<a id="2.3.2"></a>
+* Python3 fixes for azure modules
+
+## 2.3.2 "Ramble On" - 2017-08-04
+
+### Bugfixes
+* Fix partend i18n issues
+* fixed handling of extra vars for tower_job_template (#25272)
+* Python3 bugfixes
+  * Fix sorting of ec2 policies
+  * Fix digital_ocean dynamic inventory script
+  * Fix for the docker connection plugin
+  * Fix pip module when using python3's pyvenv and python3 -m venv to create virtualenvs
+* Fix for the AnsiBallZ wrapper so that it gives a better error message when
+  there's not enough disk space to create its tempdir.
+* Fix so ansilbe-galaxy install --force with unversioned roles will once again
+  overwrite old versions.
+* Fix for RabbitMQ 3.6.7 endpoint return code changing.
+* Fix for Foreman organization creation
+* fixed incorrect fail_json ref in rpm_key
+* Corrected requried on hash_name for dynamodb_table
+* Fix for fetch action plugin not validating correctly
+* Avoid vault view writing display to logs
+* htpasswd: fix passlib module version comparison
+* Fix for flowdock error message when external_user_name is missing
+* fixed corner case for delegate_to, loops and delegate_facts
+* fixed wait_for python2.4/2.5 compatibility (this is last version this is needed)
+* fix for adhoc not obeying callback options
+* fix for win_find where it fails to recursively scan empty nested directories
+* fix non-pipelined code paths for Windows (eg, ANSIBLE_KEEP_REMOTE_FILES, non-pipelined connection plugins)
+* fix for win_updates where args and check mode were ignored due to common code change
+* fix for unprivileged users to Windows runas become method
+* fix starttls code path for mail module
+* fix missing LC_TYPE in parted module
+* fix CN parsing with OpenSSL 1.1 in letsencrypt module
+* fix params assignment in jabber module
+* fix TXT record type handling in exo_dns_record module
+* fix message queue message ttl can't be 0 in rabbitmq_queue module
+* CloudStack bugfixes:
+  * fix template upload for users in cs_template module, change default to is_routing=None
+  * several fixes in cs_host module fixes hypervisor handling
+  * fix network param ignored due typo in cs_nic module
+  * fix missing type bool in module cs_zone
+  * fix KeyError: 'sshkeypair' in cs_instance module for CloudStack v4.5 and before
+* fix for win_chocolatey where trying to upgrade all the packages as per the example docs fails
+* fix for win_chocolatey where it did not fail if the version set did not exist
+* fix for win_regedit always changing a reg key if the dword values set is a hex
+* fix for wait_for on non-Linux systems with newer versions of psutil
+* fix eos_banner code and test issues
+* run tearup and teardown of EAPI service only on EAPI tests
+* fix eos_config tests so only Eth1 and Eth2 are used
+* Fix for potential bug when using legacy inventory vars for configuring the su password.
+* Fix crash in file module when directories contain non-utf8 filenames
+* Fix for dnf groupinstall with dnf-2.x
+* Fix seboolean module for incompatibility in newer Python3 selinux bindings
+* Optimization for inventory, no need to dedup at every stage, its redundant and slow
+* Fix fact gathering for package and service action plugins
+* make random_choice more error resilient (#27380)
+* ensure prefix in plugin loading to avoid conflicts
+* fix for a small number of modules (tempfile, possibly copy) which could fail
+  if the tempdir on the remote box was a symlink
++ fix non-pipelined code paths for Windows (eg, ANSIBLE_KEEP_REMOTE_FILES, non-pipelined connection plugins)
+* fix for win_updates where args and check mode were ignored due to common code change
+
+<a id="2.3.1"></a>
+
+## 2.3.1 "Ramble On" - 2017-06-01
+
+
+### Bugfixes
+* Security fix for CVE-2017-7481 - data for lookup plugins used as variables was not being correctly marked as "unsafe".
+* Fix default value of fetch module's validate_checksum to be True
+* Added fix for "meta: refresh_connection" not working with default 'smart' connection.
+* Fix template so that the --diff command line option works when the destination is a directory
+* Fix python3 bugs in pam_limits
+* Fix unbound error when using module deprecation as a single string
+* Several places in which error handling was broken due to bad conversions or just typos
+* Fix to user module for appending/setting groups on OpenBSD (flags were reversed)
+* assemble fix to use safer os.join.path, avoids charset issues
+* fixed issue with solaris facts and i18n
+* added python2.4 compatiblity fix to sysctl module
+* Fix comparison of exisiting container security opts in the docker_container module
+* fixed service module invocation of insserv on certain platforms
+* Fix traceback in os_user in an error case.
+* Fix docker container to restart a container when changing to fewer exposed ports
+* Fix tracebacks in docker_network
+* Fixes to detection of updated docker images
+* Handle detection of docker image changes when published ports is changed
+* Fix for docker_container restarting images when links list is empty.
+
+<a id="2.3"></a>
+
+## 2.3 "Ramble On" - 2017-04-12
+
+Moving to Ansible 2.3 guide http://docs.ansible.com/ansible/porting_guide_2.3.html
 
 ### Major Changes
 * Documented and renamed the previously released 'single var vaulting' feature, allowing user to use vault encryption for single variables in a normal YAML vars file.
 * Allow module_utils for custom modules to be placed in site-specific directories and shipped in roles
 * On platforms that support it, use more modern system polling API instead of select in the ssh connection plugin.
   This removes one limitation on how many parallel forks are feasible on these systems.
-* Windows supports become method "runas" to run modules as a different user, and to transparently access network resources.
-* Windows now uses pipelining when executing modules, resulting in significantly faster execution for small tasks.
+* Windows/WinRM supports (experimental) become method "runas" to run modules and scripts as a different user, and to transparently access network resources.
+* The WinRM connection plugin now uses pipelining when executing modules, resulting in significantly faster execution for small tasks.
+* The WinRM connection plugin can now manage Kerberos tickets automatically when `ansible_winrm_transport=kerberos` and `ansible_user`/`ansible_password` are specified.
 * Refactored/standardized most Windows modules, adding check-mode and diff support where possible.
 * Extended Windows module API with parameter-type support, helper functions. (i.e. Expand-Environment, Add-Warning, Add-DeprecatationWarning)
 * restructured how async works to allow it to apply to action plugins that choose to support it.
@@ -31,6 +195,7 @@ Ansible Changes By Release
 * 'service' tasks can now use async again, we had lost this capability when changed into an action plugin.
 * made any_errors_fatal inheritable from play to task and all other objects in between.
 * many small performance improvements in inventory and variable handling and in task execution.
+* Added a retry class to the ec2_asg module since customers were running into throttling errors (AWSRetry is a solution for modules using boto3 which isn't applicable here).
 
 ### Deprecations
 * Specifying --tags (or --skip-tags) multiple times on the command line
@@ -165,6 +330,7 @@ Ansible Changes By Release
   * ipa_user
 - ipinfoio_facts
 - ios:
+  * ios_banner
   * ios_system
   * ios_vrf
 - iosxr_system
@@ -258,6 +424,14 @@ Ansible Changes By Release
   * panos_service
 - postgresql_schema
 - proxmox_kvm
+- proxysql:
+  * proxysql_backend_servers
+  * proxysql_global_variables
+  * proxysql_manage_config
+  * proxysql_mysql_users
+  * proxysql_query_rules
+  * proxysql_replication_hostgroups
+  * proxysql_scheduler
 - pubnub_blocks
 - pulp_repo
 - runit
@@ -311,6 +485,7 @@ Ansible Changes By Release
   * zfs_facts
   * zpool_facts
 
+<a id="2.2.1"></a>
 
 ## 2.2.1 "The Battle of Evermore" - 2017-01-16
 
@@ -337,10 +512,11 @@ Ansible Changes By Release
 * Improvements and fixes to OpenBSD fact gathering.
 * Updated `make deb` to use pbuilder. Use `make local_deb` for the previous non-pbuilder build.
 * Fixed Windows async to avoid blocking due to handle inheritance.
-* Fixed bugs in the mount module on older Linux kernels and *BSDs
+* Fixed bugs in the mount module on older Linux kernels and BSDs
 * Various minor fixes for Python 3
 * Inserted some checks for jinja2-2.9, which can cause some issues with Ansible currently.
 
+<a id="2.2"></a>
 
 ## 2.2 "The Battle of Evermore" - 2016-11-01
 
@@ -655,12 +831,16 @@ Notice given that the following will be removed in Ansible 2.4:
   * nxos_template
   * ops_template
 
+<a id="2.1.4"></a>
+
 ## 2.1.4 "The Song Remains the Same" - 2017-01-16
 
 * Security fix for CVE-2016-9587 - An attacker with control over a client system being managed by Ansible and the ability to send facts back to the Ansible server could use this flaw to execute arbitrary code on the Ansible server as the user and group Ansible is running as.
 * Fixed a bug with conditionals in loops, where undefined variables and other errors will defer raising the error until the conditional has been evaluated.
 * Added a version check for jinja2-2.9, which does not fully work with Ansible currently.
 
+<a id="2.1.3"></a>
+
 ## 2.1.3 "The Song Remains the Same" - 2016-11-04
 
 * Security fix for CVE-2016-8628 - Command injection by compromised server via fact variables. In some situations, facts returned by modules could overwrite connection-based facts or some other special variables, leading to injected commands running on the Ansible controller as the user running Ansible (or via escalated permissions).
@@ -673,11 +853,13 @@ Notice given that the following will be removed in Ansible 2.4:
   login_password as no_log so the password is obscured when logging.
 * Fixed several bugs related to locating files relative to role/playbook directories.
 * Fixed a bug in the way hosts were tested for failed states, resulting in incorrectly skipped block sessions.
-* Fixed a bug in the way our custom JSON encoder is used for the to_json* filters.
+* Fixed a bug in the way our custom JSON encoder is used for the `to_json*` filters.
 * Fixed some bugs related to the use of non-ascii characters in become passwords.
 * Fixed a bug with Azure modules which may be using the latest rc6 library.
 * Backported some docker_common fixes.
 
+<a id="2.1.2"></a>
+
 ## 2.1.2 "The Song Remains the Same" - 2016-09-29
 
 ### Minor Changes
@@ -740,6 +922,8 @@ Module fixes:
   Use `_fixup_perms2` if support for previous releases is not required.
   Otherwise use `_fixup_perms` with `recursive=False`.
 
+<a id="2.1"></a>
+
 ## 2.1 "The Song Remains the Same"
 
 ###Major Changes:
@@ -927,6 +1111,8 @@ Module fixes:
   completely in 2.3, after which time it will be an error.
 * play_hosts magic variable, use ansible_play_batch or ansible_play_hosts instead.
 
+<a id="2.0.2"></a>
+
 ## 2.0.2 "Over the Hills and Far Away"
 
 * Backport of the 2.1 feature to ensure per-item callbacks are sent as they occur,
@@ -968,10 +1154,12 @@ Module fixes:
   permissions on the temporary file too leniently on a temporary file that was
   executed as a script.  Addresses CVE-2016-3096
 * Fix a bug in the uri module where setting headers via module params that
-  start with HEADER_ were causing a traceback.
+  start with `HEADER_` were causing a traceback.
 * Fix bug in the free strategy that was causing it to synchronize its workers
   after every task (making it a lot more like linear than it should have been).
 
+<a id="2.0.1"></a>
+
 ## 2.0.1 "Over the Hills and Far Away"
 
 * Fixes a major compatibility break in the synchronize module shipped with
@@ -1013,6 +1201,8 @@ Module fixes:
   this might cause an error if settings environment on play depending on 'ansible_env'
   which was previouslly ignored
 
+<a id="2.0"></a>
+
 ## 2.0 "Over the Hills and Far Away" - Jan 12, 2016
 
 ###Major Changes:

MANIFEST.in

@@ -2,6 +2,8 @@ prune ticket_stubs
 prune hacking
 include README.md COPYING
 include requirements.txt
+include .coveragerc
+include .yamllint
 include examples/hosts
 include examples/ansible.cfg
 include lib/ansible/module_utils/powershell.ps1

Makefile

@@ -92,6 +92,10 @@ RPMRELEASE = $(RELEASE)
 ifneq ($(OFFICIAL),yes)
     RPMRELEASE = 100.git$(DATE)$(GITINFO)
 endif
+ifeq ($(PUBLISH),nightly)
+    # https://fedoraproject.org/wiki/Packaging:Versioning#Snapshots
+    RPMRELEASE = $(RELEASE).$(DATE)git.$(GIT_HASH)
+endif
 RPMNVR = "$(NAME)-$(VERSION)-$(RPMRELEASE)$(RPMDIST)"
 
 # MOCK build parameters
@@ -193,17 +197,17 @@ sdist_upload: clean docs
 rpmcommon: $(MANPAGES) sdist
 	@mkdir -p rpm-build
 	@cp dist/*.gz rpm-build/
-	@sed -e 's#^Version:.*#Version: $(VERSION)#' -e 's#^Release:.*#Release: $(RPMRELEASE)%{?dist}#' $(RPMSPEC) >rpm-build/$(NAME).spec
+	@sed -e 's#^Version:.*#Version: $(VERSION)#' -e 's#^Release:.*#Release: $(RPMRELEASE)%{?dist}$(REPOTAG)#' $(RPMSPEC) >rpm-build/$(NAME).spec
 
 mock-srpm: /etc/mock/$(MOCK_CFG).cfg rpmcommon
-	$(MOCK_BIN) -r $(MOCK_CFG) --resultdir rpm-build/  --buildsrpm --spec rpm-build/$(NAME).spec --sources rpm-build/
+	$(MOCK_BIN) -r $(MOCK_CFG) $(MOCK_ARGS) --resultdir rpm-build/  --buildsrpm --spec rpm-build/$(NAME).spec --sources rpm-build/
 	@echo "#############################################"
 	@echo "Ansible SRPM is built:"
 	@echo rpm-build/*.src.rpm
 	@echo "#############################################"
 
 mock-rpm: /etc/mock/$(MOCK_CFG).cfg mock-srpm
-	$(MOCK_BIN) -r $(MOCK_CFG) --resultdir rpm-build/ --rebuild rpm-build/$(NAME)-*.src.rpm
+	$(MOCK_BIN) -r $(MOCK_CFG) $(MOCK_ARGS) --resultdir rpm-build/ --rebuild rpm-build/$(NAME)-*.src.rpm
 	@echo "#############################################"
 	@echo "Ansible RPM is built:"
 	@echo rpm-build/*.noarch.rpm
@@ -303,3 +307,7 @@ webdocs:
 docs: $(MANPAGES)
 
 alldocs: docs webdocs
+
+version:
+	@echo $(VERSION)
+

RELEASES.txt

@@ -1,81 +1,88 @@
 Ansible Releases at a Glance
 ============================
 
-For more info http://docs.ansible.com/ansible/dev_guide/developing_releases.html
-
-Active Development
-++++++++++++++++++
-
-2.3    TBD - in progress
-2.2    "The Battle of Evermore" - maintenance
-
-Released
-++++++++
-
-2.2.0    "The Battle of Evermore"	11-01-2016
-2.1.3  "The Song Remains the Same"  11-04-2016
-2.1.2  "The Song Remains the Same"  10-29-2016
-2.1.1  "The Song Remains the Same"  07-28-2016
-2.1.0  "The Song Remains the Same"  05-25-2016
-2.0.2  "Over the Hills and Far Away" 04-19-2016
-2.0.1  "Over the Hills and Far Away" 02-24-2016
-2.0.0  "Over the Hills and Far Away" 01-12-2016
-1.9.6  "Dancing In the Streets" 04-15-2016
-1.9.5  "Dancing In the Streets" 03-21-2016
-1.9.4  "Dancing In the Streets" 10-09-2015
-1.9.3  "Dancing In the Streets" 09-03-2015
-1.9.2  "Dancing In the Streets" 06-24-2015
-1.9.1  "Dancing In the Streets" 04-27-2015
-1.9.0  "Dancing In the Streets" 03-25-2015
-1.8.4  "You Really Got Me" ---- 02-19-2015
-1.8.3  "You Really Got Me" ---- 02-17-2015
-1.8.2  "You Really Got Me" ---- 12-04-2014
-1.8.1  "You Really Got Me" ---- 11-26-2014
-1.7.2  "Summer Nights" -------- 09-24-2014
-1.7.1  "Summer Nights" -------- 08-14-2014
-1.7    "Summer Nights" -------- 08-06-2014
-1.6.10 "The Cradle Will Rock" - 07-25-2014
-1.6.9  "The Cradle Will Rock" - 07-24-2014
-1.6.8  "The Cradle Will Rock" - 07-22-2014
-1.6.7  "The Cradle Will Rock" - 07-21-2014
-1.6.6  "The Cradle Will Rock" - 07-01-2014
-1.6.5  "The Cradle Will Rock" - 06-25-2014
-1.6.4  "The Cradle Will Rock" - 06-25-2014
-1.6.3  "The Cradle Will Rock" - 06-09-2014
-1.6.2  "The Cradle Will Rock" - 05-23-2014
-1.6.1  "The Cradle Will Rock" - 05-07-2014
-1.6    "The Cradle Will Rock" - 05-05-2014 
-1.5.5  "Love Walks In" -------- 04-18-2014
-1.5.4  "Love Walks In" -------- 04-01-2014
-1.5.3  "Love Walks In" -------- 03-13-2014
-1.5.2  "Love Walks In" -------- 03-11-2014
-1.5.1  "Love Walks In" -------- 03-10-2014
-1.5    "Love Walks In" -------- 02-28-2014
-1.4.5  "Could This Be Magic?" - 02-12-2014
-1.4.4  "Could This Be Magic?" - 01-06-2014
-1.4.3  "Could This Be Magic?" - 12-20-2013
-1.4.2  "Could This Be Magic?" - 12-18-2013
-1.4.1  "Could This Be Magic?" - 11-27-2013
-1.4    "Could This Be Magic?" - 11-21-2013
-1.3.4  "Top of the World" ----- 10-29-2013
-1.3.3  "Top of the World" ----- 10-09-2013
-1.3.2  "Top of the World" ----- 09-19-2013
-1.3.1  "Top of the World" ----- 09-16-2013
-1.3    "Top of the World" ----- 09-13-2013
-1.2.3  "Hear About It Later" -- 08-21-2013
-1.2.2  "Hear About It Later" -- 07-05-2013
-1.2.1  "Hear About It Later" -- 07-04-2013
-1.2    "Right Now" ------------ 06-10-2013
-1.1    "Mean Street" ---------- 04-02-2013
-1.0    "Eruption" ------------- 02-01-2013
-0.9    "Dreams" --------------- 11-30-2012
-0.8    "Cathedral" ------------ 10-19-2012
-0.7    "Panama" --------------- 09-06-2012
-0.6    "Cabo" ----------------- 08-06-2012
-0.5    "Amsterdam" ------------ 07-04-2012
-0.4    "Unchained" ------------ 05-23-2012
-0.3    "Baluchitherium" ------- 04-23-2012
-0.0.2  Untitled
-0.0.1  Untitled
-
+VERSION  RELEASE     CODE NAME
+++++++++++++++++++++++++++++++
 
+2.5.0    TBD         "Kashmir"
+2.4.2    11-29-2017  "Dancing Days"
+2.4.1    10-25-2017  "Dancing Days"
+2.4.0    09-19-2017  "Dancing Days"
+2.3.3    12-20-2017  "Ramble On"
+2.3.2    08-04-2017  "Ramble On"
+2.3.1    06-01-2017  "Ramble On"
+2.3.0    04-12-2017  "Ramble On"
+2.2.3    05-09-2017  "The Battle of Evermore"
+2.2.2    03-27-2017  "The Battle of Evermore"
+2.2.1    01-16-2017  "The Battle of Evermore"
+2.2.0    11-01-2016  "The Battle of Evermore"
+2.1.6    06-01-2017  "The Song Remains the Same"
+2.1.5    03-27-2017  "The Song Remains the Same"
+2.1.4    01-16-2017  "The Song Remains the Same"
+2.1.3    11-04-2016  "The Song Remains the Same"
+2.1.2    09-29-2016  "The Song Remains the Same"
+2.1.1    07-28-2016  "The Song Remains the Same"
+2.1.0    05-25-2016  "The Song Remains the Same"
+2.0.2    04-19-2016  "Over the Hills and Far Away"
+2.0.1    02-24-2016  "Over the Hills and Far Away"
+2.0.0    01-12-2016  "Over the Hills and Far Away"
+1.9.6    04-15-2016  "Dancing In the Streets"
+1.9.5    03-21-2016  "Dancing In the Streets"
+1.9.4    10-09-2015  "Dancing In the Streets"
+1.9.3    09-03-2015  "Dancing In the Streets"
+1.9.2    06-24-2015  "Dancing In the Streets"
+1.9.1    04-27-2015  "Dancing In the Streets"
+1.9.0    03-25-2015  "Dancing In the Streets"
+1.8.4    02-19-2015  "You Really Got Me"
+1.8.3    02-17-2015  "You Really Got Me"
+1.8.2    12-04-2014  "You Really Got Me"
+1.8.1    11-26-2014  "You Really Got Me"
+1.8.0    11-25-2014  "You Really Got Me"
+1.7.2    09-24-2014  "Summer Nights"
+1.7.1    08-14-2014  "Summer Nights"
+1.7.0    08-06-2014  "Summer Nights"
+1.6.10   07-25-2014  "The Cradle Will Rock"
+1.6.9    07-24-2014  "The Cradle Will Rock"
+1.6.8    07-22-2014  "The Cradle Will Rock"
+1.6.7    07-21-2014  "The Cradle Will Rock"
+1.6.6    07-01-2014  "The Cradle Will Rock"
+1.6.5    06-25-2014  "The Cradle Will Rock"
+1.6.4    06-25-2014  "The Cradle Will Rock"
+1.6.3    06-09-2014  "The Cradle Will Rock"
+1.6.2    05-23-2014  "The Cradle Will Rock"
+1.6.1    05-07-2014  "The Cradle Will Rock"
+1.6.0    05-05-2014  "The Cradle Will Rock"
+1.5.5    04-18-2014  "Love Walks In"
+1.5.4    04-01-2014  "Love Walks In"
+1.5.3    03-13-2014  "Love Walks In"
+1.5.2    03-11-2014  "Love Walks In"
+1.5.1    03-10-2014  "Love Walks In"
+1.5.0    02-28-2014  "Love Walks In"
+1.4.5    02-12-2014  "Could This Be Magic?"
+1.4.4    01-06-2014  "Could This Be Magic?"
+1.4.3    12-20-2013  "Could This Be Magic?"
+1.4.2    12-18-2013  "Could This Be Magic?"
+1.4.1    11-27-2013  "Could This Be Magic?"
+1.4.0    11-21-2013  "Could This Be Magic?"
+1.3.4    10-29-2013  "Top of the World"
+1.3.3    10-09-2013  "Top of the World"
+1.3.2    09-19-2013  "Top of the World"
+1.3.1    09-16-2013  "Top of the World"
+1.3.0    09-13-2013  "Top of the World"
+1.2.3    08-21-2013  "Right Now"
+1.2.2    07-05-2013  "Right Now"
+1.2.1    07-04-2013  "Right Now"
+1.2.0    06-10-2013  "Right Now"
+1.1.0    04-02-2013  "Mean Street"
+1.0.0    02-01-2013  "Eruption"
+0.9.0    11-30-2012  "Dreams"
+0.8.0    10-19-2012  "Cathedral"
+0.7.0    09-06-2012  "Panama"
+0.6.0    08-06-2012  "Cabo"
+0.5.0    07-04-2012  "Amsterdam"
+0.4.0    05-23-2012  "Unchained"
+0.3.0    04-23-2012  "Baluchitherium"
+0.2.0    ?           "Untitled"
+0.1.0    ?           "Untitled"
+0.0.2    ?           "Untitled"
+0.0.1    ?           "Untitled"

ROADMAP.rst

@@ -9,7 +9,7 @@ The roadmap will be updated by version. Based on team and community feedback, an
 
 These roadmaps are the team's *best guess* roadmaps based on the Ansible team's experience and are also based on requests and feedback from the community.  There are things that may not make it on due to time constraints, lack of community maintainers, etc.  And there may be things that got missed, so each roadmap is published both as an idea of what is upcoming in Ansible, and as a medium for seeking further feedback from the community. Here are the good places for you to submit feedback:
 
-- Add to the agenda of a `Core IRC Meetings <https://github.com/ansible/community/blob/master/MEETINGS.md>`_ (preferred)
+- Add to the agenda of a `Core IRC Meetings <https://github.com/ansible/community/blob/master/meetings/README.md>`_ (preferred)
 - Ansible's google-group: ansible-devel
 - AnsibleFest conferences.
 - IRC freenode channel: #ansible-devel (this one may have things lost in lots of conversation, so a caution).

VERSION

@@ -1 +1 @@
-2.3.0 0.0.devel
+2.3.4.0 0.1.rc1

bin/ansible-connection

@@ -39,6 +39,7 @@ import time
 import traceback
 import syslog
 import datetime
+import logging
 
 from io import BytesIO
 
@@ -62,7 +63,7 @@ def do_fork():
         if pid > 0:
             return pid
 
-        os.chdir("/")
+        #os.chdir("/")
         os.setsid()
         os.umask(0)
 
@@ -109,25 +110,34 @@ def recv_data(s):
         data += d
     return data
 
+
 class Server():
 
     def __init__(self, path, play_context):
-        display.vvvv("starting new persistent socket with path %s" % path, play_context.remote_addr)
 
         self.path = path
         self.play_context = play_context
 
+        display.display(
+            'creating new control socket for host %s:%s as user %s' %
+            (play_context.remote_addr, play_context.port, play_context.remote_user),
+            log_only=True
+        )
+
+        display.display('control socket path is %s' % path, log_only=True)
+        display.display('current working directory is %s' % os.getcwd(), log_only=True)
+
         self._start_time = datetime.datetime.now()
 
-        display.vvv("using connection plugin %s" % self.play_context.connection, play_context.remote_addr)
+        display.display("using connection plugin %s" % self.play_context.connection, log_only=True)
 
         self.conn = connection_loader.get(play_context.connection, play_context, sys.stdin)
         self.conn._connect()
         if not self.conn.connected:
-            raise AnsibleConnectionFailure('unable to connect to remote host')
+            raise AnsibleConnectionFailure('unable to connect to remote host %s' % self._play_context.remote_addr)
 
         connection_time = datetime.datetime.now() - self._start_time
-        display.vvvv('connection established in %s' % connection_time, play_context.remote_addr)
+        display.display('connection established to %s in %s' % (play_context.remote_addr, connection_time), log_only=True)
 
         self.socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
         self.socket.bind(path)
@@ -165,7 +175,7 @@ class Server():
                 signal.alarm(C.PERSISTENT_CONNECT_TIMEOUT)
                 try:
                     (s, addr) = self.socket.accept()
-                    display.vvvv('incoming request accepted on persistent socket', self.play_context.remote_addr)
+                    display.display('incoming request accepted on persistent socket', log_only=True)
                     # clear the alarm
                     # FIXME: potential race condition here between the accept and
                     #        time to this call.
@@ -177,14 +187,13 @@ class Server():
                     data = recv_data(s)
                     if not data:
                         break
-                    display.vvvv("received data on socket with len %s" % len(data), self.play_context.remote_addr)
 
-                    signal.alarm(C.DEFAULT_TIMEOUT)
+                    signal.alarm(self.play_context.timeout)
 
                     rc = 255
                     try:
                         if data.startswith(b'EXEC: '):
-                            display.vvvv("socket operation is EXEC", self.play_context.remote_addr)
+                            display.display("socket operation is EXEC", log_only=True)
                             cmd = data.split(b'EXEC: ')[1]
                             (rc, stdout, stderr) = self.conn.exec_command(cmd)
                         elif data.startswith(b'PUT: ') or data.startswith(b'FETCH: '):
@@ -192,16 +201,16 @@ class Server():
                             stdout = stderr = ''
                             try:
                                 if op == 'FETCH:':
-                                    display.vvvv("socket operation is FETCH", self.play_context.remote_addr)
+                                    display.display("socket operation is FETCH", log_only=True)
                                     self.conn.fetch_file(src, dst)
                                 elif op == 'PUT:':
-                                    display.vvvv("socket operation is PUT", self.play_context.remote_addr)
+                                    display.display("socket operation is PUT", log_only=True)
                                     self.conn.put_file(src, dst)
                                 rc = 0
                             except:
                                 pass
                         elif data.startswith(b'CONTEXT: '):
-                            display.vvvv("socket operation is CONTEXT", self.play_context.remote_addr)
+                            display.display("socket operation is CONTEXT", log_only=True)
                             pc_data = data.split(b'CONTEXT: ')[1]
 
                             src = StringIO(pc_data)
@@ -214,7 +223,7 @@ class Server():
                             self.dispatch(self.conn, 'update_play_context', pc)
                             continue
                         else:
-                            display.vvvv("socket operation is UNKNOWN", self.play_context.remote_addr)
+                            display.display("socket operation is UNKNOWN", log_only=True)
                             stdout = ''
                             stderr = 'Invalid action specified'
                     except:
@@ -223,20 +232,20 @@ class Server():
 
                     signal.alarm(0)
 
-                    display.vvvv("socket operation completed with rc %s" % rc, self.play_context.remote_addr)
+                    display.display("socket operation completed with rc %s" % rc, log_only=True)
 
                     send_data(s, to_bytes(str(rc)))
                     send_data(s, to_bytes(stdout))
                     send_data(s, to_bytes(stderr))
                 s.close()
         except Exception as e:
-            display.vvvv(traceback.format_exc())
+            display.display(traceback.format_exec(), log_only=True)
         finally:
             # when done, close the connection properly and cleanup
             # the socket file so it can be recreated
             end_time = datetime.datetime.now()
             delta = end_time - self._start_time
-            display.v('shutting down control socket, connection was active for %s secs' % delta, self.play_context.remote_addr)
+            display.display('shutting down control socket, connection was active for %s secs' % delta, log_only=True)
             try:
                 self.conn.close()
                 self.socket.close()
@@ -268,8 +277,6 @@ def main():
         sys.stderr.write(traceback.format_exc())
         sys.exit("FAIL: %s" % e)
 
-    display.verbosity = pc.verbosity
-
     ssh = connection_loader.get('ssh', class_only=True)
     m = ssh._create_control_path(pc.remote_addr, pc.port, pc.remote_user)
 
@@ -290,16 +297,20 @@ def main():
             try:
                 server = Server(sf_path, pc)
             except AnsibleConnectionFailure as exc:
-                display.vvv(str(exc), pc.remote_addr)
+                display.display('connecting to host %s returned an error' % pc.remote_addr, log_only=True)
+                display.display(str(exc), log_only=True)
                 rc = 1
             except Exception as exc:
-                display.vvv(traceback.format_exc(), pc.remote_addr)
+                display.display('failed to create control socket for host %s' % pc.remote_addr, log_only=True)
+                display.display(traceback.format_exc(), log_only=True)
                 rc = 1
             fcntl.lockf(lock_fd, fcntl.LOCK_UN)
             os.close(lock_fd)
             if rc == 0:
                 server.run()
             sys.exit(rc)
+    else:
+        display.display('re-using existing socket for %s@%s:%s' % (pc.remote_user, pc.remote_addr, pc.port), log_only=True)
     fcntl.lockf(lock_fd, fcntl.LOCK_UN)
     os.close(lock_fd)
 
@@ -324,8 +335,8 @@ def main():
                 time.sleep(C.PERSISTENT_CONNECT_INTERVAL)
                 attempts += 1
                 if attempts > C.PERSISTENT_CONNECT_RETRIES:
-                    display.vvvv('number of connection attempts exceeded, unable to connect to control socket')
-                    display.vvvv('persistent_connect_interval=%s, persistent_connect_retries=%s' % (C.PERSISTENT_CONNECT_INTERVAL, C.PERSISTENT_CONNECT_RETRIES))
+                    display.display('number of connection attempts exceeded, unable to connect to control socket', pc.remote_addr, pc.remote_user, log_only=True)
+                    display.display('persistent_connect_interval=%s, persistent_connect_retries=%s' % (C.PERSISTENT_CONNECT_INTERVAL, C.PERSISTENT_CONNECT_RETRIES), pc.remote_addr, pc.remote_user, log_only=True)
                     sys.stderr.write('failed to connect to control socket')
                     sys.exit(255)
 

contrib/inventory/digital_ocean.py

@@ -138,9 +138,13 @@ import sys
 import re
 import argparse
 from time import time
-import ConfigParser
 import ast
 
+try:
+    import ConfigParser
+except ImportError:
+    import configparser as ConfigParser
+
 try:
     import json
 except ImportError:

contrib/inventory/docker.py

@@ -525,7 +525,7 @@ class AnsibleDockerClient(Client):
             msg = "You asked for verification that Docker host name matches %s. The actual hostname is %s. " \
                 "Most likely you need to set DOCKER_TLS_HOSTNAME or pass tls_hostname with a value of %s. " \
                 "You may also use TLS without verification by setting the tls parameter to true." \
-                % (self.auth_params['tls_hostname'], match.group(1))
+                % (self.auth_params['tls_hostname'], match.group(1), match.group(1))
             self.fail(msg)
         self.fail("SSL Exception: %s" % (error))
 

contrib/inventory/fleet.py

@@ -94,7 +94,6 @@ elif options.host:
     if len(details) > 0:
         #pass through the port, in case it's non standard.
         result = details[0]
-        result
 
     print(json.dumps(result))
     sys.exit(1)

contrib/inventory/linode.py

@@ -197,7 +197,7 @@ class LinodeInventory(object):
         try:
             return Linode.find(api_id=linode_id)
         except chube_api.linode_api.ApiError as e:
-            sys.exit("Looks like Linode's API is down:\n%" % e)
+            sys.exit("Looks like Linode's API is down:\n%s" % e)
 
     def populate_datacenter_cache(self):
         """Creates self._datacenter_cache, containing all Datacenters indexed by ID."""

contrib/inventory/openstack.py

@@ -121,6 +121,7 @@ def get_host_groups(inventory, refresh=False):
 def append_hostvars(hostvars, groups, key, server, namegroup=False):
     hostvars[key] = dict(
         ansible_ssh_host=server['interface_ip'],
+        ansible_host=server['interface_ip'],
         openstack=server)
     for group in get_groups_from_server(server, namegroup=namegroup):
         groups[group].append(key)

docs/docsite/_themes/srtd/layout.html

@@ -14,6 +14,8 @@
   <meta charset="utf-8">
   <meta name="viewport" content="width=device-width, initial-scale=1.0">
 
+  <script type="text/javascript" src="//www.redhat.com/dtm.js"></script>
+
   <!-- Google Tag Manager Data Layer -->
   <script>
    dataLayer = [];
@@ -149,18 +151,23 @@
 
   <div class="wy-grid-for-nav">
 
-    {# SIDE NAV, TOGGLES ON MOBILE #}
+    {# SIDE NAV TOGGLES ON MOBILE #}
     <nav data-toggle="wy-nav-shift" class="wy-nav-side">
 
       <div style="background-color:#5bbdbf;height=80px;margin:'auto auto auto auto'">
-        <a class="DocSiteProduct-header DocSiteProduct-header--core"
-          href="http://docs.ansible.com/ansible/">
-          <div class="DocSiteProduct-productName">
-            <div class="DocSiteProduct-logoText">
-              ANSIBLE
+        <a class="DocSiteProduct-header DocSiteProduct-header--core" href="http://docs.ansible.com">
+            <div class="DocSiteProduct-productName">
+                <div class="DocSiteProduct-logoText">
+                    Ansible Core   
+                    <div class="DocSiteProduct-CurrentVersion" align="right">
+                      v2.3
+                  </div>
+                </div>
             </div>
-          </div>
         </a>
+        
+
+          <div class="DocSiteProduct-CheckVersionPara">For previous versions, see the  <a class="DocSiteProduct-versionheader" href="http://docs.ansible.com/">documentation archive.</a></div> 
       </div>
 
       <div id="menu-id" class="wy-menu wy-menu-vertical" data-spy="affix">
@@ -246,6 +253,10 @@ _hsq.push(["setContentType", "standard-page"]);
         })(document, "script", "hs-analytics",300000);
 </script>
 <!-- end analytics -->
-
+<script type="text/javascript">
+if (("undefined" !== typeof _satellite) && ("function" === typeof _satellite.pageBottom)) {
+_satellite.pageBottom();
+}
+</script>
 </body>
 </html>

docs/docsite/_themes/srtd/static/css/theme.css

@@ -4715,6 +4715,13 @@ span[id*='MathJax-Span'] {
     font-size: 20px;
 }
 
+.DocSiteProduct-CheckVersionPara {
+    padding-bottom: 20px;
+    margin-left: 10px;
+    margin-right: 10px;
+}
+
+
 .wy-menu-vertical a {
     padding: 0;
 }

docs/docsite/rst/community.rst

@@ -29,8 +29,8 @@ We recommend using Google search to see if a topic has been answered recently, b
 
 Before you post, be sure you are running the latest stable version of Ansible.  You can check this by comparing the output of ``ansible --version`` with the version indicated on `PyPi <https://pypi.python.org/pypi/ansible>`_.
 
-Alternatively, you can also join our IRC channel - #ansible on irc.freenode.net.  It's a very high traffic channel as well, if you don't get an answer you like, please stop by our mailing list, which is more likely
-to get attention of core developers since it's asynchronous.
+Alternatively, you can also join our IRC channel - ``#ansible`` on irc.freenode.net.  It's a very high traffic channel as well, so if you don't get an answer you like, please stop by our mailing list, which is more likely
+to get the attention of core developers since it's asynchronous.
 
 I'd Like To Keep Up With Release Announcements
 ----------------------------------------------
@@ -102,7 +102,9 @@ documenting a new feature, submit a GitHub pull request to  the code that
 lives in the ``docsite/rst`` subdirectory of the project for most pages, and there is an "Edit on GitHub"
 link up on those.
 
-Module documentation is generated from a DOCUMENTATION structure embedded in the source code of each module, which is in `/lib/ansible/modules/ <https://github.com/ansible/ansible/tree/devel/lib/ansible/modules/>`_.
+Module documentation is generated from a ``DOCUMENTATION`` structure embedded in the source code of each module, which is in `/lib/ansible/modules/ <https://github.com/ansible/ansible/tree/devel/lib/ansible/modules/>`_.
+
+For more information on module documentation see `How to document modules <http://docs.ansible.com/ansible/dev_guide/developing_modules_documenting.html>`_.
 
 Aside from modules, the main docs are in restructured text
 format.  
@@ -110,7 +112,7 @@ format.
 If you aren’t comfortable with restructured text, you can also open a ticket on 
 GitHub about any errors you spot or sections you would like to see added. For more information
 on creating pull requests, please refer to the
-`github help guide <https://help.github.com/articles/using-pull-requests>`_.
+`GitHub help guide <https://help.github.com/articles/using-pull-requests>`_.
 
 For Current and Prospective Developers
 =======================================
@@ -141,14 +143,9 @@ Note that we do keep Ansible to a particular aesthetic, so if you are unclear ab
 is a good fit or not, having the discussion on the development list is often a lot easier than having
 to modify a pull request later.
 
-New module developers should read through `developing modules <http://docs.ansible.com/ansible/dev_guide/developing_modules.html>`_ for helpful pointers
-and information about running adhoc tests `testing modules <http://docs.ansible.com/ansible/dev_guide/developing_modules.html#testing-modules>`_.
+New module developers should read through `developing modules <http://docs.ansible.com/ansible/dev_guide/developing_modules.html>`_.
 
-When submitting patches, be sure to run the unit tests first ``make tests`` and always use, these are the same basic
-tests that will automatically run on Shippable when creating the PR. There are more in depth tests in the ``tests/integration``
-directory, classified as destructive and non_destructive, run these if they pertain to your modification. They are set up
-with tags so you can run subsets, some of the tests require cloud credentials and will only run if they are provided.
-When adding new features or fixing bugs it would be nice to add new tests to avoid regressions. For more information about testing see `test/README.md <https://github.com/ansible/ansible/blob/devel/test/README.md>`_.
+For more information about testing see `testing <http://docs.ansible.com/ansible/dev_guide/testing.html>`_.
 
 In order to keep the history clean and better audit incoming code, we will require resubmission of pull requests that
 contain merge commits.  Use ``git pull --rebase`` (rather than ``git pull``) and ``git rebase`` (rather than ``git merge``). Also be sure to use topic
@@ -174,11 +171,15 @@ Contributions can be for new features like modules, or to fix bugs you or others
 are interested in writing new modules to be included in the core Ansible distribution, please refer
 to the `module development documentation <http://docs.ansible.com/developing_modules.html>`_.
 
-Ansible's aesthetic encourages simple, readable code and consistent, conservatively extending,
-backwards-compatible improvements.  Code developed for Ansible needs to support Python 2.6+,
-while code in modules must run under Python 2.4 or higher.  Please also use a 4-space indent
-and no tabs, we do not enforce 80 column lines, we are fine with 120-140. We do not take 'style only'
-requests unless the code is nearly unreadable, we are "PEP8ish", but not strictly compliant.
+Ansible's aesthetic encourages simple, readable code and consistent,
+conservatively extending, backwards-compatible improvements. When contributing code to Ansible, 
+please observe the following guidelines:
+
+- Code developed for Ansible needs to support Python2-2.6 or higher and Python3-3.5 or higher.
+- Use a 4-space indent, not  tabs.
+- We do not enforce 80 column lines; up to 160 columns are acceptable.
+- We do not accept 'style only' pull requests unless the code is nearly unreadable.
+- We are "PEP8ish", but not strictly compliant, see :doc:`testing_pep8` for more information.
 
 You can also contribute by testing and revising other requests, especially if it is one you are interested
 in using. Please keep your comments clear and to the point, courteous and constructive, tickets are not a
@@ -228,7 +229,7 @@ IRC Meetings
 ------------
 
 The Ansible community holds regular IRC meetings on various topics, and anyone who is interested is invited to 
-participate. For more information about Ansible meetings, consult the `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/MEETINGS.md>`_.
+participate. For more information about Ansible meetings, consult the `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
 
 Release Numbering
 -----------------
@@ -256,42 +257,39 @@ IRC Channel
 
 Ansible has several IRC channels on Freenode (irc.freenode.net):
 
-- #ansible - For general use questions and support.
-- #ansible-devel - For discussions on developer topics and code related to features/bugs.
-- #ansible-meeting - For public community meetings. We will generally announce these on one or more of the above mailing lists. See the `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/MEETINGS.md>`_
-- #ansible-notices - Mostly bot output from things like Github, etc.
+- ``#ansible`` - For general use questions and support.
+- ``#ansible-devel`` - For discussions on developer topics and code related to features/bugs.
+- ``#ansible-aws`` - For discussions on Amazon Web Services.
+- ``#ansible-container`` - For discussions on Ansible Container.
+- ``#ansible-vmware`` - For discussions on Ansible & VMware.
+- ``#ansible-windows`` - For discussions on Ansible & Windows.
+- ``#ansible-meeting`` - For public community meetings. We will generally announce these on one or more of the above mailing lists. See the `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/meetings/README.md>`_
+- ``#ansible-notices`` - Mostly bot output from things like Github, etc.
 
 Notes on Priority Flags
 -----------------------
 
 Ansible was one of the top 5 projects with the most OSS contributors on GitHub in 2013, and has over 1400 contributors
 to the project to date, not to mention a very large user community that has downloaded the application well over a million
-times.
-
-As a result, we have a LOT of incoming activity to process.
+times. As a result, we have a LOT of incoming activity to process.
 
 In the interest of transparency, we're telling you how we sort incoming requests.
 
-In our bug tracker you'll notice some labels - P1, P2, P3, P4, and P5.  These are our internal
+In our bug tracker you'll notice some labels - P1 and P2.  These are the internal
 priority orders that we use to sort tickets.
 
 With some exceptions for easy merges (like documentation typos for instance),
 we're going to spend most of our time working on P1 and P2 items first, including pull requests.
-These usually relate to important bugs or features affecting large segments of the userbase.  So if you see something categorized
-"P3 or P4", and it's not appearing to get a lot of immediate attention, this is why.
+These usually relate to important bugs or features affecting large segments of the userbase.  If you see something categorized without P1 or P2, and it's not appearing to get a lot of immediate attention, this is why.
 
-These labels don't really have definition - they are a simple ordering.  However something
+These labels don't really have a definition - they are a simple ordering.  However something
 affecting a major module (yum, apt, etc) is likely to be prioritized higher than a module
 affecting a smaller number of users.
 
-Since we place a strong emphasis on testing and code review, it may take a few months for a minor feature to get merged.
+Since we place a strong emphasis on testing and code review, it may take a few months for a minor feature to get merged. This doesn't mean that we'll be exhausting all of the higher-priority queues before getting to your ticket; we will also take periodic sweeps through the lower priority queues and give them some attention as well, particularly in the area of new module changes. 
 
-Don't worry though -- we'll also take periodic sweeps through the lower priority queues and give
-them some attention as well, particularly in the area of new module changes.  So it doesn't necessarily
-mean that we'll be exhausting all of the higher-priority queues before getting to your ticket.
 
-Every bit of effort helps - if you're wishing to expedite the inclusion of a P3 feature pull request for instance, the best thing you can do
-is help close P2 bug reports.
+Every bit of effort helps - if you're wishing to expedite the inclusion of a P3 feature pull request for instance, the best thing you can do is help close P2 bug reports.
 
 Community Code of Conduct
 =========================
@@ -419,7 +417,7 @@ of our community members and want everyone to feel welcome at our events, both o
 offline.
 
 We expect all participants, organizers, speakers, and attendees to follow these policies at
-our all of our event venues and event-related social events.
+all of our event venues and event-related social events.
 
 The Ansible Community Code of Conduct is licensed under the Creative Commons
 Attribution-Share Alike 3.0 license. Our Code of Conduct was adapted from Codes of Conduct

docs/docsite/rst/community/code_of_conduct.rst

@@ -0,0 +1,144 @@
+*************************
+Community Code of Conduct
+*************************
+
+.. contents:: Topics
+
+Every community can be strengthened by a diverse variety of viewpoints, insights,
+opinions, skillsets, and skill levels. However, with diversity comes the potential for
+disagreement and miscommunication. The purpose of this Code of Conduct is to ensure that
+disagreements and differences of opinion are conducted respectfully and on their own
+merits, without personal attacks or other behavior that might create an unsafe or
+unwelcoming environment.
+
+These policies are not designed to be a comprehensive set of Things You Cannot Do. We ask
+that you treat your fellow community members with respect and courtesy, and in general,
+Don't Be A Jerk. This Code of Conduct is meant to be followed in spirit as much as in
+letter and is not exhaustive.
+
+All Ansible events and participants therein are governed by this Code of Conduct and
+anti-harassment policy. We expect organizers to enforce these guidelines throughout all events,
+and we expect attendees, speakers, sponsors, and volunteers to help ensure a safe
+environment for our whole community. Specifically, this Code of Conduct covers
+participation in all Ansible-related forums and mailing lists, code and documentation
+contributions, public IRC channels, private correspondence, and public meetings.
+
+Ansible community members are...
+
+**Considerate**
+
+Contributions of every kind have far-ranging consequences. Just as your work depends on
+the work of others, decisions you make surrounding your contributions to the Ansible
+community will affect your fellow community members. You are strongly encouraged to take
+those consequences into account while making decisions.
+
+**Patient**
+
+Asynchronous communication can come with its own frustrations, even in the most responsive
+of communities. Please remember that our community is largely built on volunteered time,
+and that questions, contributions, and requests for support may take some time to receive
+a response. Repeated "bumps" or "reminders" in rapid succession are not good displays of
+patience. Additionally, it is considered poor manners to ping a specific person with
+general questions. Pose your question to the community as a whole, and wait patiently for
+a response.
+
+**Respectful**
+
+Every community inevitably has disagreements, but remember that it is
+possible to disagree respectfully and courteously. Disagreements are never an excuse for
+rudeness, hostility, threatening behavior, abuse (verbal or physical), or personal attacks.
+
+**Kind**
+
+Everyone should feel welcome in the Ansible community, regardless of their background.
+Please be courteous, respectful and polite to fellow community members. Do not make or
+post offensive comments related to skill level, gender, gender identity or expression,
+sexual orientation, disability, physical appearance, body size, race, or religion.
+Sexualized images or imagery, real or implied violence, intimidation, oppression,
+stalking, sustained disruption of activities, publishing the personal information of
+others without explicit permission to do so, unwanted physical contact, and unwelcome
+sexual attention are all strictly prohibited.  Additionally, you are encouraged not to
+make assumptions about the background or identity of your fellow community members.
+
+**Inquisitive**
+
+The only stupid question is the one that does not get asked. We
+encourage our users to ask early and ask often. Rather than asking whether you can ask a
+question (the answer is always yes!), instead, simply ask your question. You are
+encouraged to provide as many specifics as possible. Code snippets in the form of Gists or
+other paste site links are almost always needed in order to get the most helpful answers.
+Refrain from pasting multiple lines of code directly into the IRC channels - instead use
+gist.github.com or another paste site to provide code snippets.
+
+**Helpful**
+
+The Ansible community is committed to being a welcoming environment for all users,
+regardless of skill level. We were all beginners once upon a time, and our community
+cannot grow without an environment where new users feel safe and comfortable asking questions.
+It can become frustrating to answer the same questions repeatedly; however, community
+members are expected to remain courteous and helpful to all users equally, regardless of
+skill or knowledge level. Avoid providing responses that prioritize snideness and snark over
+useful information. At the same time, everyone is expected to read the provided
+documentation thoroughly. We are happy to answer questions, provide strategic guidance,
+and suggest effective workflows, but we are not here to do your job for you.
+
+Anti-harassment policy
+======================
+
+Harassment includes (but is not limited to) all of the following behaviors:
+
+- Offensive comments related to gender (including gender expression and identity), age, sexual orientation, disability, physical appearance, body size, race, and religion
+- Derogatory terminology including words commonly known to be slurs
+- Posting sexualized images or imagery in public spaces
+- Deliberate intimidation
+- Stalking
+- Posting others' personal information without explicit permission
+- Sustained disruption of talks or other events
+- Inappropriate physical contact
+- Unwelcome sexual attention
+
+Participants asked to stop any harassing behavior are expected to comply immediately.
+Sponsors are also subject to the anti-harassment policy. In particular, sponsors should
+not use sexualized images, activities, or other material. Meetup organizing staff and
+other volunteer organizers should not use sexualized attire or otherwise create a
+sexualized environment at community events.
+
+In addition to the behaviors outlined above, continuing to behave a certain way after you
+have been asked to stop also constitutes harassment, even if that behavior is not
+specifically outlined in this policy. It is considerate and respectful to stop doing
+something after you have been asked to stop, and all community members are expected to
+comply with such requests immediately.
+
+Policy violations
+=================
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by
+contacting `codeofconduct@ansible.com <mailto:codeofconduct@ansible.com>`_, to any channel
+operator in the community IRC channels, or to the local organizers of an event. Meetup
+organizers are encouraged to prominently display points of contact for reporting unacceptable
+behavior at local events.
+
+If a participant engages in harassing behavior, the meetup organizers may take any action
+they deem appropriate. These actions may include but are not limited to warning the
+offender, expelling the offender from the event, and barring the offender from future
+community events.
+
+Organizers will be happy to help participants contact security or local law enforcement,
+provide escorts to an alternate location, or otherwise assist those experiencing
+harassment to feel safe for the duration of the meetup. We value the safety and well-being
+of our community members and want everyone to feel welcome at our events, both online and
+offline.
+
+We expect all participants, organizers, speakers, and attendees to follow these policies at
+all of our event venues and event-related social events.
+
+The Ansible Community Code of Conduct is licensed under the Creative Commons
+Attribution-Share Alike 3.0 license. Our Code of Conduct was adapted from Codes of Conduct
+of other open source projects, including:
+
+* Contributor Covenant
+* Elastic
+* The Fedora Project
+* OpenStack
+* Puppet Labs
+* Ubuntu

docs/docsite/rst/community/communication.rst

@@ -0,0 +1,68 @@
+*************
+Communicating
+*************
+
+.. contents:: Topics
+
+Mailing List Information
+========================
+
+Ansible has several mailing lists.  Your first post to the mailing list will be moderated (to reduce spam), so please allow up to a day or so for your first post to appear.
+
+`Ansible Project List <https://groups.google.com/forum/#!forum/ansible-project>`_ is for sharing Ansible tips, answering questions, and general user discussion.
+
+`Ansible Development List <https://groups.google.com/forum/#!forum/ansible-devel>`_ is for learning how to develop on Ansible, asking about prospective feature design, or discussions about extending ansible or features in progress.
+
+`Ansible Announce list <https://groups.google.com/forum/#!forum/ansible-announce>`_ is a read-only list that shares information about new releases of Ansible, and also rare infrequent event information, such as announcements about an upcoming AnsibleFest, which is our official conference series.
+
+`Ansible Container List <https://groups.google.com/forum/#!forum/ansible-container>`_ is for users and developers of the Ansible Container project.
+
+`Ansible Lockdown List <https://groups.google.com/forum/#!forum/ansible-lockdown>`_ is for all things related to Ansible Lockdown projects, including DISA STIG automation and CIS Benchmarks.
+
+To subscribe to a group from a non-Google account, you can send an email to the subscription address requesting the subscription. For example: `ansible-devel+subscribe@googlegroups.com`
+
+IRC Channel
+===========
+
+Ansible has several IRC channels on Freenode (irc.freenode.net).
+
+General Channels
+----------------
+
+- ``#ansible`` - For general use questions and support.
+- ``#ansible-devel`` - For discussions on developer topics and code related to features/bugs.
+- ``#ansible-meeting`` - For public community meetings. We will generally announce these on one or more of the above mailing lists. See the `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/meetings/README.md>`_
+- ``#ansible-notices`` - Mostly bot output from things like GitHub, etc.
+
+Working Group
+-------------
+
+- ``#ansible-aws`` - For discussions on Amazon Web Services.
+- ``#ansible-community`` - Channel for discussing Ansible Community related things.
+- ``#ansible-container`` - For discussions on Ansible Container.
+- ``#ansible-jboss`` - Channel for discussing JBoss and Ansible related things.
+- ``#ansible-network`` - Channel for discussing Network and Ansible related things.
+- ``#ansible-news`` - Channel for discussing Ansible Communication & News related things.
+- ``#ansible-vmware`` - For discussions on Ansible & VMware.
+- ``#ansible-windows`` - For discussions on Ansible & Windows.
+
+
+Language specific channels
+--------------------------
+
+- ``#ansible-es`` - Channel for Spanish speaking Ansible community.
+- ``#ansible-fr`` - Channel for French speaking Ansible community.
+
+
+IRC Meetings
+------------
+
+The Ansible community holds regular IRC meetings on various topics, and anyone who is interested is invited to
+participate. For more information about Ansible meetings, consult the `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
+
+Tower Support Questions
+========================
+
+Ansible `Tower <https://ansible.com/tower>`_ is a UI, Server, and REST endpoint for Ansible.
+
+If you have a question about Ansible Tower, visit `Red Hat support <https://access.redhat.com/products/ansible-tower-red-hat/>`_ rather than using the IRC channel or the general project mailing list.

docs/docsite/rst/community/development_process.rst

@@ -0,0 +1,113 @@
+The Ansible Development Process
+===============================
+
+.. contents:: Topics
+
+This section discusses how the Ansible development and triage process works.
+
+Road Maps
+=========
+
+The Ansible Core team provides a road map for each upcoming release. These road maps can be found `here <http://docs.ansible.com/ansible/latest/roadmap/>`_.
+
+Pull Requests
+=============
+
+Ansible accepts code via **pull requests** ("PRs" for short). GitHub provides a great overview of `how the pull request process works <https://help.github.com/articles/about-pull-requests/>`_ in general.
+
+Because Ansible receives many pull requests, we use an automated process to help us through the process of reviewing and merging pull requests. That process is managed by **Ansibullbot**.
+
+Ansibullbot
+===========
+
+Overview
+--------
+
+`Ansibullbot`_ serves many functions:
+
+- Responds quickly to PR submitters to thank them for submitting their PR
+- Identifies the community maintainer responsible for reviewing PRs for any files affected
+- Tracks the current status of PRs
+- Pings responsible parties to remind them of any PR actions for which they may be responsible
+- Provides maintainers with the ability to move PRs through the workflow
+- Identifies PRs abandoned by their submitters so that we can close them
+- Identifies modules abandoned by their maintainers so that we can find new maintainers
+
+Community Maintainers
+---------------------
+
+Each module has at least one assigned maintainer, listed in a `maintainer's file`_:
+
+.. _Ansibullbot: https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md
+.. _maintainer's file: https://github.com/ansible/ansible/blob/devel/.github/BOTMETA.yml
+
+Some modules have no community maintainers assigned. In this case, the maintainer is listed as ``$team_ansible``. Ultimately, it’s our goal to have at least one community maintainer for every module.
+
+The maintainer’s job is to review PRs and decide whether that PR should be merged (``shipit``) or revised (``needs_revision``).
+
+The ultimate goal of any pull request is to reach **shipit** status, where the Core team then decides whether the PR is ready to be merged. Not every PR that reaches the **shipit** label is actually ready to be merged, but the better our reviewers are, and the better our guidelines are, the more likely it will be that a PR that reaches **shipit** will be mergeable.
+
+
+
+Workflow
+--------
+
+Ansibullbot runs continuously. You can generally expect to see changes to your issue or pull request within thirty minutes. Ansibullbot examines every open pull request in the repositories, and enforces state roughly according to the following workflow:
+
+-  If a pull request has no workflow labels, it’s considered **new**. Files in the pull request are identified, and the maintainers of those files are pinged by the bot, along with instructions on how to review the pull request. (Note: sometimes we strip labels from a pull request to “reboot” this process.)
+-  If the module maintainer is not ``$team_ansible``, the pull request then goes into the **community_review** state.
+-  If the module maintainer is ``$team_ansible``, the pull request then goes into the **core_review** state (and probably sits for a while).
+-  If the pull request is in **community_review** and has received comments from the maintainer:
+
+   -  If the maintainer says ``shipit``, the pull request is labeled **shipit**, whereupon the Core team assesses it for final merge.
+   -  If the maintainer says ``needs_info``, the pull request is labeled **needs_info** and the submitter is asked for more info.
+   -  If the maintainer says **needs_revision**, the pull request is labeled **needs_revision** and the submitter is asked to fix some things.
+
+-  If the submitter says ``ready_for_review``, the pull request is put back into **community_review** or **core_review** and the maintainer is notified that the pull request is ready to be reviewed again.
+-  If the pull request is labeled **needs_revision** or **needs_info** and the submitter has not responded lately:
+
+   -  The submitter is first politely pinged after two weeks, pinged again after two more weeks and labeled **pending action**, and the issue or pull request will be closed two weeks after that.
+   -  If the submitter responds at all, the clock is reset.
+-  If the pull request is labeled **community_review** and the reviewer has not responded lately:
+
+   -  The reviewer is first politely pinged after two weeks, pinged again after two more weeks and labeled **pending_action**, and then may be reassigned to ``$team_ansible`` or labeled **core_review**, or often the submitter of the pull request is asked to step up as a maintainer.
+-  If Shippable tests fail, or if the code is not able to be merged, the pull request is automatically put into **needs_revision** along with a message to the submitter explaining why.
+
+
+There are corner cases and frequent refinements, but this is the workflow in general.
+
+PR Labels
+---------
+
+There are two types of PR Labels generally: *workflow labels* and *information labels*.
+
+Workflow Labels
+~~~~~~~~~~~~~~~
+
+-  **community_review**: Pull requests for modules that are currently awaiting review by their maintainers in the Ansible community.
+-  **core_review**: Pull requests for modules that are currently awaiting review by their maintainers on the Ansible Core team.
+-  **needs_info**: Waiting on info from the submitter.
+-  **needs_rebase**: Waiting on the submitter to rebase. (Note: no longer used by the bot.)
+-  **needs_revision**: Waiting on the submitter to make changes.
+-  **shipit**: Waiting for final review by the core team for potential merge.
+
+Informational Labels
+~~~~~~~~~~~~~~~~~~~~
+
+-  **backport**: this is applied automatically if the PR is requested against any branch that is not devel. The bot immediately assigns the labels backport and ``core_review``.
+-  **bugfix_pull_request**: applied by the bot based on the templatized description of the PR.
+-  **cloud**: applied by the bot based on the paths of the modified files.
+-  **docs_pull_request**: applied by the bot based on the templatized description of the PR.
+-  **easyfix**: applied manually, inconsistently used but sometimes useful.
+-  **feature_pull_request**: applied by the bot based on the templatized description of the PR.
+-  **networking**: applied by the bot based on the paths of the modified files.
+-  **owner_pr**: largely deprecated. Formerly workflow, now informational. Originally, PRs submitted by the maintainer would automatically go to **shipit** based on this label. If the submitter is also a maintainer, we notify the other maintainers and still require one of the maintainers (including the submitter) to give a **shipit**.
+-  **pending_action**: applied by the bot to PRs that are not moving. Reviewed every couple of weeks by the community team, who tries to figure out the appropriate action (closure, asking for new maintainers, etc).
+
+
+Special Labels
+~~~~~~~~~~~~~~
+
+-  **new_plugin**: this is for new modules or plugins that are not yet in Ansible.
+
+   **Note:** `new_plugin` kicks off a completely separate process, and frankly it doesn’t work very well at present. We’re working our best to improve this process.

docs/docsite/rst/community/how_can_I_help.rst

@@ -0,0 +1,71 @@
+How To Help
+===========
+
+.. contents:: Topics
+
+There are many ways to help the Ansible project.
+
+Become a power user
+-------------------
+
+A great way to help the Ansible project is to become a power user:
+
+* Use Ansible everywhere you can
+* Take tutorials and classes
+* Read the `official documentation <http://docs.ansible.com/ansible/latest/index.html>`_
+* Study some of the `many excellent books <https://www.amazon.com/s/ref=nb_sb_ss_c_2_7?url=search-alias%3Dstripbooks&field-keywords=ansible&sprefix=ansible%2Caps%2C260>`_ about Ansible
+* `Get certified <https://www.ansible.com/training-certification>`_.
+
+When you become a power user, your ability and opportunities to help the Ansible project in other ways will multiply quickly.
+
+Ask and answer questions online
+-------------------------------
+
+There are many forums online where Ansible users ask and answer questions. Reach out and communicate with your fellow Ansible users. 
+
+You can find the official :ref:`Ansible communication channels <communication>`.
+
+Participate in your local meetup
+--------------------------------
+
+There are Ansible meetups `all over the world <https://www.meetup.com/topics/ansible/>`_. Join your local meetup. Attend regularly. Ask good questions. Volunteer to give a presentation about how you use Ansible.
+
+If there isn't a meetup near you, we'll be happy to help you `start one <https://www.ansible.com/ansible-meetup-organizer>`_.
+
+File and verify issues
+----------------------
+
+All software has bugs, and Ansible is no exception. When you find a bug, you can help tremendously by :ref:`telling us about it <reporting_bugs_and_features>`.
+
+
+If you should discover that the bug you're trying to file already exists in an issue, you can help by verifying the behavior of the reported bug with a comment in that issue, or by reporting any additional information.
+
+Review and submit pull requests
+-------------------------------
+
+As you become more familiar with how Ansible works, you may be able to fix issues or develop new features yourself. If you think you've got a solution to a bug you've found in Ansible, or if you've got a new feature that you've written and would like to share with millions of Ansible users, read all about the `Ansible development process <http://docs.ansible.com/ansible/latest/community/development_process.rst>` to learn how to get your code accepted into Ansible.
+
+Another good way to help is to review pull requests that other Ansible users have submitted. The Ansible community keeps a full list of `open pull requests by file <https://ansible.sivel.net/byfile.html>`_, so if there's a particular module or plug-in that particularly interests you, you can easily keep track of all the relevant new pull requests and provide testing or feedback.
+
+Become a module maintainer
+--------------------------
+
+Once you've learned about the development process and have contributed code to a particular module, we encourage you to become a maintainer of that module. There are hundreds of different modules in Ansible, and the vast majority of them are written and maintained entirely by members of the Ansible community.
+
+To learn more about the responsibilities of being an Ansible module maintainer, please read our :ref:`module maintainer guidelines <maintainers>`.
+
+Join a working group
+--------------------
+
+Working groups are a way for Ansible community members to self-organize around particular topics of interest. We have working groups around various topics. To join or create a working group, please read the `Ansible working group guidelines <https://github.com/ansible/community/blob/master/WORKING-GROUPS.md>`_.
+
+
+Teach Ansible to others
+-----------------------
+
+We're working on a standardized Ansible workshop called `Lightbulb <https://github.com/ansible/lightbulb>`_ that can provide a good hands-on introduction to Ansible usage and concepts.
+
+Social media
+------------
+
+If you like Ansible and just want to spread the good word, feel free to share on your social media platform of choice, and let us know by using ``@ansible`` or ``#ansible``. We'll be looking for you.

docs/docsite/rst/community/index.rst

@@ -0,0 +1,26 @@
+*********************
+Community Information
+*********************
+
+Ansible Community Guide
+=======================
+
+Welcome to the Ansible Community Guide!
+
+The purpose of this guide is to teach you everything you need to know about being a contributing member of the Ansible community.
+
+To get started, select one of the following topics.
+
+
+.. toctree::
+   :maxdepth: 1
+
+   development_process
+   reporting_bugs_and_features
+   how_can_I_help
+   maintainers
+   communication
+   other_tools_and_programs
+
+
+

docs/docsite/rst/community/maintainers.rst

@@ -0,0 +1,71 @@
+****************************
+Module Maintainer Guidelines
+****************************
+
+.. contents:: Topics
+
+Thank you for being a maintainer of one Ansible's community modules! This guide provides module maintainers an overview of their responsibilities, resources for additional information, and links to helpful tools.
+
+In addition to the information below, module maintainers should be familiar with:
+
+* :ref:`General Ansible community development practices <../community>`
+* Documentation on :ref:`module development <developing_modules.html>`
+
+
+Maintainer Responsibilities
+===========================
+
+When you contribute a new module to the `ansible/ansible <https://github.com/ansible/ansible>`_ repository, you become the maintainer for that module once it has been merged. Maintainership empowers you with the authority to accept, reject, or request revisions to pull requests on your module -- but as they say, "with great power comes great responsibility."
+
+Maintainers of Ansible modules are expected to provide feedback, responses, or actions on pull requests or issues to the module(s) they maintain in a reasonably timely manner.
+
+It is also recommended that you occasionally revisit the `contribution guidelines <https://github.com/ansible/ansible/blob/devel/CONTRIBUTING.md>`_, as they are continually refined. Occasionally, you may be requested to update your module to move it closer to the general accepted standard requirements. We hope for this to be infrequent, and will always be a request with a fair amount of lead time (ie: not by tomorrow!).
+
+Finally, following the `ansible-devel <https://groups.google.com/forum/#!forum/ansible-devel>`_ mailing list can be a great way to participate in the broader Ansible community, and a place where you can influence the overall direction, quality, and goals of Ansible and its modules. If you're not on this relatively low-volume list, please join us here: https://groups.google.com/forum/#!forum/ansible-devel
+
+The Ansible community hopes that you will find that maintaining your module is as rewarding for you as having the module is for the wider community.
+
+Pull Requests, Issues, and Workflow
+===================================
+
+Pull Requests
+-------------
+
+Module pull requests are located in the `main Ansible repository <https://github.com/ansible/ansible/pulls>`_.
+
+Because of the high volume of pull requests, notification of PRs to specific modules are routed by an automated bot to the appropriate maintainer for handling. It is recommended that you set an appropriate notification process to receive notifications which mention your GitHub ID.
+
+Issues
+------
+
+Issues for modules, including bug reports, documentation bug reports, and feature requests, are tracked in the `ansible repository <https://github.com/ansible/ansible/issues>`_.
+
+Issues for modules are routed to their maintainers via an automated process. This process is still being refined, and currently depends upon the issue creator to provide adequate details (specifically, providing the proper module name) in order to route it correctly. If you are a maintainer of a specific module, it is recommended that you periodically search module issues for issues which mention your module's name (or some variation on that name), as well as setting an appropriate notification process for receiving notification of mentions of your GitHub ID.
+
+PR Workflow
+-----------
+
+Automated routing of pull requests is handled by a tool called `Ansibot <https://github.com/ansible/ansibullbot>`_.
+
+Being moderately familiar with how the workflow behind the bot operates can be helpful to you, and -- should things go awry -- your feedback can be helpful to the folks that continually help Ansibullbot to evolve.
+
+A detailed explanation of the PR workflow can be seen here: https://github.com/ansible/community/blob/master/PR-FLOW.md
+
+Extras maintainers list
+-----------------------
+
+The full list of maintainers for modules is located here: https://github.com/ansible/ansibullbot/blob/master/MAINTAINERS.txt
+
+Changing Maintainership
+-----------------------
+
+Communities change over time, and no one maintains a module forever. If you'd like to propose an additional maintainer for your module, please submit a PR to the maintainers file with the Github username of the new maintainer.
+
+If you'd like to step down as a maintainer, please submit a PR to the maintainers file removing your Github ID from the module in question. If that would leave the module with no maintainers, put "ansible" as the maintainer.  This will indicate that the module is temporarily without a maintainer, and the Ansible community team will search for a new maintainer.
+
+Tools and other Resources
+-------------------------
+
+* https://ansible.sivel.net/pr/byfile.html -- a full list of all open Pull Requests, organized by file.
+* Ansibullbot: https://github.com/ansible/ansibullbot
+* Triage / pull request workflow and information, including definitions for Labels in GitHub: https://github.com/ansible/community/blob/master/PR-FLOW.md

docs/docsite/rst/community/other_tools_and_programs.rst

@@ -0,0 +1,18 @@
+************************
+Other Tools And Programs
+************************
+
+The Ansible community provides several useful tools for working with the Ansible project. This is a list
+of some of the most popular of these tools.
+
+- `PR by File <https://ansible.sivel.net/pr/byfile.html>`_ shows a current list of all open pull requests by individual file. An essential tool for Ansible module maintainers.
+
+- `Ansible Lint <https://github.com/willthames/ansible-lint>`_ is a widely used, highly configurable best-practices linter for Ansible playbooks.
+
+- `Ansible Review <http://willthames.github.io/2016/06/28/announcing-ansible-review.html>`_ is an extension of Ansible Lint designed for code review.
+
+- `jctanner's Ansible Tools <https://github.com/jctanner/ansible-tools>`_ is a miscellaneous collection of useful helper scripts for Ansible development.
+
+- `Ansigenome <https://github.com/nickjj/ansigenome>`_ is a command line tool designed to help you manage your Ansible roles.
+
+- `Awesome Ansible <https://github.com/jdauphant/awesome-ansible>`_ is a collaboratively curated list of awesome Ansible resources.

docs/docsite/rst/community/reporting_bugs_and_features.rst

@@ -0,0 +1,35 @@
+**************************************
+Reporting Bugs And Requesting Features
+**************************************
+
+.. contents:: Topics
+
+Reporting A Bug
+===============
+
+Ansible practices responsible disclosure - if this is a security related bug, email `security@ansible.com <mailto:security@ansible.com>`_ instead of filing a ticket or posting to the Google Group and you will receive a prompt response.
+
+Ansible bugs should be reported to `github.com/ansible/ansible/issues <https://github.com/ansible/ansible/issues>`_ after
+signing up for a free GitHub account.  Before reporting a bug, please use the bug/issue search
+to see if the issue has already been reported. This is listed on the bottom of the docs page for any module.
+
+Knowing your Ansible version and the exact commands you are running, and what you expect, saves time and helps us help everyone with their issues more quickly. For that reason, we provide an issue template; please fill it out as completely and as accurately as possible.
+
+Do not use the issue tracker for "how do I do this" type questions.  These are great candidates for IRC or the mailing list instead where things are likely to be more of a discussion.
+
+To be respectful of reviewers' time and allow us to help everyone efficiently, please  provide minimal well-reduced and well-commented examples versus sharing your entire production playbook.  Include playbook snippets and output where possible.
+
+When sharing YAML in playbooks, formatting can be preserved by using `code blocks  <https://help.github.com/articles/creating-and-highlighting-code-blocks/>`_.
+
+For multiple-file content, we encourage use of gist.github.com.  Online pastebin content can expire, so it's nice to have things around for a longer term if they are referenced in a ticket.
+
+If you are not sure if something is a bug yet, you are welcome to ask about something on the mailing list or IRC first.
+
+As we are a very high volume project, if you determine that you do have a bug, please be sure to open the issue yourself to ensure we have a record of it. Don’t rely on someone else in the community to file the bug report for you.
+
+Requesting a feature
+====================
+
+The best way to get a feature into Ansible is to submit a pull request.
+
+The next best way of getting a feature into Ansible is to submit a proposal through the `Ansible proposal process  <https://github.com/ansible/proposals>` .

docs/docsite/rst/community/triage_process.rst

@@ -0,0 +1,8 @@
+**************
+Triage Process
+**************
+
+The issue and PR triage processes are driven by the `Ansibot <https://github.com/ansible/ansibullbot>`. Whenever an issue or PR is filed, the Ansibot examines the issue to ensure that all relevant data is present, and handles the routing of the issue as it works its way to eventual completion.
+
+For details on how Ansibot manages the triage process, please consult the `Ansibot
+Issue Guide <https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md>`.

docs/docsite/rst/dev_guide/developing_inventory.rst

@@ -19,22 +19,22 @@ Script Conventions
 When the external node script is called with the single argument ``--list``, the script must output a JSON encoded hash/dictionary of all the groups to be managed to stdout. Each group's value should be either a hash/dictionary containing a list of each host/IP, potential child groups, and potential group variables, or simply a list of host/IP addresses, like so::
 
     {
-        "databases"   : {
-            "hosts"   : [ "host1.example.com", "host2.example.com" ],
-            "vars"    : {
-                "a"   : true
+        "databases": {
+            "hosts": ["host1.example.com", "host2.example.com"],
+            "vars": {
+                "a": true
             }
         },
-        "webservers"  : [ "host2.example.com", "host3.example.com" ],
-        "atlanta"     : {
-            "hosts"   : [ "host1.example.com", "host4.example.com", "host5.example.com" ],
-            "vars"    : {
-                "b"   : false
+        "webservers": ["host2.example.com", "host3.example.com"],
+        "atlanta": {
+            "hosts": ["host1.example.com", "host4.example.com", "host5.example.com"],
+            "vars": {
+                "b": false
             },
-            "children": [ "marietta", "5points" ]
+            "children": ["marietta", "5points"]
         },
-        "marietta"    : [ "host6.example.com" ],
-        "5points"     : [ "host7.example.com" ]
+        "marietta": ["host6.example.com"],
+        "5points": ["host7.example.com"]
     }
 
 .. versionadded:: 1.0
@@ -46,9 +46,9 @@ hash/dictionary, or a hash/dictionary of variables to make available to template
 if the script does not wish to do this, printing an empty hash/dictionary is the way to go::
 
     {
-        "favcolor"   : "red",
-        "ntpserver"  : "wolf.example.com",
-        "monitoring" : "pack.example.com"
+        "favcolor": "red",
+        "ntpserver": "wolf.example.com",
+        "monitoring": "pack.example.com"
     }
 
 .. _inventory_script_tuning:
@@ -74,13 +74,28 @@ The data to be added to the top level JSON dictionary looks like this::
         # results of inventory script as above go here
         # ...
 
-        "_meta" : {
-           "hostvars" : {
-              "moocow.example.com"     : { "asdf" : 1234 },
-              "llama.example.com"      : { "asdf" : 5678 }
-           }
+        "_meta": {
+            "hostvars": {
+                "moocow.example.com": {
+                    "asdf" : 1234
+                },
+                "llama.example.com": {
+                    "asdf": 5678
+                }
+            }
         }
+    }
 
+To satisfy the requirements of using ``_meta``, to prevent ansible from calling your inventory with ``--host`` you must at least populate ``_meta`` with an empty ``hostvars`` dictionary, such as::
+
+    {
+
+        # results of inventory script as above go here
+        # ...
+
+        "_meta": {
+            "hostvars": {}
+        }
     }
 
 .. seealso::

docs/docsite/rst/dev_guide/developing_module_utilities.rst

@@ -36,12 +36,12 @@ The following is a list of module_utils files and a general description. The mod
 - openstack.py - Utilities for modules that work with Openstack instances.
 - openswitch.py - Definitions and helper functions for modules that manage OpenSwitch devices
 - powershell.ps1 - Utilities for working with Microsoft Windows clients
-- pycompat24.py - Exception workaround for python 2.4
+- pycompat24.py - Exception workaround for Python 2.4
 - rax.py -  Definitions and helper functions for modules that work with Rackspace resources.
 - redhat.py - Functions for modules that manage Red Hat Network registration and subscriptions
 - service.py - Contains utilities to enable modules to work with Linux services (placeholder, not in use).
 - shell.py - Functions to allow modules to create shells and work with shell commands
-- six.py - Module utils for working with the Six python 2 and 3 compatibility library
+- six.py - Module utils for working with the Six Python 2 and 3 compatibility library
 - splitter.py - String splitting and manipulation utilities for working with Jinja2 templates
 - urls.py - Utilities for working with http and https requests
 - vca.py - Contains utilities for modules that work with VMware vCloud Air

docs/docsite/rst/dev_guide/developing_modules.rst

@@ -13,14 +13,14 @@ This section discusses how to develop, debug, review, and test modules.
 Ansible modules are reusable, standalone scripts that can be used by the Ansible API,
 or by the :command:`ansible` or :command:`ansible-playbook` programs.  They
 return information to ansible by printing a JSON string to stdout before
-exiting.  They take arguments in in one of several ways which we'll go into
+exiting.  They take arguments in one of several ways which we'll go into
 as we work through this tutorial.
 
 See :doc:`../modules` for a list of existing modules.
 
 Modules can be written in any language and are found in the path specified
 by :envvar:`ANSIBLE_LIBRARY` or the ``--module-path`` command line option or
-in the `library section of the Ansible configration file <http://docs.ansible.com/ansible/intro_configuration.html#library>`_.
+in the :ref:`library section of the Ansible configuration file <library>`.
 
 .. _module_dev_should_you:
 
@@ -31,18 +31,28 @@ develop a module. Ask the following questions:
 
 1. Does a similar module already exist?
 
-There are a lot of existing modules available, and more that are in development. You should check out the list of existing modules at :doc:`../modules` or look at the `module PRs <https://github.com/ansible/ansible/labels/module>`_ for the ansible repository on Github to see if a module that does what you want exists or is in development.
+   There are a lot of existing modules available. You should check out the list of existing modules at :doc:`../modules`
 
-2. Should you use or develop an action plugin instead?
+2. Has someone already worked on a similar pull request?
 
-Action plugins get run on the master instead of on the target. For modules like file/copy/template, some of the work needs to be done on the master before the module executes on the target. Action plugins execute first on the master and can then execute the normal module on the target if necessary.
+   It's possible that someone has already started developing a similar PR. There are a few ways to find open module pull requests:
 
-For more information about action plugins, go `here <https://docs.ansible.com/ansible/dev_guide/developing_plugins.html>`_.
+   * `GitHub new module PRs <https://github.com/ansible/ansible/labels/new_module>`_
+   * `All updates to modules <https://github.com/ansible/ansible/labels/module>`_
+   * `New module PRs listed by directory <https://ansible.sivel.net/pr/byfile.html>`_ search for `lib/ansible/modules/`
 
-3. Should you use a role instead?
+   If you find an existing PR that looks like it addresses the issue you are trying to solve, please provide feedback on the PR -  this will speed up getting the PR merged.
 
-Check out the roles documentation `here <http://docs.ansible.com/ansible/playbooks_roles.html#roles>`_.
+3. Should you use or develop an action plugin instead?
 
+   Action plugins get run on the master instead of on the target. For modules like file/copy/template, some of the work needs to be done on the master before the module executes on the target. Action plugins execute first on the master and can then execute the normal module on the target if necessary.
+
+   For more information about action plugins, :doc:`read the action
+   plugins documentation here <developing_plugins>`.
+
+4. Should you use a role instead?
+
+   Check out the :ref:`roles` documentation for details.
 
 .. _developing_modules_all:
 
@@ -59,8 +69,10 @@ The following topics will discuss how to develop and work with modules:
     Best practices, recommendations, and things to avoid.
 :doc:`developing_modules_checklist`
      Checklist for contributing your module to Ansible.
-:doc:`developing_modules_python3`
-    Adding Python 3 support to modules (all new modules must be py2.4 and py3 compatible).
+:doc:`testing`
+    Developing unit and integration tests.
+:doc:`developing_python3`
+    Adding Python 3 support to modules (all new modules must be Python-2.6 and Python-3.5 compatible).
 :doc:`developing_modules_in_groups`
     A guide for partners wanting to submit multiple modules.
 

docs/docsite/rst/dev_guide/developing_modules_best_practices.rst

@@ -18,7 +18,7 @@ and guidelines:
 
 * If packaging modules in an RPM, they only need to be installed on the control machine and should be dropped into /usr/share/ansible.  This is entirely optional and up to you.
 
-* Modules must output valid JSON only. The toplevel return type must be a hash (dictionary) although they can be nested.  Lists or simple scalar values are not supported, though they can be trivially contained inside a dictionary.
+* Modules must output valid JSON only. The top level return type must be a hash (dictionary) although they can be nested.  Lists or simple scalar values are not supported, though they can be trivially contained inside a dictionary.
 
 * In the event of failure, a key of 'failed' should be included, along with a string explanation in 'msg'.  Modules that raise tracebacks (stacktraces) are generally considered 'poor' modules, though Ansible can deal with these returns and will automatically convert anything unparseable into a failed result.  If you are using the AnsibleModule common Python code, the 'failed' element will be included for you automatically when you call 'fail_json'.
 
@@ -72,7 +72,7 @@ your debugging session will start:
 
 Setting :envvar:`ANSIBLE_KEEP_REMOTE_FILES` to ``1`` tells Ansible to keep the
 remote module files instead of deleting them after the module finishes
-executing.  Giving Ansible the ``-vvv`` optin makes Ansible more verbose.
+executing.  Giving Ansible the ``-vvv`` option makes Ansible more verbose.
 That way it prints the file name of the temporary module file for you to see.
 
 If you want to examine the wrapper file you can.  It will show a small python
@@ -103,13 +103,13 @@ When you look into the debug_dir you'll see a directory structure like this::
 
 * The :file:`args` file contains a JSON string.  The string is a dictionary
   containing the module arguments and other variables that Ansible passes into
-  the module to change it's behaviour.  If you want to modify the parameters
+  the module to change its behaviour.  If you want to modify the parameters
   that are passed to the module, this is the file to do it in.
 
 * The :file:`ansible` directory contains code from
   :mod:`ansible.module_utils` that is used by the module.  Ansible includes
   files for any :`module:`ansible.module_utils` imports in the module but not
-  no files from any other module.  So if your module uses
+  any files from any other module.  So if your module uses
   :mod:`ansible.module_utils.url` Ansible will include it for you, but if
   your module includes :mod:`requests` then you'll have to make sure that
   the python requests library is installed on the system before running the
@@ -171,7 +171,7 @@ You should never do this in a module:
 
 .. code-block:: python
 
-    print "some status message"
+    print("some status message")
 
 Because the output is supposed to be valid JSON.
 
@@ -183,7 +183,7 @@ how the command module is implemented.
 If a module returns stderr or otherwise fails to produce valid JSON, the actual output
 will still be shown in Ansible, but the command will not succeed.
 
-Don't write to files directly; use a temporary file and then use the `atomic_move` function from `ansibile.module_utils.basic` to move the updated temporary file into place. This prevents data corruption and ensures that the correct context for the file is kept.
+Don't write to files directly; use a temporary file and then use the `atomic_move` function from `ansible.module_utils.basic` to move the updated temporary file into place. This prevents data corruption and ensures that the correct context for the file is kept.
 
 Avoid creating a module that does the work of other modules; this leads to code duplication and divergence, and makes things less uniform, unpredictable and harder to maintain. Modules should be the building blocks. Instead of creating a module that does the work of other modules, use Plays and Roles instead.  
 

docs/docsite/rst/dev_guide/developing_modules_checklist.rst

@@ -22,8 +22,8 @@ Contributing Modules Checklist
 The following  checklist items are important guidelines for people who want to contribute to the development of modules to Ansible on GitHub. Please read the guidelines before you submit your PR/proposal.
 
 * The shebang must always be ``#!/usr/bin/python``.  This allows ``ansible_python_interpreter`` to work
-* Modules must be written to support Python 2.4. If this is not possible, required minimum python version and rationale should be explained in the requirements section in ``DOCUMENTATION``.  This minimum requirement will be advanced to Python-2.6 in Ansible-2.4.
-* Modules must be written to use proper Python-3 syntax.  At some point in the future we'll come up with rules for running on Python-3 but we're not there yet.  See :doc:`developing_modules_python3` for help on how to do this.
+* Modules must be written to support Python 2.6. If this is not possible, required minimum Python version and rationale should be explained in the requirements section in ``DOCUMENTATION``.  In Ansible-2.3 the minimum requirement for modules was Python-2.4.
+* Modules must be written to use proper Python-3 syntax.  At some point in the future we'll come up with rules for running on Python-3 but we're not there yet.  See :doc:`developing_python3` for help on how to do this.
 * Modules must have a metadata section.  For the vast majority of new modules,
   the metadata should look exactly like this:
 
@@ -33,7 +33,7 @@ The following  checklist items are important guidelines for people who want to c
                         'supported_by': 'community',
                         'metadata_version': '1.0'}
 
-The complete module metadata specification is here: https://docs.ansible.com/ansible/dev_guide/developing_modules_documenting.html#ansible-metadata-block
+The complete module metadata specification is here: `Ansible metadata block <https://docs.ansible.com/ansible/dev_guide/developing_modules_documenting.html#ansible-metadata-block>`_
 
 * Documentation: Make sure it exists
     * Module documentation should briefly and accurately define what each module and option does, and how it works with others in the underlying system. Documentation should be written for broad audience--readable both by experts and non-experts. This documentation is not meant to teach a total novice, but it also should not be reserved for the Illuminati (hard balance).
@@ -76,7 +76,7 @@ The complete module metadata specification is here: https://docs.ansible.com/ans
     * Avoid catchall exceptions, they are not very useful unless the underlying API gives very good error messages pertaining the attempted action.
 * Module-dependent guidelines: Additional module guidelines may exist for certain families of modules.
     * Be sure to check out the modules themselves for additional information.
-        * Amazon: https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/GUIDELINES.md
+        * `Amazon <https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/cloud/amazon/GUIDELINES.md>`_
     * Modules should make use of the "extends_documentation_fragment" to ensure documentation available. For example, the AWS module should include::
 
         extends_documentation_fragment:
@@ -108,7 +108,7 @@ The complete module metadata specification is here: https://docs.ansible.com/ans
         main()
 
 * Try to normalize parameters with other modules, you can have aliases for when user is more familiar with underlying API name for the option
-* Being pep8 compliant is nice, but not a requirement. Specifically, the 80 column limit now hinders readability more that it improves it
+* Being `PEP 8 <https://www.python.org/dev/peps/pep-0008/>`_ compliant is a requirement. See :doc:`testing_pep8` for more information.
 * Avoid '`action`/`command`', they are imperative and not declarative, there are other ways to express the same thing
 * Do not add `list` or `info` state options to an existing module - create a new `_facts` module.
 * If you are asking 'how can I have a module execute other modules' ... you want to write a role

docs/docsite/rst/dev_guide/developing_modules_documenting.rst

@@ -14,15 +14,16 @@ syntax highlighting before you include it in your Python file.
 
 All modules must have the following sections defined in this order:
 
-1. ANSIBLE_METADATA
-2. DOCUMENTATION
-3. EXAMPLES
-4. RETURNS
-5. Python imports
+1. Copyright
+2. ANSIBLE_METADATA
+3. DOCUMENTATION
+4. EXAMPLES
+5. RETURNS
+6. Python imports
 
 .. note:: Why don't the imports go first?
 
-  Keen Python programmers may notice that contrary to PEP8's advice we don't put ``imports`` at the top of the file. This is because the ``ANSIBLE_METADATA`` through ``RETURNS`` sections are not used by the module code itself; they are essentially extra docstrings for the file. The imports are placed after these special variables for the same reason as PEP8 puts the imports after the introductory comments and docstrings. This keeps the active parts of the code together and the pieces which are purely informational apart. The decision to exclude E402 is based on readability (which is what PEP8 is about). Documentation strings in a module are much more similar to module level docstrings, than code, and are never utilized by the module itself. Placing the imports below this documentation and closer to the code, consolidates and groups all related code in a congruent manner to improve readability, debugging and understanding.
+  Keen Python programmers may notice that contrary to PEP 8's advice we don't put ``imports`` at the top of the file. This is because the ``ANSIBLE_METADATA`` through ``RETURNS`` sections are not used by the module code itself; they are essentially extra docstrings for the file. The imports are placed after these special variables for the same reason as PEP 8 puts the imports after the introductory comments and docstrings. This keeps the active parts of the code together and the pieces which are purely informational apart. The decision to exclude E402 is based on readability (which is what PEP 8 is about). Documentation strings in a module are much more similar to module level docstrings, than code, and are never utilized by the module itself. Placing the imports below this documentation and closer to the code, consolidates and groups all related code in a congruent manner to improve readability, debugging and understanding.
 
 .. warning:: Why do some modules have imports at the bottom of the file?
 
@@ -30,6 +31,41 @@ All modules must have the following sections defined in this order:
 
 
 
+Copyright
+----------------------
+
+The beginning of every module should look about the same. After the shebang,
+there should be at least two lines covering copyright and licensing of the
+code.
+
+.. code-block:: python
+
+    #!/usr/bin/python
+    # Copyright (c) 2017 Ansible Project
+    # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+Every file should have a copyright line with the original copyright holder.
+Major additions to the module (for instance, rewrites)  may add additional
+copyright lines. Code from the Ansible community should typically be assigned
+as "Copyright (c) 2017 Ansible Project" which covers all contributors. Any
+legal questions need to review the source control history, so an exhaustive
+copyright header is not necessary.
+
+The license declaration should be ONLY one line, not the full GPL prefix. If
+you notice a module with the full prefix, feel free to switch it to the
+one-line declaration instead.
+
+When adding a copyright line after completing a significant feature or rewrite,
+add the newer line above the older one, like so:
+
+.. code-block:: python
+
+    #!/usr/bin/python
+    # Copyright (c) 2017 [New Contributor(s)]
+    # Copyright (c) 2015 [Original Contributor(s)]
+    # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+
 ANSIBLE_METADATA Block
 ----------------------
 
@@ -82,14 +118,12 @@ Fields
 :supported_by: This field records who supports the module.
    Default value is ``community``. Values are:
 
-   :core: Maintained by the Ansible core team. Core team will fix
-      bugs, add new features, and review PRs.
-   :community: This module is maintained by the community at large,
-      which is responsible for fixing bugs, adding new features, and
-      reviewing changes.
-   :curated: The Ansible Core Team is the gatekeeper for this module. They
-      will review PRs from the community before merging but might not generate
-      fixes and code for new features on their own.
+   :core:
+   :curated:
+   :community:
+   
+   For information on what the support level values entail, please see
+   `Modules Support <http://docs.ansible.com/ansible/modules_support.html>`_.
 
 :status: This field records information about the module that is
    important to the end user. It’s a list of strings. The default value
@@ -118,7 +152,8 @@ Include it in your module file like this:
 .. code-block:: python
 
     #!/usr/bin/python
-    # Copyright header....
+    # Copyright (c) 2017 [REPLACE THIS]
+    # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
 
     DOCUMENTATION = '''
     ---
@@ -170,6 +205,8 @@ The following fields can be used and are all required unless specified otherwise
     * If `required` is false/missing, `default` may be specified (assumed 'null' if missing).
     * Ensure that the default parameter in the docs matches the default parameter in the code.
     * The default option must not be listed as part of the description.
+    * If the option is a boolean value, you can use any of the boolean values recognized by Ansible:
+      (such as true/false or yes/no).  Choose the one that reads better in the context of the option.
   :choices:
     List of option values. Should be absent if empty.
   :type:
@@ -221,7 +258,7 @@ RETURN Block
 The RETURN section documents what the module returns, and is required for all new modules.
 
 For each value returned, provide a ``description``, in what circumstances the value is ``returned``,
-the ``type`` of the value and a ``sample``.  For example, from the ``copy`` module::
+the ``type`` of the value and a ``sample``.  For example, from the ``copy`` module:
 
 
 The following fields can be used and are all required unless specified otherwise.

docs/docsite/rst/dev_guide/developing_modules_general.rst

@@ -33,9 +33,9 @@ Ok, let's get going with an example.  We'll use Python.  For starters, save this
     import json
 
     date = str(datetime.datetime.now())
-    print json.dumps({
+    print(json.dumps({
         "time" : date
-    })
+    }))
 
 .. _module_testing:
 
@@ -146,10 +146,10 @@ a lot shorter than this:
                 # can be added.
 
                 if rc != 0:
-                    print json.dumps({
+                    print(json.dumps({
                         "failed" : True,
                         "msg"    : "failed setting the time"
-                    })
+                    }))
                     sys.exit(1)
 
                 # when things do not fail, we do not
@@ -160,10 +160,10 @@ a lot shorter than this:
                 # notifiers to be used in playbooks.
 
                 date = str(datetime.datetime.now())
-                print json.dumps({
+                print(json.dumps({
                     "time" : date,
                     "changed" : True
-                })
+                }))
                 sys.exit(0)
 
     # if no parameters are sent, the module may or
@@ -171,9 +171,9 @@ a lot shorter than this:
     # return the time
 
     date = str(datetime.datetime.now())
-    print json.dumps({
+    print(json.dumps({
         "time" : date
-    })
+    }))
 
 Let's test that module::
 
@@ -311,7 +311,7 @@ Supporting Check Mode
 `````````````````````
 .. versionadded:: 1.1
 
-Modules may optionally support `check mode <http://docs.ansible.com/ansible/playbooks_checkmode.html>`. If the user runs Ansible in check mode, a module should try to predict and report whether changes will occur but not actually make any changes (modules that do not support check mode will also take no action, but just will not report what changes they might have made).
+Modules may optionally support `check mode <http://docs.ansible.com/ansible/playbooks_checkmode.html>`_. If the user runs Ansible in check mode, a module should try to predict and report whether changes will occur but not actually make any changes (modules that do not support check mode will also take no action, but just will not report what changes they might have made).
 
 For your module to support check mode, you must pass ``supports_check_mode=True`` when instantiating the AnsibleModule object. The AnsibleModule.check_mode attribute will evaluate to True when check mode is enabled. For example:
 

docs/docsite/rst/dev_guide/developing_modules_in_groups.rst

@@ -19,14 +19,16 @@ Before you start coding
 Although it's tempting to get straight into coding, there are a few things to be aware of first. This list of prerequisites is designed to help ensure that you develop high-quality modules that flow easily through the review process and get into Ansible more quickly.
 
 * Read though all the pages linked off :doc:`developing_modules`; paying particular focus to the :doc:`developing_modules_checklist`.
-* For new modules going into Ansible 2.4 we are raising the bar so they must pass ``pep8 --ignore=E402 --max-line-length=160`` cleanly.
+* For new modules going into Ansible 2.4 we are raising the bar so they must be PEP 8 compliant. See :doc:`testing_pep8` for more information.
 * Starting with Ansible version 2.4, all new modules must support Python 2.6 and Python 3.5+. If this is an issue, please contact us (see the "Speak to us" section later in this document to learn how).
 * All modules shipped with Ansible must be done so under the GPLv3 license. Files under the ``lib/ansible/module_utils/`` directory should be done so under the BSD license.
 * Have a look at the existing modules and how they've been named in the :doc:`../list_of_all_modules`, especially in the same functional area (such as cloud, networking, databases).
 * Shared code can be placed into ``lib/ansible/module_utils/``
 * Shared documentation (for example describing common arguments) can be placed in ``lib/ansible/utils/module_docs_fragments/``.
-* With great power comes great responsiblity: Ansible module maintainers have a duty to help keep modules up to date. As with all successful community projects, module maintainers should keep a watchful eye for reported issues and contributions.
-* Although not required, unit and/or integration tests are strongly recomended. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see ``test/`` and the `Testing Working Group <https://github.com/ansible/community/blob/master/MEETINGS.md>`_.
+* With great power comes great responsibility: Ansible module maintainers have a duty to help keep modules up to date. As with all successful community projects, module maintainers should keep a watchful eye for reported issues and contributions.
+* Although not required, unit and/or integration tests are strongly recommended. Unit tests are especially valuable when external resources (such as cloud or network devices) are required. For more information see :doc:`testing` and the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
+  * Starting with Ansible 2.4 all :doc:`../list_of_network_modules` MUST have unit tests.
+
 
 Naming Convention
 `````````````````
@@ -63,7 +65,7 @@ On :doc:`../community` you can find how to:
 
 * Subscribe to the Mailing Lists - We suggest "Ansible Development List" (for codefreeze info) and "Ansible Announce list"
 * ``#ansible-devel`` - We have found that IRC ``#ansible-devel`` on FreeNodes IRC network works best for module developers so we can have an interactive dialogue.
-* IRC meetings - Join the various weekly IRC meetings `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/MEETINGS.md>`_
+* IRC meetings - Join the various weekly IRC meetings `meeting schedule and agenda page <https://github.com/ansible/community/blob/master/meetings/README.md>`_
 
 
 Your First Pull Request
@@ -91,12 +93,12 @@ And that's it.
 
 Before pushing your PR to GitHub it's a good idea to review the :doc:`developing_modules_checklist` again.
 
-After publishing your PR to https://github.com/ansible/ansible, a Shippable CI test should run within a few minutes. Check the results (at the end of the PR page) to ensure that it's passing (green). If it's not passing, inspect each of the results. Most of the errors should be self-explanatory and are often related to badly formatted documentation (see :doc:`../YAMLSyntax`) or code that isn't valid Python 2.4 & Python 2.6 (see :doc:`developing_modules_python3`). If you aren't sure what a Shippable test message means, copy it into the PR along with a comment and we will review.
+After publishing your PR to https://github.com/ansible/ansible, a Shippable CI test should run within a few minutes. Check the results (at the end of the PR page) to ensure that it's passing (green). If it's not passing, inspect each of the results. Most of the errors should be self-explanatory and are often related to badly formatted documentation (see :doc:`../YAMLSyntax`) or code that isn't valid Python 2.6  or valid Python 3.5 (see :doc:`developing_python3`). If you aren't sure what a Shippable test message means, copy it into the PR along with a comment and we will review.
 
 If you need further advice, consider join the ``#ansible-devel`` IRC channel (see how in the "Where to get support").
 
 
-We have a "ansibot" helper that comments on GitHub Issues and PRs which should highlight important information.
+We have a ``ansibullbot`` helper that comments on GitHub Issues and PRs which should highlight important information.
 
 
 Subsequent PRs
@@ -141,9 +143,8 @@ We realise this may be your first use of Git or GitHub. The following guides may
 * `How to sync (update) your fork <https://help.github.com/articles/syncing-a-fork/>`_
 * `How to create a Pull Request (PR) <https://help.github.com/articles/about-pull-requests/>`_
 
-Please note that in the Ansible Git Repo the main branch is called ``devel`` rather than ``master``, which is used in the offical GitHub documentation
+Please note that in the Ansible Git Repo the main branch is called ``devel`` rather than ``master``, which is used in the official GitHub documentation
 
 After your first PR has been merged ensure you "sync your fork" with ``ansible/ansible`` to ensure you've pulled in the directory structure and and shared code or documentation previously created.
 
 As stated in the GitHub documentation, always use feature branches for your PRs, never commit directly into `devel`.
-

docs/docsite/rst/dev_guide/developing_modules_python3.rst

@@ -124,6 +124,7 @@ to yield text but instead do the conversion explicitly ourselves. For example::
             # Handle the exception gracefully -- usually by displaying a good
             # user-centric error message that can be traced back to this piece
             # of code.
+            pass
 
 .. note:: Much of Ansible assumes that all encoded text is UTF-8.  At some
     point, if there is demand for other encodings we may change that, but for
@@ -285,7 +286,6 @@ For modules which also run on Python-2.4, we have to use an uglier
 construction to make this work under both Python-2.4 and Python-3::
 
     from ansible.module_utils.pycompat24 import get_exception
-    [...]
 
     try:
         a = 2/0

docs/docsite/rst/dev_guide/developing_plugins.rst

@@ -84,6 +84,7 @@ Callback plugins are created by creating a new class with the Base(Callbacks) cl
   from ansible import constants as C
 
   class CallbackModule(CallbackBase):
+      pass
 
 From there, override the specific methods from the CallbackBase that you want to provide a callback for. For plugins intended for use with Ansible version 2.0 and later, you should only override methods that start with `v2`. For a complete list of methods that you can override, please see ``__init__.py`` in the `lib/ansible/plugins/callback <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/callback>`_ directory.
 

docs/docsite/rst/dev_guide/developing_python3.rst

@@ -0,0 +1,447 @@
+====================
+Ansible and Python 3
+====================
+
+Ansible is pursuing a strategy of having one code base that runs on both
+Python-2 and Python-3 because we want Ansible to be able to manage a wide
+variety of machines.  Contributors to Ansible should be aware of the tips in
+this document so that they can write code that will run on the same versions
+of Python as the rest of Ansible.
+
+Ansible can be divided into three overlapping pieces for the purposes of
+porting:
+
+1. Controller-side code.  This is the code which runs on the machine where you
+   invoke :command:`/usr/bin/ansible`
+2. Modules.  This is the code which Ansible transmits over the wire and
+   invokes on the managed machine.
+3. module_utils code.  This is code whose primary purpose is to be used by the
+   modules to perform tasks.  However, some controller-side code might use
+   generic functions from here.
+
+Much of the knowledge of porting code will be usable on all three of these
+pieces but there are some special considerations for some of it as well.
+Information that is generally applicable to all three places is located in the
+controller-side section.
+
+--------------------------------------------
+Minimum Version of Python-3.x and Python-2.x
+--------------------------------------------
+
+In both controller side and module code, we support Python-3.5 or greater and Python-2.6 or
+greater.  Python-3.5 was chosen as a minimum because it is the earliest Python-3 version
+adopted as the default Python by a Long Term Support (LTS) Linux distribution (in this case, Ubuntu-16.04).
+Previous LTS Linux distributions shipped with a Python-2 version which users can rely upon instead of the 
+Python-3 version.
+
+For Python-2, the default is for modules to run on at least Python-2.6.  This allows
+users with older distributions that are stuck on Python-2.6 to manage their
+machines.  Modules are allowed to drop support for Python-2.6 when one of
+their dependent libraries requires a higher version of Python.  This is not an
+invitation to add unnecessary dependent libraries in order to force your
+module to be usable only with a newer version of Python; instead it is an
+acknowledgment that some libraries (for instance, boto3 and docker-py) will
+only function with a newer version of Python.
+
+.. note:: Python-2.4 Module-side Support:
+
+    Support for Python-2.4 and Python-2.5 was dropped in Ansible-2.4.  RHEL-5
+    (and its rebuilds like CentOS-5) were supported until April of 2017.
+    Ansible-2.3 was released in April of 2017 and was the last Ansible release
+    to support Python-2.4 on the module-side.
+
+-----------------------------------
+Porting Controller Code to Python 3
+-----------------------------------
+
+Most of the general tips for porting code to be used on both Python-2 and
+Python-3 applies to porting controller code.  The best place to start learning
+to port code is `Lennart Regebro's book: Porting to Python 3 <http://python3porting.com/>`_.
+
+The book describes several strategies for porting to Python 3.  The one we're
+using is `to support Python-2 and Python-3 from a single code base
+<http://python3porting.com/strategies.html#python-2-and-python-3-without-conversion>`_
+
+Controller String Strategy
+==========================
+
+Background
+----------
+
+One of the most essential things to decide upon for porting code to Python-3
+is what string model to use.  Strings can be an array of bytes (like in C) or
+they can be an array of text.  Text is what we think of as letters, digits,
+numbers, other printable symbols, and a small number of unprintable "symbols"
+(control codes).
+
+In Python-2, the two types for these (:class:`str` for bytes and
+:class:`unicode` for text) are often used interchangeably.  When dealing only
+with ASCII characters, the strings can be combined, compared, and converted
+from one type to another automatically.  When non-ASCII characters are
+introduced, Python starts throwing exceptions due to not knowing what encoding
+the non-ASCII characters should be in.
+
+Python-3 changes this behavior by making the separation between bytes (:class:`bytes`)
+and text (:class:`str`) more strict.  Python will throw an exception when
+trying to combine and compare the two types.  The programmer has to explicitly
+convert from one type to the other to mix values from each.
+
+This change makes it immediately apparent to the programmer when code is
+mixing the types inappropriately, rather than working until one of their users
+causes an exception by entering non-ASCII input.  However, it forces the
+programmer to proactively define a strategy for working with strings in their
+program so that they don't mix text and byte strings unintentionally.
+
+Unicode Sandwich
+----------------
+
+In controller-side code we use a strategy known as the Unicode Sandwich (named
+after Python-2's :class:`unicode` text type).  For Unicode Sandwich we know that
+at the border of our code and the outside world (for example, file and network IO,
+environment variables, and some library calls) we are going to receive bytes.
+We need to transform these bytes into text and use that throughout the
+internal portions of our code.  When we have to send those strings back out to
+the outside world we first convert the text back into bytes.
+To visualize this, imagine a 'sandwich' consisting of a top and bottom layer
+of bytes, a layer of conversion between, and all text type in the center.
+
+Common Borders
+--------------
+
+This is a partial list of places where we have to convert to and from bytes.
+It's not exhaustive but gives you an idea of where to watch for problems.
+
+Reading and writing to files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In Python-2, reading from files yields bytes.  In Python-3, it can yield text.
+To make code that's portable to both we don't make use of Python-3's ability
+to yield text but instead do the conversion explicitly ourselves. For example:
+
+.. code-block:: python
+
+    from ansible.module_utils._text import to_text
+
+    with open('filename-with-utf8-data.txt', 'rb') as my_file:
+        b_data = my_file.read()
+        try:
+            data = to_text(b_data, errors='surrogate_or_strict')
+        except UnicodeError:
+            # Handle the exception gracefully -- usually by displaying a good
+            # user-centric error message that can be traced back to this piece
+            # of code.
+            pass
+
+.. note:: Much of Ansible assumes that all encoded text is UTF-8.  At some
+    point, if there is demand for other encodings we may change that, but for
+    now it is safe to assume that bytes are UTF-8.
+
+Writing to files is the opposite process:
+
+.. code-block:: python
+
+    from ansible.module_utils._text import to_bytes
+
+    with open('filename.txt', 'wb') as my_file:
+        my_file.write(to_bytes(some_text_string))
+
+Note that we don't have to catch :exc:`UnicodeError` here because we're
+transforming to UTF-8 and all text strings in Python can be transformed back
+to UTF-8.
+
+Filesystem Interaction
+~~~~~~~~~~~~~~~~~~~~~~
+
+Dealing with filenames often involves dropping back to bytes because on UNIX-like
+systems filenames are bytes.  On Python-2, if we pass a text string to these
+functions, the text string will be converted to a byte string inside of the
+function and a traceback will occur if non-ASCII characters are present.  In
+Python-3, a traceback will only occur if the text string can't be decoded in
+the current locale, but it's still good to be explicit and have code which
+works on both versions:
+
+.. code-block:: python
+
+    import os.path
+
+    from ansible.module_utils._text import to_bytes
+
+    filename = u'/var/tmp/くらとみ.txt'
+    f = open(to_bytes(filename), 'wb')
+    mtime = os.path.getmtime(to_bytes(filename))
+    b_filename = os.path.expandvars(to_bytes(filename))
+    if os.path.exists(to_bytes(filename)):
+        pass
+
+When you are only manipulating a filename as a string without talking to the
+filesystem (or a C library which talks to the filesystem) you can often get
+away without converting to bytes:
+
+.. code-block:: python
+
+    import os.path
+
+    os.path.join(u'/var/tmp/café', u'くらとみ')
+    os.path.split(u'/var/tmp/café/くらとみ')
+
+On the other hand, if the code needs to manipulate the filename and also talk
+to the filesystem, it can be more convenient to transform to bytes right away
+and manipulate in bytes.
+
+.. warning:: Make sure all variables passed to a function are the same type.
+    If you're working with something like :func:`os.path.join` which takes
+    multiple strings and uses them in combination, you need to make sure that
+    all the types are the same (either all bytes or all text).  Mixing
+    bytes and text will cause tracebacks.
+
+Interacting with Other Programs
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Interacting with other programs goes through the operating system and
+C libraries and operates on things that the UNIX kernel defines.  These
+interfaces are all byte-oriented so the Python interface is byte oriented as
+well.  On both Python-2 and Python-3, byte strings should be given to Python's
+subprocess library and byte strings should be expected back from it.
+
+One of the main places in Ansible's controller code that we interact with
+other programs is the connection plugins' ``exec_command`` methods.  These
+methods transform any text strings they receive in the command (and arguments
+to the command) to execute into bytes and return stdout and stderr as byte strings 
+Higher level functions (like action plugins' ``_low_level_execute_command``)
+transform the output into text strings.
+
+Tips, tricks, and idioms to adopt
+=================================
+
+Forwards Compatibility Boilerplate
+----------------------------------
+
+Use the following boilerplate code at the top of all controller-side modules
+to make certain constructs act the same way on Python-2 and Python-3:
+
+.. code-block:: python
+
+    # Make coding more python3-ish
+    from __future__ import (absolute_import, division, print_function)
+    __metaclass__ = type
+
+``__metaclass__ = type`` makes all classes defined in the file into new-style
+classes without explicitly inheriting from :class:`object`.
+
+The ``__future__`` imports do the following:
+
+:absolute_import: Makes imports look in :attr:`sys.path` for the modules being
+    imported, skipping the directory in which the module doing the importing
+    lives.  If the code wants to use the directory in which the module doing
+    the importing, there's a new dot notation to do so.
+:division: Makes division of integers always return a float.  If you need to
+   find the quotient use ``x // y`` instead of ``x / y``.
+:print_function: Changes :func:`print` from a keyword into a function.
+
+.. seealso::
+    * `PEP 0328: Absolute Imports <https://www.python.org/dev/peps/pep-0328/#guido-s-decision>`_
+    * `PEP 0238: Division <https://www.python.org/dev/peps/pep-0238>`_
+    * `PEP 3105: Print function <https://www.python.org/dev/peps/pep-3105>`_
+
+Prefix byte strings with "b\_"
+------------------------------
+
+Since mixing text and bytes types leads to tracebacks we want to be clear
+about what variables hold text and what variables hold bytes.  We do this by
+prefixing any variable holding bytes with ``b_``.  For instance:
+
+.. code-block:: python
+
+    filename = u'/var/tmp/café.txt'
+    b_filename = to_bytes(filename)
+    with open(b_filename) as f:
+        data = f.read()
+
+We do not prefix the text strings instead because we only operate
+on byte strings at the borders, so there are fewer variables that need bytes
+than text.
+
+Bundled six
+-----------
+
+The third-party `python-six <https://pythonhosted.org/six/>`_ library exists
+to help projects create code that runs on both Python-2 and Python-3.  Ansible
+includes a version of the library in module_utils so that other modules can use it
+without requiring that it is installed on the remote system.  To make use of
+it, import it like this:
+
+.. code-block:: python
+
+    from ansible.module_utils import six
+
+.. note:: Ansible can also use a system copy of six
+
+    Ansible will use a system copy of six if the system copy is a later
+    version than the one Ansible bundles.
+
+Exceptions
+----------
+
+In order for code to function on Python-2.6+ and Python-3, use the
+new exception-catching syntax which uses the ``as`` keyword:
+
+.. code-block:: python
+
+    try:
+        a = 2/0
+    except ValueError as e:
+        module.fail_json(msg="Tried to divide by zero: %s" % e)
+
+Do **not** use the following syntax as it will fail on every version of Python-3:
+
+.. This code block won't highlight because python2 isn't recognized. This is necessary to pass tests under python 3.
+.. code-block:: python2
+
+    try:
+        a = 2/0
+    except ValueError, e:
+        module.fail_json(msg="Tried to divide by zero: %s" % e)
+
+Octal numbers
+-------------
+
+In Python-2.x, octal literals could be specified as ``0755``.  In Python-3,
+octals must be specified as ``0o755``.
+
+String formatting
+-----------------
+
+str.format() compatibility
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Starting in Python-2.6, strings gained a method called ``format()`` to put
+strings together.  However, one commonly used feature of ``format()`` wasn't
+added until Python-2.7, so you need to remember not to use it in Ansible code:
+
+.. code-block:: python
+
+    # Does not work in Python-2.6!
+    new_string = "Dear {}, Welcome to {}".format(username, location)
+
+    # Use this instead
+    new_string = "Dear {0}, Welcome to {1}".format(username, location)
+
+Both of the format strings above map positional arguments of the ``format()``
+method into the string.  However, the first version doesn't work in
+Python-2.6.  Always remember to put numbers into the placeholders so the code
+is compatible with Python-2.6.
+
+.. seealso::
+    Python documentation on `format strings <https://docs.python.org/2/library/string.html#formatstrings>`_
+
+
+Use percent format with byte strings
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In Python-3.x, byte strings do not have a ``format()`` method.  However, it
+does have support for the older, percent-formatting.
+
+.. code-block:: python
+
+    b_command_line = b'ansible-playbook --become-user %s -K %s' % (user, playbook_file)
+
+.. note:: Percent formatting added in Python-3.5
+
+    Percent formatting of byte strings was added back into Python3 in 3.5.
+    This isn't a problem for us because Python-3.5 is our minimum version.
+    However, if you happen to be testing Ansible code with Python-3.4 or
+    earlier, you will find that the byte string formatting here won't work.
+    Upgrade to Python-3.5 to test.
+
+.. seealso::
+    Python documentation on `percent formatting <https://docs.python.org/2/library/stdtypes.html#string-formatting>`_
+
+---------------------------
+Porting Modules to Python 3
+---------------------------
+
+Ansible modules are slightly harder to port than normal code from other
+projects. A lot of mocking has to go into unit testing an Ansible module so
+it's harder to test that your porting has fixed everything or to to make sure
+that later commits haven't regressed the Python-3 support.
+
+Module String Strategy
+======================
+
+There are a large number of modules in Ansible.  Most of those are maintained
+by the Ansible community at large, not by a centralized team.  To make life
+easier on them, it was decided not to break backwards compatibility by
+mandating that all strings inside of modules are text and converting between
+text and bytes at the borders; instead, we're using a native string strategy
+for now.
+
+Native strings refer to the type that Python uses when you specify a bare
+string literal:
+
+.. code-block:: python
+
+    "This is a native string"
+
+In Python-2, these are byte strings.  In Python-3 these are text strings.  The
+module_utils shipped with Ansible attempts to accept native strings as input
+to its functions and emit native strings as their output.  Modules should be
+coded to expect bytes on Python-2 and text on Python-3.
+
+Tips, tricks, and idioms to adopt
+=================================
+
+Python-2.4 Compatible Exception Syntax
+--------------------------------------
+
+Until Ansible-2.4, modules needed to be compatible with Python-2.4 as
+well.  Python-2.4 did not understand the new exception-catching syntax so
+we had to write a compatibility function that could work with both
+Python-2 and Python-3.  You may still see this used in some modules:
+
+.. code-block:: python
+
+    from ansible.module_utils.pycompat24 import get_exception
+
+    try:
+        a = 2/0
+    except ValueError:
+        e = get_exception()
+        module.fail_json(msg="Tried to divide by zero: %s" % e)
+
+Unless a change is going to be backported to Ansible-2.3, you should not
+have to use this in new code.
+
+Python 2.4 octal workaround
+---------------------------
+
+Before Ansible-2.4, modules had to be compatible with Python-2.4.
+Python-2.4 did not understand the new syntax for octal literals so we used
+the following workaround to specify octal values:
+
+.. code-block:: python
+
+    # Can't use 0755 on Python-3 and can't use 0o755 on Python-2.4
+    EXECUTABLE_PERMS = int('0755', 8)
+
+Unless a change is going to be backported to Ansible-2.3, you should not
+have to use this in new code.
+
+-------------------------------------
+Porting module_utils code to Python 3
+-------------------------------------
+
+module_utils code is largely like module code.  However, some pieces of it are
+used by the controller as well.  Because of this, it needs to be usable with
+the controller's assumptions.  This is most notable in the string strategy.
+
+Module_utils String Strategy
+============================
+
+Module_utils **must** use the Native String Strategy.  Functions in
+module_utils receive either text strings or byte strings and may emit either
+the same type as they were given or the native string for the Python version
+they are run on depending on which makes the most sense for that function.
+Functions which return strings **must** document whether they return text,
+byte, or native strings. Module-utils functions are therefore often very
+defensive in nature, converting from potential text or bytes at the
+beginning of a function and converting to the native string type at the end.

docs/docsite/rst/dev_guide/developing_releases.rst

@@ -1,57 +0,0 @@
-Releases
-========
-
-.. contents:: Topics
-   :local:
-
-.. _support_life:
-
-Support for older releases
-``````````````````````````
-
-Ansible supports the two most recent major, stable releases. Security- and bug-related fixes may be implemented in older versions, but this
-support is not guaranteed.
-
-If you are on a release older than the last two major, stable releases, please see our `Porting Guide <http://docs.ansible.com/ansible/porting_guide_2.0.html>`_.
-
-.. _schedule:
-
-Release schedule
-````````````````
-Ansible is on a 'flexible' 4 month release schedule. Sometimes the release cycle can be extended if there is a major change that requires more time (for example, a core rewrite).
-Recently the main Ansible repo `merged <https://docs.ansible.com/ansible/dev_guide/repomerge.html>`_ the separated ansible-modules-core and ansible-modules-extras repos, as such modules get released at the same time as the main Ansible repo.
-
-The major features and bugs fixed in a release should be reflected in the `CHANGELOG.md <https://github.com/ansible/ansible/blob/devel/CHANGELOG.md>`_. Minor features and bug fixes will be shown in the commit history. For example, `issue #19057 <https://github.com/ansible/ansible/pull/19057>`_ is reflected only in the commit history.
-When a fix orfeature gets added to the `devel` branch it will be part of the next release. Some bugfixes can be backported to previous releases and will be part of a minor point release if such a release is deemed necessary.
-
-Sometimes a release candidate can be extended by a few days if a bug fix makes a change that can have far-reaching consequences, so users have enough time to find any new issues that may stem from this.
-
-.. _methods:
-
-Release methods
-````````````````
-
-Ansible normally goes through a 'release candidate', issuing an RC1 for a release. If no major bugs are discovered in the release candidate after 5 business days,  we'll get a final release. Otherwise, fixes will be applied and an RC2 will be provided for testing. If no bugs are discovered in RC2 after 2 days, the final release will be made, iterating this last step and incrementing the candidate number as we find major bugs.
-
-
-.. _freezing:
-
-Release feature freeze
-``````````````````````
-
-During the release candidate process, the focus will be on bugfixes that affect the RC, new features will be delayed while we try to produce a final version. Some bugfixes that are minor or don't affect the RC will also be postponed until after the release is finalized.
-
-.. seealso::
-
-   :doc:`developing_api`
-       Python API to Playbooks and Ad Hoc Task Execution
-   :doc:`developing_modules`
-       How to develop modules
-   :doc:`developing_plugins`
-       How to develop plugins
-   `Ansible Tower <https://ansible.com/ansible-tower>`_
-       REST API endpoint and GUI for Ansible, syncs with dynamic inventory
-   `Development Mailing List <http://groups.google.com/group/ansible-devel>`_
-       Mailing list for development topics
-   `irc.freenode.net <http://irc.freenode.net>`_
-       #ansible IRC chat channel

docs/docsite/rst/dev_guide/developing_test_pr.rst

@@ -180,11 +180,11 @@ If the PR does not resolve the issue, or if you see any failures from the unit/i
    |
    | When I ran this Ubuntu 16.04 it failed with the following:
    |
-   |   ```
+   |   \```
    |   BLARG
    |   StrackTrace
    |   RRRARRGGG
-   |   ```
+   |   \```
 
 When you are done testing a feature branch, you can remove it with the following command:
 

docs/docsite/rst/dev_guide/index.rst

@@ -1,8 +1,9 @@
+*********************
 Developer Information
-=====================
+*********************
 
 Ansible Developer Guide
-```````````````````````
+=======================
 
 Welcome to the Ansible Developer Guide!
 
@@ -16,14 +17,22 @@ To get started, select one of the following topics.
 
    overview_architecture
    developing_modules
-   developing_modules_python3
+   developing_modules_general
+   developing_modules_documenting
+   developing_modules_best_practices
+   developing_modules_checklist
+   developing_modules_in_groups
+   developing_program_flow_modules
+   developing_module_utilities
    developing_plugins
    developing_inventory
-   developing_api
-   developing_module_utilities
    developing_core
+   developing_python3
+   developing_api
    developing_test_pr
    developing_rebasing
+   testing
    repomerge
-   developing_releases
+   ../release_and_maintenance
    ../committer_guidelines
+   ./style_guide/index

docs/docsite/rst/dev_guide/overview_architecture.rst

@@ -16,14 +16,14 @@ Ansible works by connecting to your nodes and pushing out small programs, called
 Your library of modules can reside on any machine, and there are no servers, daemons, or databases required. Typically you'll work with your favorite terminal program, a text editor, and probably a version control system to keep track of changes to your content.
 
 Plugins
--------
+``````````````````
 
 Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own.
 
 Inventory
 ````````````````````
 
-By default, Ansible represents what machines it manages using a very simple INI file that puts all of your managed machines in groups of your own choosing.  
+By default, Ansible represents what machines it manages using a very simple INI file that puts all of your managed machines in groups of your own choosing.
 
 To add new machines, there is no additional SSL signing server involved, so there's never any hassle deciding why a particular machine didn’t get linked up due to obscure NTP or DNS issues.
 
@@ -51,7 +51,7 @@ Playbooks can finely orchestrate multiple slices of your infrastructure topology
 
 Ansible's approach to orchestration is one of finely-tuned simplicity, as we believe your automation code should make perfect sense to you years down the road and there should be very little to remember about special syntax or features.
 
-Here's what a simple playbook looks like:: 
+Here's what a simple playbook looks like::
 
     ---
     - hosts: webservers
@@ -70,6 +70,3 @@ Extending Ansible with Plug-ins and the API
 ````````````````````````````````````````````
 
 Should you want to write your own, Ansible modules can be written in any language that can return JSON (Ruby, Python, bash, etc). Inventory can also plug in to any datasource by writing a program that speaks to that datasource and returns JSON. There's also various Python APIs for extending Ansible’s connection types (SSH is not the only transport possible), callbacks (how Ansible logs, etc), and even for adding new server side behaviors.
-
-
-

docs/docsite/rst/dev_guide/testing.rst

@@ -0,0 +1,195 @@
+***************
+Testing Ansible
+***************
+
+.. contents:: Topics
+
+Introduction
+============
+
+This document describes:
+
+* how Ansible is tested
+* how to test Ansible locally
+* how to extend the testing capabilities
+
+Types of tests
+==============
+
+At a high level we have the following classifications of tests:
+
+:compile:
+  * :doc:`testing_compile`
+  * Test python code against a variety of Python versions.
+:sanity:
+  * :doc:`testing_sanity`
+  * Sanity tests are made up of scripts and tools used to perform static code analysis.
+  * The primary purpose of these tests is to enforce Ansible coding standards and requirements.
+:integration:
+  * :doc:`testing_integration`
+  * Functional tests of modules and Ansible core functionality.
+:units:
+  * :doc:`testing_units`
+  * Tests directly against individual parts of the code base.
+
+
+If you're a developer, one of the most valuable things you can do is look at the GitHub issues list and help fix bugs.  We almost always prioritize bug fixing over feature development, so helping to fix bugs is one of the best things you can do.
+
+Even if you're not a developer, helping to test pull requests for bug fixes and features is still immensely valuable.
+
+
+Testing within GitHub & Shippable
+=================================
+
+
+Organization
+------------
+
+When Pull Requests (PRs) are created they are tested using Shippable, a Continuous Integration (CI) tool. Results are shown at the end of every PR.
+
+
+When Shippable detects an error and it can be linked back to a file that has been modified in the PR then the relevant lines will be added as a GitHub comment. For example::
+
+   The test `ansible-test sanity --test pep8` failed with the following errors:
+
+   lib/ansible/modules/network/foo/bar.py:509:17: E265 block comment should start with '# '
+
+   The test `ansible-test sanity --test validate-modules` failed with the following errors:
+   lib/ansible/modules/network/foo/bar.py:0:0: E307 version_added should be 2.4. Currently 2.3
+   lib/ansible/modules/network/foo/bar.py:0:0: E316 ANSIBLE_METADATA.metadata_version: required key not provided @ data['metadata_version']. Got None
+
+From the above example we can see that ``--test pep8`` and ``--test validate-modules`` have identified issues. The commands given allow you to run the same tests locally to ensure you've fixed the issues without having to push your changed to GitHub and wait for Shippable, for example:
+
+If you haven't already got Ansible available, use the local checkout by running::
+
+  source hacking/env-setup
+
+Then run the tests detailed in the GitHub comment::
+
+  ansible-test sanity --test pep8
+  ansible-test sanity --test validate-modules
+
+
+If there isn't a GitHub comment stating what's failed you can inspect the results by clicking on the "Details" button under the "checks have failed" message at the end of the PR.
+
+Rerunning a failing CI job
+--------------------------
+
+Occasionally you may find your PR fails due to a reason unrelated to your change. This could happen for several reasons, including:
+
+* a temporary issue accessing an external resource, such as a yum or git repo
+* a timeout creating a virtual machine to run the tests on
+
+If either of these issues appear to be the case, you can rerun the Shippable test by:
+
+* closing and re-opening the PR
+* making another change to the PR and pushing to GitHub
+
+If the issue persists, please contact us in ``#ansible-devel`` on Freenode IRC.
+
+
+How to test a PR
+================
+
+If you're a developer, one of the most valuable things you can do is look at the GitHub issues list and help fix bugs.  We almost always prioritize bug fixing over feature development, so helping to fix bugs is one of the best things you can do.
+
+Even if you're not a developer, helping to test pull requests for bug fixes and features is still immensely valuable.
+
+Ideally, code should add tests that prove that the code works. That's not always possible and tests are not always comprehensive, especially when a user doesn't have access to a wide variety of platforms, or is using an API or web service. In these cases, live testing against real equipment can be more valuable than automation that runs against simulated interfaces. In any case, things should always be tested manually the first time as well.
+
+Thankfully, helping to test Ansible is pretty straightforward, assuming you are familiar with how Ansible works.
+
+Setup: Checking out a Pull Request
+----------------------------------
+
+You can do this by:
+
+* checking out Ansible
+* making a test branch off the main branch
+* merging a GitHub issue
+* testing
+* commenting on that particular issue on GitHub
+
+Here's how:
+
+.. warning::
+   Testing source code from GitHub pull requests sent to us does have some inherent risk, as the source code
+   sent may have mistakes or malicious code that could have a negative impact on your system. We recommend
+   doing all testing on a virtual machine, whether a cloud instance, or locally.  Some users like Vagrant
+   or Docker for this, but they are optional. It is also useful to have virtual machines of different Linux or
+   other flavors, since some features (apt vs. yum, for example) are specific to those OS versions.
+
+
+Create a fresh area to work::
+
+
+   git clone https://github.com/ansible/ansible.git ansible-pr-testing
+   cd ansible-pr-testing
+
+Next, find the pull request you'd like to test and make note of the line at the top which describes the source
+and destination repositories. It will look something like this::
+
+   Someuser wants to merge 1 commit into ansible:devel from someuser:feature_branch_name
+
+.. note:: Only test ``ansible:devel``
+
+   It is important that the PR request target be ``ansible:devel``, as we do not accept pull requests into any other branch. Dot releases are cherry-picked manually by Ansible staff.
+
+The username and branch at the end are the important parts, which will be turned into git commands as follows::
+
+   git checkout -b testing_PRXXXX devel
+   git pull https://github.com/someuser/ansible.git feature_branch_name
+
+The first command creates and switches to a new branch named ``testing_PRXXXX``, where the XXXX is the actual issue number associated with the pull request (for example, 1234). This branch is based on the ``devel`` branch. The second command pulls the new code from the users feature branch into the newly created branch.
+
+.. note::
+   If the GitHub user interface shows that the pull request will not merge cleanly, we do not recommend proceeding if you are not somewhat familiar with git and coding, as you will have to resolve a merge conflict. This is the responsibility of the original pull request contributor.
+
+.. note::
+   Some users do not create feature branches, which can cause problems when they have multiple, unrelated commits in their version of ``devel``. If the source looks like ``someuser:devel``, make sure there is only one commit listed on the pull request.
+
+The Ansible source includes a script that allows you to use Ansible directly from source without requiring a
+full installation that is frequently used by developers on Ansible.
+
+Simply source it (to use the Linux/Unix terminology) to begin using it immediately::
+
+   source ./hacking/env-setup
+
+This script modifies the ``PYTHONPATH`` environment variables (along with a few other things), which will be temporarily
+set as long as your shell session is open.
+
+Testing the Pull Request
+------------------------
+
+At this point, you should be ready to begin testing!
+
+Some ideas of what to test are:
+
+* Create a test Playbook with the examples in and check if they function correctly
+* Test to see if any Python backtraces returned (that's a bug)
+* Test on different operating systems, or against different library versions
+
+
+Any potential issues should be added as comments on the pull request (and it's acceptable to comment if the feature works as well), remembering to include the output of ``ansible --version``
+
+Example::
+
+   Works for me! Tested on `Ansible 2.3.0`.  I verified this on CentOS 6.5 and also Ubuntu 14.04.
+
+If the PR does not resolve the issue, or if you see any failures from the unit/integration tests, just include that output instead:
+
+   | This doesn't work for me.
+   |
+   | When I ran this Ubuntu 16.04 it failed with the following:
+   |
+   |   \```
+   |   some output
+   |   StrackTrace
+   |   some other output
+   |   \```
+
+Want to know more about testing?
+================================
+
+If you'd like to know more about the plans for improving testing Ansible then why not join the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
+

docs/docsite/rst/dev_guide/testing_compile.rst

@@ -0,0 +1,67 @@
+*************
+Compile Tests
+*************
+
+.. contents:: Topics
+
+Overview
+========
+
+Compile tests check source files for valid syntax on all supported python versions:
+
+- 2.4 (Ansible 2.3 only)
+- 2.6
+- 2.7
+- 3.5
+- 3.6
+
+Running compile tests locally
+=============================
+
+Unit tests can be run across the whole code base by doing:
+
+.. code:: shell
+
+    cd /path/to/ansible/source
+    source hacking/env-setup
+    ansible-test compile
+
+Against a single file by doing:
+
+.. code:: shell
+
+   ansible-test compile lineinfile
+
+Or against a specific Python version by doing:
+
+.. code:: shell
+
+   ansible-test compile --python 2.7 lineinfile
+
+For advanced usage see the help:
+
+.. code:: shell
+
+   ansible-test units --help
+
+
+Installing dependencies
+=======================
+
+``ansible-test`` has a number of dependencies , for ``compile`` tests we suggest running the tests with ``--local``, which is the default
+
+The dependencies can be installed using the ``--requirements`` argument. For example:
+
+.. code:: shell
+
+   ansible-test units --requirements lineinfile
+
+
+
+The full list of requirements can be found at `test/runner/requirements <https://github.com/ansible/ansible/tree/devel/test/runner/requirements>`_. Requirements files are named after their respective commands. See also the `constraints <https://github.com/ansible/ansible/blob/devel/test/runner/requirements/constraints.txt>`_ applicable to all commands.
+
+
+Extending compile tests
+=======================
+
+If you believe changes are needed to the Compile tests please add a comment on the `Testing Working Group Agenda <https://github.com/ansible/community/blob/master/meetings/README.md>`_ so it can be discussed.

docs/docsite/rst/dev_guide/testing_httptester.rst

@@ -0,0 +1,72 @@
+**********
+httptester
+**********
+
+.. contents:: Topics
+
+Overview
+========
+
+``httptester`` is a docker container used to host certain resources required by :doc:`testing_integration`. This is to avoid CI tests requiring external resources (such as git or package repos) which, if temporarily unavailable, would cause tests to fail.
+
+HTTP Testing endpoint which provides the following capabilities:
+
+* httpbin
+* nginx
+* SSL
+* SNI
+
+
+Source files can be found at `test/utils/docker/httptester/ <https://github.com/ansible/ansible/tree/devel/test/utils/docker/httptester>`_
+
+Building
+========
+
+Docker
+------
+
+Both ways of building ``docker`` utilize the ``nginx:alpine`` image, but can
+be customized for ``Fedora``, ``Red Hat``, ``CentOS``, ``Ubuntu``,
+``Debian`` and other variants of ``Alpine``
+
+When utilizing ``packer`` or configuring with ``ansible-playbook``,
+the services will not automatically start on launch, and will have to be
+manually started using::
+
+    cd test/utils/docker/httptester
+    ./services.sh
+
+Such as when starting a docker container::
+
+    cd test/utils/docker/httptester
+    docker run -ti --rm -p 80:80 -p 443:443 --name httptester ansible/ansible:httptester /services.sh
+
+docker build
+------------
+
+::
+
+    cd test/utils/docker/httptester
+    docker build -t ansible/ansible:httptester .
+
+packer
+------
+
+The ``packer`` build will use ``ansible-playbook`` to perform the
+configuration, and will tag the image as ``ansible/ansible:httptester``::
+
+    cd test/utils/docker/httptester
+    packer build packer.json
+
+Ansible
+=======
+
+::
+    cd test/utils/docker/httptester
+    ansible-playbook -i hosts -v httptester.yml
+
+
+Extending httptester
+====================
+
+If you have sometime to improve ``httptester`` please add a comment on the `Testing Working Group Agenda <https://github.com/ansible/community/blob/master/meetings/README.md>`_ to avoid duplicated effort.

docs/docsite/rst/dev_guide/testing_integration.rst

@@ -0,0 +1,270 @@
+*****************
+Integration tests
+*****************
+
+.. contents:: Topics
+
+The Ansible integration Test system.
+
+Tests for playbooks, by playbooks.
+
+Some tests may require credentials.  Credentials may be specified with `credentials.yml`.
+
+Some tests may require root.
+
+Quick Start
+===========
+
+It is highly recommended that you install and activate the ``argcomplete`` python package.
+It provides tab completion in ``bash`` for the ``ansible-test`` test runner.
+
+Configuration
+=============
+
+Making your own version of ``integration_config.yml`` can allow for setting some
+tunable parameters to help run the tests better in your environment.  Some
+tests (e.g. cloud) will only run when access credentials are provided.  For
+more information about supported credentials, refer to ``credentials.template``.
+
+Prerequisites
+=============
+
+The tests will assume things like hg, svn, and git are installed and in path.  Some tests
+(such as those for Amazon Web Services) need separate definitions, which will be covered
+later in this document.
+
+(Complete list pending)
+
+Non-destructive Tests
+=====================
+
+These tests will modify files in subdirectories, but will not do things that install or remove packages or things
+outside of those test subdirectories.  They will also not reconfigure or bounce system services.
+
+.. note:: Running integration tests within Docker
+
+   To protect your system from any potential changes caused by integration tests, and to ensure the a sensible set of dependencies are available we recommend that you always run integration tests with the ``--docker`` option. See the `list of supported docker images <https://github.com/ansible/ansible/blob/devel/test/runner/completion/docker.txt>`_ for options.
+
+.. note:: Avoiding pulling new Docker images
+
+   Use the ``--docker-no-pull`` option to avoid pulling the latest container image. This is required when using custom local images that are not available for download.
+
+Run as follows for all POSIX platform tests executed by our CI system::
+
+    test/runner/ansible-test integration --docker fedora25 -v posix/ci/
+
+You can select specific tests as well, such as for individual modules::
+
+    test/runner/ansible-test integration -v ping
+
+By installing ``argcomplete`` you can obtain a full list by doing::
+
+    test/runner/ansible-test integration <tab><tab>
+
+Destructive Tests
+=================
+
+These tests are allowed to install and remove some trivial packages.  You will likely want to devote these
+to a virtual environment, such as Docker.  They won't reformat your filesystem::
+
+    test/runner/ansible-test integration --docker fedora25 -v destructive/
+
+Windows Tests
+=============
+
+These tests exercise the ``winrm`` connection plugin and Windows modules.  You'll
+need to define an inventory with a remote Windows 2008 or 2012 Server to use
+for testing, and enable PowerShell Remoting to continue.
+
+Running these tests may result in changes to your Windows host, so don't run
+them against a production/critical Windows environment.
+
+Enable PowerShell Remoting (run on the Windows host via Remote Desktop)::
+
+    Enable-PSRemoting -Force
+
+Define Windows inventory::
+
+    cp inventory.winrm.template inventory.winrm
+    ${EDITOR:-vi} inventory.winrm
+
+Run the Windows tests executed by our CI system::
+
+    test/runner/ansible-test windows-integration -v windows/ci/
+
+Tests in Docker containers
+==========================
+
+If you have a Linux system with Docker installed, running integration tests using the same Docker containers used by
+the Ansible continuous integration (CI) system is recommended.
+
+.. note:: Docker on non-Linux
+
+   Using Docker Engine to run Docker on a non-Linux host (such as macOS) is not recommended.
+   Some tests may fail, depending on the image used for testing.
+   Using the ``--docker-privileged`` option may resolve the issue.
+
+Running Integration Tests
+-------------------------
+
+To run all CI integration test targets for POSIX platforms in a Ubuntu 16.04 container::
+
+    test/runner/ansible-test integration -v posix/ci/ --docker
+
+You can also run specific tests or select a different Linux distribution.
+For example, to run tests for the ``ping`` module on a Ubuntu 14.04 container::
+
+    test/runner/ansible-test integration -v ping --docker ubuntu1404
+
+Container Images
+----------------
+
+Python 2
+````````
+
+Most container images are for testing with Python 2:
+
+  - centos6
+  - centos7
+  - fedora24
+  - fedora25
+  - opensuse42.1
+  - opensuse42.2
+  - ubuntu1204
+  - ubuntu1404
+  - ubuntu1604
+
+Python 3
+````````
+
+To test with Python 3 use the following images:
+
+  - ubuntu1604py3
+
+Legacy Cloud Tests
+==================
+
+Some of the cloud tests run as normal integration tests, and others run as legacy tests; see the
+:doc:`testing_integration_legacy` page for more information.
+
+
+Other configuration for Cloud Tests
+===================================
+
+In order to run some tests, you must provide access credentials in a file named
+``cloud-config-aws.yml`` or ``cloud-config-cs.ini`` in the test/integration
+directory. Corresponding .template files are available for for syntax help.  The newer AWS
+tests now use the file test/integration/cloud-config-aws.yml
+
+IAM policies for AWS
+====================
+
+Ansible needs fairly wide ranging powers to run the tests in an AWS account.  This rights can be provided to a dedicated user. These need to be configured before running the test.
+
+testing-iam-policy.json.j2
+--------------------------
+
+The testing-iam-policy.json.j2 file contains a policy which can be given to the user
+running the tests to minimize the rights of that user.  Please note that while this policy does limit the user to one region, this does not fully restrict the user (primarily due to the limitations of the Amazon ARN notation). The user will still have wide privileges for viewing account definitions, and will also able to manage some resources that are not related to testing (for example, AWS lambdas with different names).  Tests should not be run in a primary production account in any case.
+
+Other Definitions required
+--------------------------
+
+Apart from installing the policy and giving it to the user identity running the tests, a
+lambda role `ansible_integration_tests` has to be created which has lambda basic execution
+privileges.
+
+
+Network Tests
+=============
+
+This page details the specifics around testing Ansible Networking modules.
+
+
+.. important:: Network testing requirements for Ansible 2.4
+
+   Starting with Ansible 2.4, all network modules MUST include corresponding unit tests to defend functionality.
+   The unit tests must be added in the same PR that includes the new network module, or extends functionality.
+   Integration tests, although not required, are a welcome addition.
+   How to do this is explained in the rest of this document.
+
+
+Network integration tests can be ran by doing::
+
+    cd test/integration
+    ANSIBLE_ROLES_PATH=targets ansible-playbook network-all.yaml
+
+
+.. note::
+
+  * To run the network tests you will need a number of test machines and suitably configured inventory file. A sample is included in ``test/integration/inventory.network``
+  * As with the rest of the integration tests, they can be found grouped by module in ``test/integration/targets/MODULENAME/``
+
+To filter a set of test cases set ``limit_to`` to the name of the group, generally this is the name of the module::
+
+   ANSIBLE_ROLES_PATH=targets ansible-playbook -i inventory.network network-all.yaml -e "limit_to=eos_command"
+
+
+To filter a singular test case set the tags options to eapi or cli, set limit_to to the test group,
+and test_cases to the name of the test::
+
+   ANSIBLE_ROLES_PATH=targets ansible-playbook -i inventory.network network-all.yaml --tags="cli" -e "limit_to=eos_command test_case=notequal"
+
+
+
+Writing network integration tests
+---------------------------------
+
+Test cases are added to roles based on the module being testing. Test cases
+should include both cli and API test cases. Cli test cases should be
+added to ``test/integration/targets/modulename/tests/cli`` and API tests should be added to
+``test/integration/targets/modulename/tests/eapi``, or ``nxapi``.
+
+In addition to positive testing, negative tests are required to ensure user friendly warnings & errors are generated, rather than backtraces, for example:
+
+.. code-block: yaml
+
+   - name: test invalid subset (foobar)
+     eos_facts:
+       provider: "{{ cli }}"
+       gather_subset:
+         - "foobar"
+     register: result
+     ignore_errors: true
+
+   - assert:
+       that:
+         # Failures shouldn't return changes
+         - "result.changed == false"
+         # It's a failure
+         - "result.failed == true"
+         # Sensible Failure message
+         - "'Subset must be one of' in result.msg"
+
+
+Conventions
+```````````
+
+- Each test case should generally follow the pattern:
+
+  setup —> test —> assert —> test again (idempotent) —> assert —> teardown (if needed) -> done
+
+  This keeps test playbooks from becoming monolithic and difficult to
+  troubleshoot.
+
+- Include a name for each task that is not an assertion. (It's OK to add names
+  to assertions too. But to make it easy to identify the broken task within a failed
+  test, at least provide a helpful name for each task.)
+
+- Files containing test cases must end in `.yaml`
+
+
+Adding a new Network Platform
+`````````````````````````````
+
+A top level playbook is required such as ``ansible/test/integration/eos.yaml`` which needs to be references by ``ansible/test/integration/network-all.yaml``
+
+Where to find out more
+======================
+
+If you'd like to know more about the plans for improving testing Ansible then why not join the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.

docs/docsite/rst/dev_guide/testing_integration_legacy.rst

@@ -0,0 +1,105 @@
+*******************************************
+Testing using the Legacy Integration system
+*******************************************
+
+.. contents:: Topics
+
+This page details how to run the integration tests that haven't been ported to the new ``ansible-test`` framework.
+
+The following areas are still tested using the legacy ``make tests`` command:
+
+* amazon (some)
+* azure
+* cloudflare
+* cloudscale
+* cloudstack
+* consul
+* exoscale
+* gce
+* jenkins
+* rackspace
+
+Over time the above list will be reduced as tests are ported to the ``ansible-test`` framework.
+
+
+Running Cloud Tests
+====================
+
+Cloud tests exercise capabilities of cloud modules (e.g. ec2_key).  These are
+not 'tests run in the cloud' so much as tests that leverage the cloud modules
+and are organized by cloud provider.
+
+Some AWS tests may use environment variables. It is recommended to either unset any AWS environment variables( such as ``AWS_DEFAULT_PROFILE``, ``AWS_SECRET_ACCESS_KEY``, etc) or be sure that the environment variables match the credentials provided in ``credentials.yml`` to ensure the tests run with consistency to their full capability on the expected account. See `AWS CLI docs <http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html>`_ for information on creating a profile.
+
+Subsets of tests may be run by ``#commenting`` out unnecessary roles in the appropriate playbook, such as ``test/integration/amazon.yml``.
+
+In order to run cloud tests, you must provide access credentials in a file
+named ``credentials.yml``. A sample credentials file named
+``credentials.template`` is available for syntax help.
+
+Provide cloud credentials::
+
+    cp credentials.template credentials.yml
+    ${EDITOR:-vi} credentials.yml
+
+
+Other configuration
+===================
+
+In order to run some tests, you must provide access credentials in a file named
+``credentials.yml``. A sample credentials file named ``credentials.template`` is available
+for syntax help.
+
+IAM policies for AWS
+====================
+
+In order to run the tests in an AWS account ansible needs fairly wide ranging powers which
+can be provided to a dedicated user or temporary credentials using a specific policy
+configured in the AWS account.
+
+testing-iam-policy.json.j2
+--------------------------
+
+The testing-iam-policy.json.j2 file contains a policy which can be given to the user
+running the tests to give close to minimum rights required to run the tests.  Please note
+that this does not fully restrict the user; The user has wide privileges for viewing
+account definitions and is also able to manage some resources that are not related to
+testing (e.g. AWS lambdas with different names) primarily due to the limitations of the
+Amazon ARN notation.  At the very least the policy limits the user to one region, however
+tests should not be run in a primary production account in any case.
+
+Other Definitions required
+--------------------------
+
+Apart from installing the policy and giving it to the user identity running
+the tests, a lambda role `ansible_integration_tests` has to be created which
+has lambda basic execution privileges.
+
+
+Running Tests
+=============
+
+The tests are invoked via a ``Makefile``.
+
+If you haven't already got Ansible available use the local checkout by doing::
+
+  source hacking/env-setup
+
+Run the tests by doing::
+
+  cd test/integration/
+  # TARGET is the name of the test from the list at the top of this page
+  #make TARGET
+  # e.g.
+  make amazon
+  # To run all cloud tests you can do:
+  make cloud
+
+.. warning:: Possible cost of running cloud tests
+
+   Running cloud integration tests will create and destroy cloud
+   resources. Running these tests may result in additional fees associated with
+   your cloud account. Care is taken to ensure that created resources are
+   removed. However, it is advisable to inspect your AWS console to ensure no
+   unexpected resources are running.
+

docs/docsite/rst/dev_guide/testing_pep8.rst

@@ -0,0 +1,55 @@
+*****
+PEP 8
+*****
+
+.. contents:: Topics
+
+`PEP 8`_ style guidelines are enforced by `pycodestyle`_ on all python files in the repository by default.
+
+Current Rule Set
+================
+
+By default all files are tested using the current rule set.
+All `PEP 8`_ tests are executed, except those listed in the `current ignore list`_.
+
+.. warning: Updating the Rule Set
+
+   Changes to the Rule Set need approval from the Core Team, and must be done via the `Testing Working Group <https://github.com/ansible/community/blob/master/meetings/README.md>`_.
+
+Legacy Rule Set
+===============
+
+Files which are listed in the `legacy file list`_ are tested using the legacy rule set.
+
+All `PEP 8`_ tests are executed, except those listed in the `current ignore list`_ or `legacy ignore list`_.
+
+Files listed in the legacy file list which pass the current rule set will result in an error.
+
+This is intended to prevent regressions on style guidelines for files which pass the more stringent current rule set.
+
+Skipping Tests
+==============
+
+Files listed in the `skip list`_ are not tested by `pycodestyle`_.
+
+Removed Files
+=============
+
+Files which have been removed from the repository must be removed from the legacy file list and the skip list.
+
+Running Locally
+===============
+
+The `PEP 8`_ check can be run locally with::
+
+
+    ./test/runner/ansible-test sanity --test pep8 [file-or-directory-path-to-check] ...
+
+
+
+.. _PEP 8: https://www.python.org/dev/peps/pep-0008/
+.. _pycodestyle: https://pypi.python.org/pypi/pycodestyle
+.. _current ignore list: https://github.com/ansible/ansible/blob/devel/test/sanity/pep8/current-ignore.txt
+.. _legacy file list: https://github.com/ansible/ansible/blob/devel/test/sanity/pep8/legacy-files.txt
+.. _legacy ignore list: https://github.com/ansible/ansible/blob/devel/test/sanity/pep8/legacy-ignore.txt
+.. _skip list: https://github.com/ansible/ansible/blob/devel/test/sanity/pep8/skip.txt

docs/docsite/rst/dev_guide/testing_running_locally.rst

@@ -0,0 +1,56 @@
+***************
+Testing Ansible
+***************
+
+.. contents:: Topics
+
+This document describes how to:
+
+* Run tests locally using ``ansible-test``
+* Extend
+
+Requirements
+============
+
+There are no special requirements for running ``ansible-test`` on Python 2.7 or later.
+The ``argparse`` package is required for Python 2.6.
+The requirements for each ``ansible-test`` command are covered later.
+
+
+Test Environments
+=================
+
+Most ``ansible-test`` commands support running in one or more isolated test environments to simplify testing.
+
+
+Remote
+------
+
+The ``--remote`` option runs tests in a cloud hosted environment.
+An API key is required to use this feature.
+
+    Recommended for integration tests.
+
+See the `list of supported platforms and versions <https://github.com/ansible/ansible/blob/devel/test/runner/completion/remote.txt>`_ for additional details.
+
+
+Interactive Shell
+=================
+
+Use the ``ansible-test shell`` command to get an interactive shell in the same environment used to run tests. Examples:
+
+* ``ansible-test shell --docker`` - Open a shell in the default docker container.
+* ``ansible-test shell --tox --python 3.6`` - Open a shell in the Python 3.6 ``tox`` environment.
+
+Code Coverage
+=============
+
+Add the ``--coverage`` option to any test command to collect code coverage data.
+
+Reports can be generated in several different formats:
+
+* ``ansible-test coverage report`` - Console report.
+* ``ansible-test coverage html`` - HTML report.
+* ``ansible-test coverage xml`` - XML report.
+
+To clear data between test runs, use the ``ansible-test coverage erase`` command.

docs/docsite/rst/dev_guide/testing_sanity.rst

@@ -0,0 +1,75 @@
+************
+Sanity Tests
+************
+
+.. contents:: Topics
+
+Sanity tests are made up of scripts and tools used to perform static code analysis.
+The primary purpose of these tests is to enforce Ansible coding standards and requirements.
+
+Tests are run with ``ansible-test sanity``.
+All available tests are run unless the ``--test`` option is used.
+
+Available Tests
+===============
+
+Tests can be listed with ``ansible-test sanity --list-tests``.
+
+This list is a combination of two different categories of tests, "Code Smell" and "Built-in".
+
+Code Smell Tests
+----------------
+
+Miscellaneous `scripts <https://github.com/ansible/ansible/tree/devel/test/sanity/code-smell/>`_ used for enforcing coding standards and requirements, identifying trip hazards, etc.
+
+These tests are listed and accessed by script name. There is no actual test named ``code-smell``.
+
+All executable scripts added to the ``code-smell`` directory are automatically detected and executed by ``ansible-test``.
+
+Scripts in the directory which fail can be skipped by adding them to `skip.txt <https://github.com/ansible/ansible/blob/devel/test/sanity/code-smell/skip.txt>`_.
+This is useful for scripts which identify issues that have not yet been resolved in the code base.
+
+Files tested are specific to the individual test scripts and are not affected by command line arguments.
+
+Built-in Tests
+--------------
+
+These tests are integrated directly into ``ansible-test``.
+All files relevant to each test are tested unless specific files are specified.
+
+A full list of tests can be obtained by doing ``ansible-test sanity --list-tests``.
+
+ansible-doc
+~~~~~~~~~~~
+
+Verifies that ``ansible-doc`` can parse module documentation on all supported python versions.
+
+pep8
+~~~~
+
+Python static analysis for PEP 8 style guideline compliance. See :doc:`testing_pep8` for more information.
+
+pylint
+~~~~~~
+
+Python static analysis for common programming errors.
+
+rstcheck
+~~~~~~~~
+
+Check reStructuredText files for syntax and formatting issues.
+
+shellcheck
+~~~~~~~~~~
+
+Static code analysis for shell scripts using the excellent `shellcheck <https://www.shellcheck.net/>`_ tool.
+
+validate-modules
+~~~~~~~~~~~~~~~~
+
+Analyze modules for common issues in code and documentation. See :doc:`testing_validate-modules` for more information.
+
+yamllint
+~~~~~~~~
+
+Check YAML files for syntax and formatting issues.

docs/docsite/rst/dev_guide/testing_units.rst

@@ -0,0 +1,105 @@
+**********
+Unit Tests
+**********
+
+Unit tests are small isolated tests that target a specific library or module.
+
+.. contents:: Topics
+
+Available Tests
+===============
+
+Unit tests can be found in `test/units <https://github.com/ansible/ansible/tree/devel/test/units>`_, notice that the directory structure matches that of ``lib/ansible/``
+
+Running Tests
+=============
+
+Unit tests can be run across the whole code base by doing:
+
+.. code:: shell
+
+    cd /path/to/ansible/source
+    source hacking/env-setup
+    ansible-test units --tox
+
+Against a single file by doing:
+
+.. code:: shell
+
+   ansible-test units --tox apt
+
+Or against a specific Python version by doing:
+
+.. code:: shell
+
+   ansible-test units --tox --python 2.7 apt
+
+
+
+For advanced usage see the online help::
+
+   ansible-test units --help
+
+
+Installing dependencies
+=======================
+
+``ansible-test`` has a number of dependencies , for ``units`` tests we suggest using ``tox``
+
+The dependencies can be installed using the ``--requirements`` argument, which will install all the required dependencies needed for unit tests. For example:
+
+.. code:: shell
+
+   ansible-test units --tox --python 2.7 --requirements apache2_module
+
+
+.. note:: tox version requirement
+
+   When using ``ansible-test`` with ``--tox`` requires tox >= 2.5.0
+
+
+The full list of requirements can be found at `test/runner/requirements <https://github.com/ansible/ansible/tree/devel/test/runner/requirements>`_. Requirements files are named after their respective commands. See also the `constraints <https://github.com/ansible/ansible/blob/devel/test/runner/requirements/constraints.txt>`_ applicable to all commands.
+
+
+Extending unit tests
+====================
+
+
+.. warning:: What a unit test isn't
+
+   If you start writing a test that requires external services then you may be writing an integration test, rather than a unit test.
+
+Fixtures files
+``````````````
+
+To mock out fetching results from devices, you can use ``fixtures`` to read in pre-generated data.
+
+Text files live in ``test/units/modules/network/PLATFORM/fixtures/``
+
+Data is loaded using the ``load_fixture`` method
+
+See  `eos_banner test <https://github.com/ansible/ansible/blob/devel/test/units/modules/network/eos/test_eos_banner.py>`_ for a practical example.
+
+Code Coverage
+`````````````
+Most ``ansible-test`` commands allow you to collect code coverage, this is particularly useful when to indicate where to extend testing.
+
+To collect coverage data add the ``--coverage`` argument to your ``ansible-test`` command line:
+
+.. code:: shell
+
+   ansible-test units --coverage apt
+   ansible-test coverage html
+
+Results will be written to ``test/results/reports/coverage/index.html``
+
+Reports can be generated in several different formats:
+
+* ``ansible-test coverage report`` - Console report.
+* ``ansible-test coverage html`` - HTML report.
+* ``ansible-test coverage xml`` - XML report.
+
+To clear data between test runs, use the ``ansible-test coverage erase`` command. For a full list of features see the online help::
+
+   ansible-test coverage --help
+

docs/docsite/rst/dev_guide/testing_validate-modules.rst

@@ -0,0 +1,183 @@
+****************
+validate-modules
+****************
+
+.. contents:: Topics
+
+Python program to help test or validate Ansible modules.
+
+``validate-modules`` is one of the ``ansible-test`` Sanity Tests, see :doc:`testing_sanity` for more information.
+
+Originally developed by Matt Martz (@sivel)
+
+
+Usage
+=====
+
+.. code:: shell
+
+    cd /path/to/ansible/source
+    source hacking/env-setup
+    ansible-test sanity --test validate-modules
+
+Help
+====
+
+.. code:: shell
+
+    usage: validate-modules [-h] [-w] [--exclude EXCLUDE] [--arg-spec]
+                            [--base-branch BASE_BRANCH] [--format {json,plain}]
+                            [--output OUTPUT]
+                            modules [modules ...]
+
+    positional arguments:
+      modules               Path to module or module directory
+
+    optional arguments:
+      -h, --help            show this help message and exit
+      -w, --warnings        Show warnings
+      --exclude EXCLUDE     RegEx exclusion pattern
+      --arg-spec            Analyze module argument spec
+      --base-branch BASE_BRANCH
+                            Used in determining if new options were added
+      --format {json,plain}
+                            Output format. Default: "plain"
+      --output OUTPUT       Output location, use "-" for stdout. Default "-"
+
+
+Extending validate-modules
+==========================
+
+The ``validate-modules`` tool has a `schema.py <https://github.com/ansible/ansible/blob/devel/test/sanity/validate-modules/schema.py>`_ that is used to validate the YAML blocks, such as ``DOCUMENTATION`` and ``RETURNS``.
+
+
+Codes
+=====
+
+Errors
+------
+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| code    | sample message                                                                                                                             |
++=========+============================================================================================================================================+
+| **1xx** | **Locations**                                                                                                                              |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 101     | Interpreter line is not ``#!/usr/bin/python``                                                                                              |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 102     | Interpreter line is not ``#!powershell``                                                                                                   |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 103     | Did not find a call to ``main()``                                                                                                          |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 104     | Call to ``main()`` not the last line                                                                                                       |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 105     | GPLv3 license header not found                                                                                                             |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 106     | Import found before documentation variables. All imports must appear below ``DOCUMENTATION``/``EXAMPLES``/``RETURN``/``ANSIBLE_METADATA``  |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 107     | Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN``/``ANSIBLE_METADATA``                                            |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| **2xx** | **Imports**                                                                                                                                |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 201     | Did not find a ``module_utils`` import                                                                                                     |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 203     | ``requests`` import found, should use ``ansible.module_utils.urls`` instead                                                                |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 204     | ``boto`` import found, new modules should use ``boto3``                                                                                    |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 205     | ``sys.exit()`` call found. Should be ``exit_json``/``fail_json``                                                                           |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 206     | ``WANT_JSON`` not found in module                                                                                                          |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 207     | ``REPLACER_WINDOWS`` not found in module                                                                                                   |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 208     | ``module_utils`` imports should import specific components, not ``*``                                                                      |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| **3xx** | **Documentation**                                                                                                                          |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 301     | No ``DOCUMENTATION`` provided                                                                                                              |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 302     | ``DOCUMENTATION`` is not valid YAML                                                                                                        |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 303     | ``DOCUMENTATION`` fragment missing                                                                                                         |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 304     | Unknown ``DOCUMENTATION`` error                                                                                                            |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 305     | Invalid ``DOCUMENTATION`` schema                                                                                                           |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 306     | Module level ``version_added`` is not a valid version number                                                                               |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 307     | Module level ``version_added`` is incorrect                                                                                                |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 308     | ``version_added`` for new option is not a valid version number                                                                             |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 309     | ``version_added`` for new option is incorrect                                                                                              |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 310     | No ``EXAMPLES`` provided                                                                                                                   |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 311     | ``EXAMPLES`` is not valid YAML                                                                                                             |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 312     | No ``RETURN`` documentation provided                                                                                                       |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 313     | ``RETURN`` is not valid YAML                                                                                                               |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 314     | No ``ANSIBLE_METADATA`` provided                                                                                                           |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 315     | ``ANSIBLE_METADATA`` is not valid YAML                                                                                                     |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 316     | Invalid ``ANSIBLE_METADATA`` schema                                                                                                        |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 317     | option is marked as required but specifies a default. Arguments with a default should not be marked as required                            |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 318     | Module deprecated, but DOCUMENTATION.deprecated is missing                                                                                 |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 319     | ``RETURN`` fragments missing  or invalid                                                                                                   |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| **4xx** | **Syntax**                                                                                                                                 |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 401     | Python ``SyntaxError`` while parsing module                                                                                                |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 402     | Indentation contains tabs                                                                                                                  |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 403     | Type comparison using ``type()`` found. Use ``isinstance()`` instead                                                                       |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| **5xx** | **Naming**                                                                                                                                 |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 501     | Official Ansible modules must have a ``.py`` extension for python modules or a ``.ps1`` for powershell modules                             |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 502     | Ansible module subdirectories must contain an ``__init__.py``                                                                              |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 503     | Missing python documentation file                                                                                                          |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+
+Warnings
+--------
+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| code    | sample message                                                                                                                             |
++=========+============================================================================================================================================+
+| **1xx** | **Locations**                                                                                                                              |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 107     | Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN``/``ANSIBLE_METADATA`` for legacy modules                         |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| **2xx** | **Imports**                                                                                                                                |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 208     | ``module_utils`` imports should import specific components for legacy module, not ``*``                                                    |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 291     | Try/Except ``HAS_`` expression missing                                                                                                     |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 292     | Did not find ``ansible.module_utils.basic`` import                                                                                         |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| **3xx** | **Documentation**                                                                                                                          |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 312     | No ``RETURN`` documentation provided for legacy module                                                                                     |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 391     | Unknown pre-existing ``DOCUMENTATION`` error                                                                                               |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+
+| 392     | Pre-existing ``DOCUMENTATION`` fragment missing                                                                                            |
++---------+--------------------------------------------------------------------------------------------------------------------------------------------+

docs/docsite/rst/faq.rst

@@ -119,10 +119,9 @@ used when executing python modules.   Thus, you can point to any python you want
 system does not point to a Python 2.X interpreter.  
 
 Some Linux operating systems, such as Arch, may only have Python 3 installed by default.  This is not sufficient and you will
-get syntax errors trying to run modules with Python 3.  Python 3 is essentially not the same
-language as Python 2.  Ansible modules currently need to support older Pythons for users that  still have Enterprise Linux 5 deployed, so they are not yet ported to run under Python 3.0.  This is not a problem though as you can just install Python 2 also on a managed host.
-
-Python 3.0 support will likely be addressed at a later point in time when usage becomes more mainstream.
+get syntax errors trying to run modules with Python 3.  Python 3 is essentially not the same language as Python 2.  Python 3
+support is being worked on but some Ansible modules are not yet ported to run under Python 3.0.  This is not a problem though
+as you can just install Python 2 also on a managed host.
 
 Do not replace the shebang lines of your python modules.  Ansible will do this for you automatically at deploy time.
 

docs/docsite/rst/guide_packet.rst

@@ -0,0 +1,309 @@
+**********************************
+Using Ansible with the Packet host
+**********************************
+
+Introduction
+============
+
+`Packet.net <https://packet.net>`_ is a bare metal infrastructure host that's supported by Ansible (>=2.3) via a dynamic inventory script and two cloud modules. The two modules are:
+
+- packet_sshkey: adds a public SSH key from file or value to the Packet infrastructure. Every subsequently-created device will have this public key installed in .ssh/authorized_keys.
+- packet_device: manages servers on Packet. You can use this module to create, restart and delete devices.
+
+Note, this guide assumes you are familiar with Ansible and how it works. If you're not, have a look at their `docs <http://docs.ansible.com/>`_ before getting started.
+
+Requirements
+============
+
+The Packet modules and inventory script connect to the Packet API using the packet-python package. You can install it with pip:
+
+.. code-block:: bash
+
+    $ pip install packet-python
+
+In order to check the state of devices created by Ansible on Packet, it's a good idea to install one of the `Packet CLI clients <https://www.packet.net/developers/integrations/api-cli/>`_. Otherwise you can check them via the `Packet portal <https://app.packet.net/portal>`_.
+
+To use the modules and inventory script you'll need a Packet API token. You can generate an API token via the Packet portal `here <https://app.packet.net/portal#/api-keys>`__. The simplest way to authenticate yourself is to set the Packet API token in an environment variable:
+
+.. code-block:: bash
+
+    $ export PACKET_API_TOKEN=Bfse9F24SFtfs423Gsd3ifGsd43sSdfs
+
+If you're not comfortable exporting your API token, you can pass it as a parameter to the modules.
+
+On Packet, devices and reserved IP addresses belong to `projects <https://www.packet.net/developers/api/projects/>`_. In order to use the packet_device module, you need to specify the UUID of the project in which you want to create or manage devices. You can find a project's UUID in the Packet portal `here <https://app.packet.net/portal#/projects/list/table/>`_ (it's just under the project table) or via one of the available `CLIs <https://www.packet.net/developers/integrations/api-cli/>`_.
+
+
+If you want to use a new SSH keypair in this tutorial, you can generate it to ``./id_rsa`` and ``./id_rsa.pub`` as:
+
+.. code-block:: bash
+
+    $ ssh-keygen -t rsa -f ./id_rsa
+
+If you want to use an existing keypair, just copy the private and public key over to the playbook directory.
+
+
+Device Creation
+===============
+
+The following code block is a simple playbook that creates one `Type 0 <https://www.packet.net/bare-metal/servers/type-0/>`_ server (the 'plan' parameter). You have to supply 'plan' and 'operating_system'. 'location' defaults to 'ewr1' (Parsippany, NJ). You can find all the possible values for the parameters via a `CLI client <https://www.packet.net/developers/integrations/api-cli/>`_.
+
+.. code-block:: yaml
+
+    # playbook_create.yml
+
+    - name: create ubuntu device
+      hosts: localhost
+      tasks:
+
+      - packet_sshkey:
+          key_file: ./id_rsa.pub
+          label: tutorial key
+
+      - packet_device:
+          project_id: <your_project_id>
+          hostnames: myserver
+          operating_system: ubuntu_16_04
+          plan: baremetal_0
+          facility: sjc1
+
+After running ``ansible-playbook playbook_create.yml``, you should have a server provisioned on Packet. You can verify via a CLI or in the `Packet portal <https://app.packet.net/portal#/projects/list/table>`__.
+
+If you get an error with the message "failed to set machine state present, error: Error 404: Not Found", please verify your project UUID.
+
+
+Updating Devices
+================
+
+The two parameters used to uniquely identify Packet devices are: "device_ids" and "hostnames". Both parameters accept either a single string (later converted to a one-element list), or a list of strings.
+
+The 'device_ids' and 'hostnames' parameters are mutually exclusive. The following values are all acceptable:
+
+- device_ids: a27b7a83-fc93-435b-a128-47a5b04f2dcf
+
+- hostnames: mydev1
+
+- device_ids: [a27b7a83-fc93-435b-a128-47a5b04f2dcf, 4887130f-0ccd-49a0-99b0-323c1ceb527b]
+
+- hostnames: [mydev1, mydev2]
+
+In addition, hostnames can contain a special '%d' formatter along with a 'count' parameter that lets you easily expand hostnames that follow a simple name and number pattern; i.e. ``hostnames: "mydev%d", count: 2`` will expand to [mydev1, mydev2].
+
+If your playbook acts on existing Packet devices, you can only pass the 'hostname' and 'device_ids' parameters. The following playbook shows how you can reboot a specific Packet device by setting the 'hostname' parameter:
+
+.. code-block:: yaml
+
+    # playbook_reboot.yml
+
+    - name: reboot myserver
+      hosts: localhost
+      tasks:
+
+      - packet_device:
+          project_id: <your_project_id>
+          hostnames: myserver
+          state: rebooted
+
+You can also identify specific Packet devices with the 'device_ids' parameter. The device's UUID can be found in the `Packet Portal <https://app.packet.net/portal>`_ or by using a `CLI <https://www.packet.net/developers/integrations/api-cli/>`_. The following playbook removes a Packet device using the 'device_ids' field:
+
+.. code-block:: yaml
+
+    # playbook_remove.yml
+
+    - name: remove a device
+      hosts: localhost
+      tasks:
+
+      - packet_device:
+          project_id: <your_project_id>
+          device_ids: <myserver_device_id>
+          state: absent
+
+
+More Complex Playbooks
+======================
+
+In this example, we'll create a CoreOS cluster with `user data <https://support.packet.net/en/support/solutions/articles/22000058261-the-basics-of-cloud-config-and-user-data>`_.
+
+
+The CoreOS cluster will use `etcd <https://coreos.com/etcd/>`_ for discovery of other servers in the cluster. Before provisioning your servers, you'll need to generate a discovery token for your cluster:
+
+.. code-block:: bash
+
+    $ curl -w "\n" 'https://discovery.etcd.io/new?size=3'
+
+The following playbook will create an SSH key, 3 Packet servers, and then wait until SSH is ready (or until 5 minutes passed). Make sure to substitute the discovery token URL in 'user_data', and the 'project_id' before running ``ansible-playbook``. Also, feel free to change 'plan' and 'facility'.
+
+.. code-block:: yaml
+
+    # playbook_coreos.yml
+
+    - name: Start 3 CoreOS nodes in Packet and wait until SSH is ready
+      hosts: localhost
+      tasks:
+
+      - packet_sshkey:
+          key_file: ./id_rsa.pub
+          label: new
+
+      - packet_device:
+          hostnames: [coreos-one, coreos-two, coreos-three]
+          operating_system: coreos_beta
+          plan: baremetal_0
+          facility: ewr1
+          project_id: <your_project_id>
+          wait: true
+          user_data: |
+            #cloud-config
+            coreos:
+              etcd2:
+                discovery: https://discovery.etcd.io/<token>
+                advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001
+                initial-advertise-peer-urls: http://$private_ipv4:2380
+                listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
+                listen-peer-urls: http://$private_ipv4:2380
+              fleet:
+                public-ip: $private_ipv4
+              units:
+                - name: etcd2.service
+                  command: start
+                - name: fleet.service
+                  command: start
+        register: newhosts
+
+      - name: wait for ssh
+        wait_for:
+          delay: 1
+          host: "{{ item.public_ipv4 }}"
+          port: 22
+          state: started
+          timeout: 500
+        with_items: "{{ newhosts.devices }}"
+
+
+As with most Ansible modules, the default states of the Packet modules are idempotent, meaning the resources in your project will remain the same after re-runs of a playbook. Thus, we can keep the ``packet_sshkey`` module call in our playbook. If the public key is already in your Packet account, the call will have no effect.
+
+The second module call provisions 3 Packet Type 0 (specified using the 'plan' parameter) servers in the project identified via the 'project_id' parameter. The servers are all provisioned with CoresOS beta (the 'operating_system' parameter) and are customized with cloud-config user data passed to the 'user_data' parameter.
+
+The ``packet_device`` module has a boolean 'wait' parameter that defaults to 'false'. If set to 'true', Ansible will wait until the GET API call for a device will contain an Internet-routeable IP address. The 'wait' parameter allows us to use the IP address of the device as soon as it's available.
+
+Run the playbook:
+
+.. code-block:: bash
+
+    $ ansible-playbook playbook_coreos.yml
+
+Once the playbook quits, your new devices should be reachable via SSH. Try to connect to one and check if etcd has started properly:
+
+.. code-block:: bash
+
+    tomk@work $ ssh -i id_rsa core@$one_of_the_servers_ip
+    core@coreos-one ~ $ etcdctl cluster-health
+
+Once you create a couple of devices, you might appreciate the dynamic inventory script...
+
+
+Dynamic Inventory Script
+========================
+
+The dynamic inventory script queries the Packet API for a list of hosts, and exposes it to Ansible so you can easily identify and act on Packet devices. You can find it in Ansible's git repo at `contrib/inventory/packet_net.py <https://github.com/ansible/ansible/blob/devel/contrib/inventory/packet_net.py>`_.
+
+The inventory script is configurable via a `ini file <https://github.com/ansible/ansible/blob/devel/contrib/inventory/packet_net.ini>`_.
+
+If you want to use the inventory script, you must first export your Packet API token to a PACKET_API_TOKEN environment variable.
+
+You can either copy the inventory and ini config out from the cloned git repo, or you can download it to your working directory like so:
+
+.. code-block:: bash
+
+    $ wget https://github.com/ansible/ansible/raw/devel/contrib/inventory/packet_net.py
+    $ chmod +x packet_net.py
+    $ wget https://github.com/ansible/ansible/raw/devel/contrib/inventory/packet_net.ini
+
+In order to understand what the inventory script gives to Ansible you can run:
+
+.. code-block:: bash
+
+    $ ./packet_net.py --list
+
+It should print a JSON document looking similar to following trimmed dictionary:
+
+.. code-block:: json
+
+    {
+      "_meta": {
+        "hostvars": {
+          "147.75.64.169": {
+            "packet_billing_cycle": "hourly",
+            "packet_created_at": "2017-02-09T17:11:26Z",
+            "packet_facility": "ewr1",
+            "packet_hostname": "coreos-two",
+            "packet_href": "/devices/d0ab8972-54a8-4bff-832b-28549d1bec96",
+            "packet_id": "d0ab8972-54a8-4bff-832b-28549d1bec96",
+            "packet_locked": false,
+            "packet_operating_system": "coreos_beta",
+            "packet_plan": "baremetal_0",
+            "packet_state": "active",
+            "packet_updated_at": "2017-02-09T17:16:35Z",
+            "packet_user": "core",
+            "packet_userdata": "#cloud-config\ncoreos:\n  etcd2:\n    discovery: https://discovery.etcd.io/e0c8a4a9b8fe61acd51ec599e2a4f68e\n    advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001\n    initial-advertise-peer-urls: http://$private_ipv4:2380\n    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001\n    listen-peer-urls: http://$private_ipv4:2380\n  fleet:\n    public-ip: $private_ipv4\n  units:\n    - name: etcd2.service\n      command: start\n    - name: fleet.service\n      command: start"
+          }
+        }
+      },
+      "baremetal_0": [
+        "147.75.202.255",
+        "147.75.202.251",
+        "147.75.202.249",
+        "147.75.64.129",
+        "147.75.192.51",
+        "147.75.64.169"
+      ],
+      "coreos_beta": [
+        "147.75.202.255",
+        "147.75.202.251",
+        "147.75.202.249",
+        "147.75.64.129",
+        "147.75.192.51",
+        "147.75.64.169"
+      ],
+      "ewr1": [
+        "147.75.64.129",
+        "147.75.192.51",
+        "147.75.64.169"
+      ],
+      "sjc1": [
+        "147.75.202.255",
+        "147.75.202.251",
+        "147.75.202.249"
+      ],
+      "coreos-two": [
+        "147.75.64.169"
+      ],
+      "d0ab8972-54a8-4bff-832b-28549d1bec96": [
+        "147.75.64.169"
+      ]
+    }
+
+In the ``['_meta']['hostvars']`` key, there is a list of devices (uniquely identified by their public IPv4 address) with their parameters. The other keys under ``['_meta']`` are lists of devices grouped by some parameter. Here, it is type (all devices are of type baremetal_0), operating system, and facility (ewr1 and sjc1).
+
+In addition to the parameter groups, there are also one-item groups with the UUID or hostname of the device.
+
+You can now target groups in playbooks! The following playbook will install a role that supplies resources for an Ansible target into all devices in the "coreos_beta" group:
+
+.. code-block:: yaml
+
+    # playbook_bootstrap.yml
+
+    - hosts: coreos_beta
+      gather_facts: false
+      roles:
+        - defunctzombie.coreos-boostrap
+
+Don't forget to supply the dynamic inventory in the ``-i`` argument!
+
+.. code-block:: bash
+
+    $ ansible-playbook -u core -i packet_net.py playbook_bootstrap.yml
+
+
+If you have any questions or comments let us know! help@packet.net

docs/docsite/rst/guides.rst

@@ -14,5 +14,6 @@ This section is new and evolving.  The idea here is to explore particular use ca
    guide_vagrant
    guide_rolling_upgrade
    guide_docker
+   guide_packet
 
 Pending topics may include: Docker, Jenkins, Google Compute Engine, Linode/DigitalOcean, Continuous Deployment, and more.

docs/docsite/rst/index.rst

@@ -7,7 +7,7 @@ About Ansible
 Welcome to the Ansible documentation!
 
 Ansible is an IT automation tool.  It can configure systems, deploy software, and orchestrate more advanced IT tasks
-such as continuous deployments or zero downtime rolling updates.  
+such as continuous deployments or zero downtime rolling updates.
 
 Ansible's main goals are simplicity and ease-of-use. It also has a strong focus on security and reliability, featuring a minimum of moving parts, usage of OpenSSH for transport (with an accelerated socket mode and pull modes as alternatives), and a language that is designed around auditability by humans--even those not familiar with the program.
 
@@ -16,7 +16,7 @@ We believe simplicity is relevant to all sizes of environments, so we design for
 Ansible manages machines in an agent-less manner. There is never a question of how to
 upgrade remote daemons or the problem of not being able to manage systems because daemons are uninstalled.  Because OpenSSH is one of the most peer-reviewed open source components, security exposure is greatly reduced. Ansible is decentralized--it relies on your existing OS credentials to control access to remote machines. If needed, Ansible can easily connect with Kerberos, LDAP, and other centralized authentication management systems.
 
-This documentation covers the current released version of Ansible (|version|) and also some development version features (|versiondev|).  For recent features, we note in each section the version of Ansible where the feature was added.  
+This documentation covers the current released version of Ansible (|version|) and also some development version features (|versiondev|).  For recent features, we note in each section the version of Ansible where the feature was added.
 
 Ansible, Inc. releases a new major release of Ansible approximately every two months.  The core application evolves somewhat conservatively, valuing simplicity in language design and setup. However, the community around new modules and plugins being developed and contributed moves very quickly, typically adding 20 or so new modules in each release.
 
@@ -33,13 +33,13 @@ Ansible, Inc. releases a new major release of Ansible approximately every two mo
    modules_by_category
    guides
    dev_guide/index
+   community/index
    tower
-   community
    galaxy
    test_strategies
    faq
    glossary
    YAMLSyntax
-   porting_guide_2.0
+   porting_guides
    python_3_support
-
+   release_and_maintenance

docs/docsite/rst/intro_configuration.rst

@@ -86,6 +86,20 @@ different locations::
 Most users will not need to use this feature.  See :doc:`dev_guide/developing_plugins` for more details.
 
 
+.. _allow_unsafe_lookups:
+
+allow_unsafe_lookups
+====================
+
+.. versionadded:: 2.2.3, 2.3.1
+
+When enabled, this option allows lookup plugins (whether used in variables as `{{lookup('foo')}}` or as a loop as `with_foo`) to return data that is **not** marked "unsafe". By default, such data is marked as unsafe to prevent the templating engine from evaluating any jinja2 templating language, as this could represent a security risk.
+
+This option is provided to allow for backwards-compatibility, however users should first consider adding `allow_unsafe=True` to any lookups which may be expected to contain data which may be run through the templating engine later. For example::
+
+    {{lookup('pipe', '/path/to/some/command', allow_unsafe=True)}}
+
+
 .. _allow_world_readable_tmpfiles:
 
 allow_world_readable_tmpfiles

docs/docsite/rst/intro_installation.rst

@@ -270,7 +270,7 @@ Then install Ansible with [1]_::
 
 Or if you are looking for the latest development version::
 
-    pip install git+git://github.com/ansible/ansible.git@devel
+    pip install git+https://github.com/ansible/ansible.git@devel
 
 If you are installing on OS X Mavericks, you may encounter some noise from your compiler.  A workaround is to do the following::
 
@@ -285,8 +285,6 @@ Tarballs of Tagged Releases
 
 Packaging Ansible or wanting to build a local package yourself, but don't want to do a git checkout?  Tarballs of releases are available on the `Ansible downloads <http://releases.ansible.com/ansible>`_ page.
 
-These releases are also tagged in the `git repository <https://github.com/ansible/ansible/releases>`_ with the release version.
-
 .. _from_source:
 
 Running From Source

docs/docsite/rst/intro_networking.rst

@@ -169,9 +169,9 @@ Ansible allows you to use conditionals to control the flow of your playbooks. An
 * contains - Object contains specified item
 
 
-Conditional statements evalute the results from the commands that are
+Conditional statements evaluate the results from the commands that are
 executed remotely on the device.  Once the task executes the command
-set, the waitfor argument can be used to evalute the results before
+set, the wait_for argument can be used to evaluate the results before
 returning control to the Ansible playbook.
 
 For example::

docs/docsite/rst/intro_windows.rst

@@ -107,13 +107,13 @@ Edit your /etc/krb5.conf (which should be installed as a result of installing pa
 
 In the section that starts with
 
-.. code-block:: bash
+.. code-block:: ini
 
    [realms]
 
 add the full domain name and the fully qualified domain names of your primary and secondary Active Directory domain controllers.  It should look something like this:
 
-.. code-block:: bash
+.. code-block:: ini
 
    [realms]
    
@@ -125,7 +125,7 @@ add the full domain name and the fully qualified domain names of your primary an
 
 and in the [domain_realm] section add a line like the following for each domain you want to access:
 
-.. code-block:: bash
+.. code-block:: ini
 
     [domain_realm]
         .my.domain.com = MY.DOMAIN.COM

docs/docsite/rst/modules_support.rst

@@ -26,8 +26,7 @@ Curated
 
 Some examples of Curated modules are submitted by other companies or maintained by the community. Maintainers of these types of modules must watch for any issues reported or pull requests raised against the module.
 
-Core Committers will review all modules becoming Curated or updates to existing Curated modules. Core committers will also ensure that any issues that arise due to Ansible engine changes will be remediated.
-
+Core Committers will review all modules becoming Curated.  Core Committers will review proposed changes to existing Curated modules once the community maintainers of the module have approved the changes. Core committers will also ensure that any issues that arise due to Ansible engine changes will be remediated.
 Also, it is strongly recommended (but not presently required) for these types of modules to have unit tests.
 
 These modules are currently shipped with Ansible, but might be shipped separately in the future.

docs/docsite/rst/network_debug_troubleshooting.rst

@@ -0,0 +1,474 @@
+.. _network_debug_troubleshooting:
+
+***************************************
+Network Debug and Troubleshooting Guide
+***************************************
+
+.. contents:: Topics
+
+Introduction
+============
+
+Starting with Ansible version 2.1, you can now use the familiar Ansible models of playbook authoring and module development to manage heterogeneous networking devices. Ansible supports a growing number of network devices using both CLI over SSH and API (when available) transports.
+
+This section discusses how to debug and troubleshoot network modules in Ansible 2.3.
+
+
+
+
+
+How to troubleshoot
+===================
+
+This section covers troubleshooting issues with Network Modules.
+
+Errors generally fall into one of the following categories:
+
+:Authentication issues:
+  * Not correctly specifying credentials
+  * Remote device (network switch/router) not falling back to other other authentication methods
+  * SSH key issues
+:Timeout issues:
+  * Can occur when trying to pull a large amount of data
+  * May actually be masking a authentication issue
+:Playbook issues:
+  * Use of ``delegate_to``, instead of ``ProxyCommand``
+  * Not using ``connection: local``
+
+
+.. warning:: ``unable to open shell``
+
+  The ``unable to open shell`` message is new in Ansible 2.3, it means that the ``ansible-connection`` daemon has not been able to successfully
+  talk to the remote network device. This generally means that there is an authentication issue. See the "Authentication and connection issues" section
+  in this document for more information.
+
+.. _enable_network_logging:
+
+Enabling Networking logging and how to read the logfile
+-------------------------------------------------------
+
+**Platforms:** Any
+
+Ansible 2.3 features improved logging to help diagnose and troubleshoot issues regarding Ansible Networking modules.
+
+Because logging is very verbose it is disabled by default. It can be enabled via the ``ANSIBLE_LOG_PATH`` and ``ANISBLE_DEBUG`` options::
+
+   # Specify the location for the log file
+   export ANSIBLE_LOG_PATH=~/ansible.log
+   # Enable Debug
+   export ANSIBLE_DEBUG=True
+
+   # Run with 4*v for connection level verbosity
+   ansible-playbook -vvvv ...
+
+After Ansible has finished running you can inspect the log file:
+
+.. code::
+
+  2017-03-30 13:19:52,740 p=28990 u=fred |  creating new control socket for host veos01:22 as user admin
+  2017-03-30 13:19:52,741 p=28990 u=fred |  control socket path is /home/fred/.ansible/pc/ca5960d27a
+  2017-03-30 13:19:52,741 p=28990 u=fred |  current working directory is /home/fred/ansible/test/integration
+  2017-03-30 13:19:52,741 p=28990 u=fred |  using connection plugin network_cli
+  ...
+  2017-03-30 13:20:14,771 paramiko.transport userauth is OK
+  2017-03-30 13:20:15,283 paramiko.transport Authentication (keyboard-interactive) successful!
+  2017-03-30 13:20:15,302 p=28990 u=fred |  ssh connection done, setting terminal
+  2017-03-30 13:20:15,321 p=28990 u=fred |  ssh connection has completed successfully
+  2017-03-30 13:20:15,322 p=28990 u=fred |  connection established to veos01 in 0:00:22.580626
+
+
+From the log notice:
+
+* ``p=28990`` Is the PID (Process ID) of the ``ansible-connection`` process
+* ``u=fred`` Is the user `running` ansible, not the remote-user you are attempting to connect as
+* ``creating new control socket for host veos01:22 as user admin`` host:port as user
+* ``control socket path is`` location on disk where the persistent connection socket is created
+* ``using connection plugin network_cli`` Informs you that persistent connection is being used
+* ``connection established to veos01 in 0:00:22.580626`` Time taken to obtain a shell on the remote device
+
+
+.. note: Port None ``creating new control socket for host veos01:None``
+
+   If the log reports the port as ``None`` this means that the default port is being used.
+   A future Ansible release will improve this message so that the port is always logged.
+
+Because the log files are verbose, you can use grep to look for specific information. For example, once you have identified the ```pid`` from the ``creating new control socket for host`` line you can search for other connection log entries::
+
+  grep "p=28990" $ANSIBLE_LOG_PATH
+
+Isolating an error
+------------------
+
+**Platforms:** Any
+
+As with any effort to troubleshoot it's important to simplify the test case as much as possible.
+
+For Ansible this can be done by ensuring you are only running against one remote device:
+
+* Using ``ansible-playbook --limit switch1.example.net...``
+* Using an ad-hoc ``ansible`` command
+
+`ad-hoc` refers to running Ansible to perform some quick command using ``/usr/bin/ansible``, rather than the orchestration language, which is ``/usr/bin/ansible-playbook``. In this case we can ensure connectivity by attempting to execute a single command on the remote device::
+
+  ansible -m eos_command -a 'commands=?' -i inventory switch1.example.net -e 'ansible_connection=local' -u admin -k
+
+In the above example, we:
+
+* connect to ``switch1.example.net`` specified in the inventory file ``inventory``
+* use the module ``eos_command``
+* run the command ``?``
+* connect using the username ``admin``
+* inform ansible to prompt for the ssh password by specifying ``-k``
+
+If you have SSH keys configured correctly, you don't need to specify the ``-k`` parameter
+
+If the connection still fails you can combine it with the enable_network_logging parameter. For example::
+
+   # Specify the location for the log file
+   export ANSIBLE_LOG_PATH=~/ansible.log
+   # Enable Debug
+   export ANSIBLE_DEBUG=True
+   # Run with 4*v for connection level verbosity
+   ansible -m eos_command -a 'commands=?' -i inventory switch1.example.net -e 'ansible_connection=local' -u admin -k
+
+Then review the log file and find the relevant error message in the rest of this document.
+
+.. For details on other ways to authenticate, see LINKTOAUTHHOWTODOCS.
+
+.. _unable_to_open_shell:
+
+Category "Unable to open shell"
+===============================
+
+
+**Platforms:** Any
+
+The ``unable to open shell`` message is new in Ansible 2.3. This message means that the ``ansible-connection`` daemon has not been able to successfully talk to the remote network device. This generally means that there is an authentication issue. It is a "catch all" message, meaning you need to enable ``ANSIBLE_LOG_PATH`` to find the underlying issues.
+
+
+
+For example:
+
+.. code-block:: none
+
+  TASK [prepare_eos_tests : enable cli on remote device] **************************************************
+  fatal: [veos01]: FAILED! => {"changed": false, "failed": true, "msg": "unable to open shell"}
+
+
+or:
+
+
+.. code-block:: none
+
+   TASK [ios_system : configure name_servers] *************************************************************
+   task path:
+   fatal: [ios-csr1000v]: FAILED! => {
+       "changed": false,
+       "failed": true,
+       "msg": "unable to open shell",
+       "rc": 255
+   }
+
+Suggestions to resolve:
+
+Follow the steps detailed in enable_network_logging_.
+
+Once you've identified the error message from the log file, the specific solution can be found in the rest of this document.
+
+
+
+Error: "[Errno -2] Name or service not known"
+---------------------------------------------
+
+**Platforms:** Any
+
+Indicates that the remote host you are trying to connect to can not be reached
+
+For example:
+
+.. code-block:: yaml
+
+   2017-04-04 11:39:48,147 p=15299 u=fred |  control socket path is /home/fred/.ansible/pc/ca5960d27a
+   2017-04-04 11:39:48,147 p=15299 u=fred |  current working directory is /home/fred/git/ansible-inc/stable-2.3/test/integration
+   2017-04-04 11:39:48,147 p=15299 u=fred |  using connection plugin network_cli
+   2017-04-04 11:39:48,340 p=15299 u=fred |  connecting to host veos01 returned an error
+   2017-04-04 11:39:48,340 p=15299 u=fred |  [Errno -2] Name or service not known
+
+
+Suggestions to resolve:
+
+* If you are using the ``provider:`` options ensure that it's suboption ``host:`` is set correctly.
+* If you are not using ``provider:`` nor top-level arguments ensure your inventory file is correct.
+
+
+
+
+
+Error: "Authentication failed"
+------------------------------
+
+**Platforms:** Any
+
+Occurs if the credentials (username, passwords, or ssh keys) passed to ``ansible-connection`` (via ``ansible`` or ``ansible-playbook``) can not be used to connect to the remote device.
+
+
+
+For example:
+
+.. code-block:: yaml
+
+   <ios01> ESTABLISH CONNECTION FOR USER: cisco on PORT 22 TO ios01
+   <ios01> Authentication failed.
+
+
+Suggestions to resolve:
+
+If you are specifying credentials via ``password:`` (either directly or via ``provider:``) or the environment variable ``ANSIBLE_NET_PASSWORD`` it is possible that ``paramiko`` (the Python SSH library that Ansible uses) is using ssh keys, and therefore the credentials you are specifying are being ignored. To find out if this is the case, disable "look for keys". This can be done like this:
+
+.. code-block:: yaml
+
+   export ANSIBLE_PARAMIKO_LOOK_FOR_KEYS=False
+
+To make this a permanent change, add the following to your ``ansible.cfg`` file:
+
+.. code-block:: ini
+
+   [paramiko_connection]
+   look_for_keys = False
+
+
+Error: "connecting to host <hostname> returned an error" or "Bad address"
+-------------------------------------------------------------------------
+
+This may occur if the SSH fingerprint hasn't been added to Paramiko's (the Python SSH library) know hosts file.
+
+When using persistent connections with Paramiko, the connection runs in a background process.  If the host doesn't already have a valid SSH key, by default Ansible will prompt to add the host key.  This will cause connections running in background processes to fail.
+
+For example:
+
+.. code-block:: yaml
+
+   2017-04-04 12:06:03,486 p=17981 u=fred |  using connection plugin network_cli
+   2017-04-04 12:06:04,680 p=17981 u=fred |  connecting to host veos01 returned an error
+   2017-04-04 12:06:04,682 p=17981 u=fred |  (14, 'Bad address')
+   2017-04-04 12:06:33,519 p=17981 u=fred |  number of connection attempts exceeded, unable to connect to control socket
+   2017-04-04 12:06:33,520 p=17981 u=fred |  persistent_connect_interval=1, persistent_connect_retries=30
+
+
+Suggestions to resolve:
+
+Use ``ssh-keyscan`` to pre-populate the known_hosts. You need to ensure the keys are correct.
+
+.. code-block:: shell
+
+   ssh-keyscan veos01
+
+
+or
+
+You can tell Ansible to automatically accept the keys
+
+Environment variable method::
+
+  export ANSIBLE_PARAMIKO_HOST_KEY_AUTO_ADD=True
+  ansible-playbook ...
+
+``ansible.cfg`` method:
+
+ansible.cfg
+
+.. code-block: ini
+
+  [paramiko_connection]
+  host_key_auto_add = True
+
+
+
+.. warning: Security warning
+
+   Care should be taken before accepting keys.
+
+Error: "No authentication methods available"
+--------------------------------------------
+
+For example:
+
+.. code-block:: yaml
+
+   2017-04-04 12:19:05,670 p=18591 u=fred |  creating new control socket for host veos01:None as user admin
+   2017-04-04 12:19:05,670 p=18591 u=fred |  control socket path is /home/fred/.ansible/pc/ca5960d27a
+   2017-04-04 12:19:05,670 p=18591 u=fred |  current working directory is /home/fred/git/ansible-inc/ansible-workspace-2/test/integration
+   2017-04-04 12:19:05,670 p=18591 u=fred |  using connection plugin network_cli
+   2017-04-04 12:19:06,606 p=18591 u=fred |  connecting to host veos01 returned an error
+   2017-04-04 12:19:06,606 p=18591 u=fred |  No authentication methods available
+   2017-04-04 12:19:35,708 p=18591 u=fred |  number of connection attempts exceeded, unable to connect to control socket
+   2017-04-04 12:19:35,709 p=18591 u=fred |  persistent_connect_interval=1, persistent_connect_retries=30
+
+
+Suggestions to resolve:
+
+No password or SSH key supplied
+
+Clearing Out Persistent Connections
+-----------------------------------
+
+**Platforms:** Any
+
+In Ansible 2.3, persistent connection sockets are stored in ``~/.ansible/pc`` for all network devices.  When an Ansible playbook runs, the persistent socket connection is displayed when verbose output is specified.
+
+``<switch> socket_path: /home/fred/.ansible/pc/f64ddfa760``
+
+To clear out a persistent connection before it times out (the default timeout is 30 seconds
+of inactivity), simple delete the socket file.
+
+
+
+Timeout issues
+==============
+
+Timeouts
+--------
+
+All network modules support a timeout value that can be set on a per task
+basis.  The timeout value controls the amount of time in seconds before the
+task will fail if the command has not returned.
+
+For example:
+
+.. FIXME: Detail error here
+
+
+Suggestions to resolve:
+
+.. code-block:: yaml
+
+    - name: save running-config
+      ios_command:
+        commands: copy running-config startup-config
+        provider: "{{ cli }}"
+        timeout: 30
+
+Some operations take longer than the default 10 seconds to complete.  One good
+example is saving the current running config on IOS devices to startup config.
+In this case, changing the timeout value form the default 10 seconds to 30
+seconds will prevent the task from failing before the command completes
+successfully.
+
+
+
+Playbook issues
+===============
+
+This section details issues are caused by issues with the Playbook itself.
+
+Error: "invalid connection specified, expected connection=local, got ssh"
+-------------------------------------------------------------------------
+
+**Platforms:** Any
+
+Network modules require that the connection is set to ``local``.  Any other
+connection setting will cause the playbook to fail.  Ansible will now detect
+this condition and return an error message:
+
+.. code-block:: yaml
+
+    fatal: [nxos01]: FAILED! => {
+        "changed": false,
+        "failed": true,
+        "msg": "invalid connection specified, expected connection=local, got ssh"
+    }
+
+
+To fix this issue, set the connection value to ``local`` using one of the
+following methods:
+
+* Set the play to use ``connection: local``
+* Set the task to use ``connection: local``
+* Run ansible-playbook using the ``-c local`` setting
+
+Error: "Unable to enter configuration mode"
+-------------------------------------------
+
+**Platforms:** eos and ios
+
+This occurs when you attempt to run a task that requires privileged mode in a user mode shell.
+
+For example:
+
+.. code-block:: yaml
+
+  TASK [ios_system : configure name_servers] *****************************************************************************
+  task path:
+  fatal: [ios-csr1000v]: FAILED! => {
+      "changed": false,
+      "failed": true,
+     "msg": "unable to enter configuration mode",
+      "rc": 255
+  }
+
+Suggestions to resolve:
+
+Add ``authorize: yes`` to the task. For example:
+
+.. code-block:: yaml
+
+  - name: configure hostname
+    ios_system:
+      provider:
+        hostname: foo
+        authorize: yes
+    register: result
+
+If the user requires a password to go into privileged mode, this can be specified with ``auth_pass``; if ``auth_pass`` isn't set, the environment variable ``ANSIBLE_NET_AUTHORIZE`` will be used instead.
+
+
+Add `authorize: yes` to the task. For example:
+
+.. code-block:: yaml
+
+  - name: configure hostname
+    ios_system:
+    provider:
+      hostname: foo
+      authorize: yes
+      auth_pass: "{{ mypasswordvar }}"
+  register: result
+
+
+.. delete_to not honoured
+   ----------------------
+
+   FIXME Do we get an error message
+
+   FIXME Link to howto
+
+
+
+
+   fixmes
+   ======
+
+   Error: "number of connection attempts exceeded, unable to connect to control socket"
+   ------------------------------------------------------------------------------------
+
+   **Platforms:** Any
+
+   This occurs when Ansible wasn't able to connect to the remote device and obtain a shell with the timeout.
+
+
+   This information is available when ``ANSIBLE_LOG_PATH`` is set see (FIXMELINKTOSECTION):
+
+   .. code-block:: yaml
+
+     less $ANSIBLE_LOG_PATH
+     2017-03-10 15:32:06,173 p=19677 u=fred |  number of connection attempts exceeded, unable to connect to control socket
+     2017-03-10 15:32:06,174 p=19677 u=fred |  persistent_connect_interval=1, persistent_connect_retries=10
+     2017-03-10 15:32:06,222 p=19669 u=fred |  fatal: [veos01]: FAILED! => {
+
+   Suggestions to resolve:
+
+   Do stuff For example:
+
+   .. code-block:: yaml
+
+   	Example stuff

docs/docsite/rst/playbooks_best_practices.rst

@@ -38,6 +38,7 @@ The top level of the directory would contain files and directories like so::
        hostname2              # ""
 
     library/                  # if any custom modules, put them here (optional)
+    module_utils/             # if any custom module_utils to support modules, put them here (optional)
     filter_plugins/           # if any custom filter plugins, put them here (optional)
 
     site.yml                  # master playbook
@@ -62,6 +63,7 @@ The top level of the directory would contain files and directories like so::
             meta/             #
                 main.yml      #  <-- role dependencies
             library/          # roles can also include custom modules
+            module_utils/     # roles can also include custom module_utils
             lookup_plugins/   # or other types of plugins, like lookup in this case
 
         webtier/              # same kind of structure as "common" was above, done for the webtier role
@@ -97,6 +99,7 @@ Alternatively you can put each inventory file with its ``group_vars``/``host_var
              stagehost2       # ""
 
     library/
+    module_utils/
     filter_plugins/
 
     site.yml
@@ -422,7 +425,7 @@ This will pull in variables based on the OS name.
 Bundling Ansible Modules With Playbooks
 +++++++++++++++++++++++++++++++++++++++
 
-If a playbook has a "./library" directory relative to its YAML file, this directory can be used to add ansible modules that will
+If a playbook has a :file:`./library` directory relative to its YAML file, this directory can be used to add ansible modules that will
 automatically be in the ansible module path.  This is a great way to keep modules that go with a playbook together.  This is shown
 in the directory structure example at the start of this section.
 

docs/docsite/rst/playbooks_blocks.rst

@@ -9,20 +9,18 @@ by the tasks enclosed by a block. i.e. a `when` will be applied to the tasks, no
 
 
 .. code-block:: YAML
- :emphasize-lines: 2
+ :emphasize-lines: 3
  :caption: Block example
 
     tasks:
-      - block:
+      - name: Install Apache
+        block:
           - yum: name={{ item }} state=installed
             with_items:
               - httpd
               - memcached
-
           - template: src=templates/src.j2 dest=/etc/foo.conf
-
           - service: name=bar state=started enabled=True
-
         when: ansible_distribution == 'CentOS'
         become: true
         become_user: root
@@ -32,6 +30,9 @@ In the example above, each of the 3 tasks will be executed after appending the `
 and evaluating it in the task's context. Also they inherit the privilege escalation directives enabling "become to root"
 for all the enclosed tasks.
 
+. versionadded:: 2.3
+
+The `name:` keyword for `blocks:` was added in Ansible 2.3.
 
 .. _block_error_handling:
 
@@ -41,20 +42,21 @@ Error Handling
 Blocks also introduce the ability to handle errors in a way similar to exceptions in most programming languages.
 
 .. code-block:: YAML
- :emphasize-lines: 2,6,10
+ :emphasize-lines: 3,7,11
  :caption: Block error handling example
 
-  tasks:
-   - block:
-       - debug: msg='i execute normally'
-       - command: /bin/false
-       - debug: msg='i never execute, cause ERROR!'
-     rescue:
-       - debug: msg='I caught an error'
-       - command: /bin/false
-       - debug: msg='I also never execute :-('
-     always:
-       - debug: msg="this always executes"
+   tasks:
+    - name: Attempt and gracefull roll back demo
+      block:
+        - debug: msg='I execute normally'
+        - command: /bin/false
+        - debug: msg='I never execute, due to the above task failing'
+      rescue:
+        - debug: msg='I caught an error'
+        - command: /bin/false
+        - debug: msg='I also never execute :-('
+      always:
+        - debug: msg="this always executes"
 
 
 The tasks in the ``block`` would execute normally, if there is any error the ``rescue`` section would get executed
@@ -65,20 +67,21 @@ error did or did not occur in the ``block`` and ``rescue`` sections.
 Another example is how to run handlers after an error occurred :
 
 .. code-block:: YAML
- :emphasize-lines: 4,8
+ :emphasize-lines: 5,9
  :caption: Block run handlers in error handling
 
   tasks:
-   - block:
-       - debug: msg='i execute normally'
-         notify: run me even after an error
-       - command: /bin/false
-     rescue:
-       - name: make sure all handlers run
-         meta: flush_handlers
-  handlers:
-    - name: run me even after an error
-      debug: msg='this handler runs even on error'
+    - name: Attempt and gracefull roll back demo
+      block:
+        - debug: msg='I execute normally'
+          notify: run me even after an error
+        - command: /bin/false
+      rescue:
+        - name: make sure all handlers run
+          meta: flush_handlers
+   handlers:
+     - name: run me even after an error
+       debug: msg='this handler runs even on error'
 
 .. seealso::
 
@@ -93,4 +96,3 @@ Another example is how to run handlers after an error occurred :
 
 
 
-

docs/docsite/rst/playbooks_delegation.rst

@@ -182,7 +182,7 @@ In 2.0, the directive `delegate_facts` may be set to `True` to assign the task's
           with_items: "{{groups['dbservers']}}"
 
 The above will gather facts for the machines in the dbservers group and assign the facts to those machines and not to app_servers.
-This way you can lookup `hostvars['dbhost1']['default_ipv4_addresses'][0]` even though dbservers were not part of the play, or left out by using `--limit`.
+This way you can lookup `hostvars['dbhost1']['default_ipv4']['address']` even though dbservers were not part of the play, or left out by using `--limit`.
 
 
 .. _run_once:

docs/docsite/rst/playbooks_filters.rst

@@ -588,6 +588,9 @@ To replace text in a string with regex, use the "regex_replace" filter::
 
     # convert "localhost:80" to "localhost, 80" using named groups
     {{ 'localhost:80' | regex_replace('^(?P<host>.+):(?P<port>\\d+)$', '\\g<host>, \\g<port>') }}
+    
+    # convert "localhost:80" to "localhost"
+    {{ 'localhost:80' | regex_replace(':80') }}
 
 .. note:: Prior to ansible 2.0, if "regex_replace" filter was used with variables inside YAML arguments (as opposed to simpler 'key=value' arguments),
    then you needed to escape backreferences (e.g. ``\\1``) with 4 backslashes (``\\\\``) instead of 2 (``\\``).

docs/docsite/rst/playbooks_lookups.rst

@@ -116,27 +116,27 @@ Examples
 --------
 Basic lookup. Fails if example/test doesn't exist::
 
-    password="{{ lookup('pass', 'example/test')}}`
+    password="{{ lookup('passwordstore', 'example/test')}}`
 
 Create pass with random 16 character password. If password exists just give the password::
 
-    password="{{ lookup('pass', 'example/test create=true')}}`
+    password="{{ lookup('passwordstore', 'example/test create=true')}}`
 
 Different size password::
 
-    password="{{ lookup('pass', 'example/test create=true length=42')}}`
+    password="{{ lookup('passwordstore', 'example/test create=true length=42')}}`
 
 Create password and overwrite the password if it exists. As a bonus, this module includes the old password inside the pass file::
 
-    password="{{ lookup('pass', 'example/test create=true overwrite=true')}}`
+    password="{{ lookup('passwordstore', 'example/test create=true overwrite=true')}}`
 
 Return the value for user in the KV pair user: username::
 
-    password="{{ lookup('pass', 'example/test subkey=user')}}`
+    password="{{ lookup('passwordstore', 'example/test subkey=user')}}`
 
 Return the entire password file content::
 
-    password="{{ lookup('pass', 'example/test returnall=true')}}`
+    password="{{ lookup('passwordstore', 'example/test returnall=true')}}`
 
 The location of the password-store directory can be specified in the following ways:
   - Default is ~/.password-store

docs/docsite/rst/playbooks_loops.rst

@@ -112,8 +112,7 @@ And you want to print every user's name and phone number.  You can loop through
       - name: Print phone records
         debug:
           msg: "User {{ item.key }} is {{ item.value.name }} ({{ item.value.telephone }})"
-        with_dict:
-          - "{{ users }}"
+        with_dict: "{{ users }}"
 
 .. _looping_over_fileglobs:
 

docs/docsite/rst/playbooks_vault.rst

@@ -136,7 +136,26 @@ As of version 2.3, Ansible can now use a vaulted variable that lives in an other
     other_plain_text: othervalue
 
 
-This vaulted variable be decrypted with the supplied vault secret and used as a normal variable. The `ansible-vault` command line supports stdin and stdout for encrypting data on the fly, which can be used from your favorite editor to create these vaulted variables; you just have to be sure to add the `!vault` tag so both Ansible and YAML are aware of the need to decrypt. The `|` is also required, as vault encryption results in a multi-line string.
+This vaulted variable be decrypted with the supplied vault secret and used as a normal variable. The `ansible-vault` command line supports `STDIN` and `STDOUT` for encrypting data on the fly, which can be used from your favorite editor to create these vaulted variables; you just have to be sure to add the `!vault` tag so both Ansible and YAML are aware of the need to decrypt. The `|` is also required, as vault encryption results in a multi-line string. The leading spaces will be ignored and some indentation is required for it to be valid YAML.
+
+As of version 2.3, one way to generate the inline secret is to use `ansible-vault encrypt_string` which will output the secret to `STDOUT`::
+
+   $ ansible-vault encrypt_string "42"
+   !vault-encrypted |
+          $ANSIBLE_VAULT;1.1;AES256
+          <vault cipher text here>
+
+   $ ansible-vault encrypt_string "42" --stdin-name "the_answer"
+    the_answer: !vault-encrypted |
+          $ANSIBLE_VAULT;1.1;AES256
+          <vault cipher text here>
+
+   $ echo -n "the plaintext to encrypt" | ansible-vault encrypt_string
+   !vault-encrypted |
+          $ANSIBLE_VAULT;1.1;AES256
+          <vault cipher text here>
+
+Note the use of `echo -n`. If you use just `echo` the encrypted string will have a new line (`\n`) on the end.
 
 
 .. _speeding_up_vault:

docs/docsite/rst/porting_guide_2.0.rst

@@ -1,14 +1,26 @@
-Porting Guide
-=============
+.. _porting_2.0_guide:
 
+*************************
+Ansible 2.0 Porting Guide
+*************************
+
+This section discusses the behavioral changes between Ansible 1.x and Ansible 2.0.
+
+It is intended to assist in updating your playbooks, plugins and other parts of your Ansible infrastructure so they will work with this version of Ansible.
+
+
+We suggest you read this page along with `Ansible Changelog <https://github.com/ansible/ansible/blob/devel/CHANGELOG.md#2.0>`_ to understand what updates you may need to make.
+
+This document is part of a collection on porting. The complete list of porting guides can be found at :ref:`porting guides <porting_guides>`.
+
+.. contents:: Topics
 
 Playbook
---------
+========
 
-* backslash escapes When specifying parameters in jinja2 expressions in YAML
-  dicts, backslashes sometimes needed to be escaped twice. This has been fixed
-  in 2.0.x so that escaping once works. The following example shows how
-  playbooks must be modified::
+This section discusses any changes you may need to make to your playbooks.
+
+.. code-block:: yaml
 
     # Syntax in 1.9.x
     - debug:
@@ -88,16 +100,16 @@ uses key=value escaping which has not changed.  The other option is to check for
 * dnf module has been rewritten. Some minor changes in behavior may be observed.
 * win_updates has been rewritten and works as expected now.
 * from 2.0.1 onwards, the implicit setup task from gather_facts now correctly inherits everything from play, but this might cause issues for those setting
-  `environment` at the play level and depending on `ansible_env` existing. Previouslly this was ignored but now might issue an 'Undefined' error.
+  `environment` at the play level and depending on `ansible_env` existing. Previously this was ignored but now might issue an 'Undefined' error.
 
 Deprecated
-~~~~~~~~~~
+----------
 
 While all items listed here will show a deprecation warning message, they still work as they did in 1.9.x. Please note that they will be removed in 2.2 (Ansible always waits two major releases to remove a deprecated feature).
 
-* Bare variables in `with_` loops should instead use the “{{var}}” syntax, which helps eliminate ambiguity.
+* Bare variables in ``with_`` loops should instead use the ``"{ {var }}"`` syntax, which helps eliminate ambiguity.
 * The ansible-galaxy text format requirements file. Users should use the YAML format for requirements instead.
-* Undefined variables within a `with_` loop’s list currently do not interrupt the loop, but they do issue a warning; in the future, they will issue an error.
+* Undefined variables within a ``with_`` loop’s list currently do not interrupt the loop, but they do issue a warning; in the future, they will issue an error.
 * Using dictionary variables to set all task parameters is unsafe and will be removed in a future version. For example::
 
     - hosts: localhost
@@ -151,9 +163,9 @@ Should now be::
 
 
 Other caveats
-~~~~~~~~~~~~~
+-------------
 
-Here are some corner cases encountered when updating, these are mostly caused by the more stringent parser validation and the capture of errors that were previouslly ignored.
+Here are some corner cases encountered when updating. These are mostly caused by the more stringent parser validation and the capture of errors that were previously ignored.
 
 * Bad variable composition::
 
@@ -201,32 +213,33 @@ Here are some corner cases encountered when updating, these are mostly caused by
 
     with_items: "{{var1 + var2}}"
 
-  The bare feature itself is deprecated as an undefined variable is indistiguishable from a string which makes it difficult to display a proper error.
+  The bare feature itself is deprecated as an undefined variable is indistinguishable from a string which makes it difficult to display a proper error.
 
 Porting plugins
----------------
+===============
 
 In ansible-1.9.x, you would generally copy an existing plugin to create a new one. Simply implementing the methods and attributes that the caller of the plugin expected made it a plugin of that type. In ansible-2.0, most plugins are implemented by subclassing a base class for each plugin type. This way the custom plugin does not need to contain methods which are not customized.
 
 
 Lookup plugins
-~~~~~~~~~~~~~~
+--------------
+
 * lookup plugins ; import version
 
 
 Connection plugins
-~~~~~~~~~~~~~~~~~~
+------------------
 
 * connection plugins
 
 Action plugins
-~~~~~~~~~~~~~~
+--------------
 
 
 * action plugins
 
 Callback plugins
-~~~~~~~~~~~~~~~~
+----------------
 
 Although Ansible 2.0 provides a new callback API the old one continues to work
 for most callback plugins.  However, if your callback plugin makes use of
@@ -262,15 +275,15 @@ populates the callback with them.  Here's a short snippet that shows you how:
 
 
 Connection plugins
-~~~~~~~~~~~~~~~~~~
+------------------
 
 * connection plugins
 
 
 Hybrid plugins
---------------
+==============
 
-In specific cases you may want a plugin that supports both ansible-1.9.x *and* ansible-2.0. Much like porting plugins from v1 to v2, you need to understand how plugins work in each version and support both requirements. It may mean playing tricks on Ansible.
+In specific cases you may want a plugin that supports both ansible-1.9.x *and* ansible-2.0. Much like porting plugins from v1 to v2, you need to understand how plugins work in each version and support both requirements.
 
 Since the ansible-2.0 plugin system is more advanced, it is easier to adapt your plugin to provide similar pieces (subclasses, methods) for ansible-1.9.x as ansible-2.0 expects. This way your code will look a lot cleaner.
 
@@ -290,8 +303,9 @@ You may find the following tips useful:
 
 
 Lookup plugins
-~~~~~~~~~~~~~~
-As a simple example we are going to make a hybrid ``fileglob`` lookup plugin.  The ``fileglob`` lookup plugin is pretty simple to understand
+--------------
+
+As a simple example we are going to make a hybrid ``fileglob`` lookup plugin.
 
 .. code-block:: python
 
@@ -357,28 +371,28 @@ As a simple example we are going to make a hybrid ``fileglob`` lookup plugin.  T
 
 
 Connection plugins
-~~~~~~~~~~~~~~~~~~
+------------------
 
 * connection plugins
 
 Action plugins
-~~~~~~~~~~~~~~
+--------------
 
 * action plugins
 
 Callback plugins
-~~~~~~~~~~~~~~~~
+----------------
 
 * callback plugins
 
 Connection plugins
-~~~~~~~~~~~~~~~~~~
+------------------
 
 * connection plugins
 
 
 Porting custom scripts
-----------------------
+======================
 
-Custom scripts that used the ``ansible.runner.Runner`` API in 1.x have to be ported in 2.x.  Please refer to: :doc:`dev_guide//developing_api`
+Custom scripts that used the ``ansible.runner.Runner`` API in 1.x have to be ported in 2.x.  Please refer to: :doc:`dev_guide/developing_api`
 

docs/docsite/rst/porting_guide_2.3.rst

@@ -0,0 +1,246 @@
+.. _porting_2.3_guide:
+
+*************************
+Ansible 2.3 Porting Guide
+*************************
+
+This section discusses the behavioral changes between Ansible 2.2 and Ansible 2.3.
+
+It is intended to assist in updating your playbooks, plugins and other parts of your Ansible infrastructure so they will work with this version of Ansible.
+
+
+We suggest you read this page along with `Ansible Changelog <https://github.com/ansible/ansible/blob/devel/CHANGELOG.md#2.3>`_ to understand what updates you may need to make.
+
+This document is part of a collection on porting. The complete list of porting guides can be found at :ref:`porting guides <porting_guides>`.
+
+.. contents:: Topics
+
+Playbook
+========
+
+Restructued async to work with action plugins
+---------------------------------------------
+
+In Ansible 2.2 (and possibly earlier) the `async:` keyword could not be used in conjunction with the action plugins such as `service`. This limitation has been removed in Ansible 2.3
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+    - name: Install nginx asynchronously
+      service:
+        name: nginx
+        state: restarted
+      async: 45
+
+
+OpenBSD version facts
+---------------------
+
+The `ansible_distribution_release` and `ansible_distribution_version` facts on OpenBSD hosts were reversed in Ansible 2.2 and earlier. This has been changed so that version has the numeric portion and release has the name of the release.
+
+**OLD** In Ansible 2.2 (and earlier)
+
+
+.. code-block:: yaml
+
+    "ansible_distribution": "OpenBSD"
+    "ansible_distribution_release": "6.0",
+    "ansible_distribution_version": "release",
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+    "ansible_distribution": "OpenBSD",
+    "ansible_distribution_release": "release",
+    "ansible_distribution_version": "6.0",
+
+
+Names Blocks
+------------
+
+Blocks can now have names, this allows you to avoid the ugly `# this block is for...` comments.
+
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+    - name: Block test case
+      hosts: localhost
+      tasks:
+       - name: Attempt to setup foo
+         block:
+           - debug: msg='I execute normally'
+           - command: /bin/false
+           - debug: msg='I never execute, cause ERROR!'
+         rescue:
+           - debug: msg='I caught an error'
+           - command: /bin/false
+           - debug: msg='I also never execute :-('
+         always:
+           - debug: msg="this always executes"
+
+
+Use of multiple tags
+--------------------
+
+Specifying ``--tags`` (or ``--skip-tags``) multiple times on the command line currently leads to the last specified tag overriding all the other specified tags. This behaviour is deprecated. In the future, if you specify --tags multiple times the tags will be merged together. From now on, using ``--tags`` multiple times on one command line will emit a deprecation warning. Setting the ``merge_multiple_cli_tags`` option to True in the ``ansible.cfg`` file will enable the new behaviour.
+
+In 2.4, the default will be to merge the tags. You can enable the old overwriting behavior via the config option.
+In 2.5, multiple ``--tags`` options will be merged with no way to go back to the old behaviour.
+
+
+Other caveats
+-------------
+
+Here are some rare cases that might be encountered when updating. These are mostly caused by the more stringent parser validation and the capture of errors that were previously ignored.
+
+
+* Made ``any_errors_fatal`` inheritable from play to task and all other objects in between.
+
+Modules
+=======
+
+No major changes in this version.
+
+Modules removed
+---------------
+
+No major changes in this version.
+
+Deprecation notices
+-------------------
+
+The following modules will be removed in Ansible 2.5. Please update your playbooks accordingly.
+
+* :ref:`ec2_vpc <ec2_vpc>`
+* :ref:`cl_bond <cl_bond>`
+* :ref:`cl_bridge <cl_bridge>`
+* :ref:`cl_img_install <cl_img_install>`
+* :ref:`cl_interface <cl_interface>`
+* :ref:`cl_interface_policy <cl_interface_policy>`
+* :ref:`cl_license <cl_license>`
+* :ref:`cl_ports <cl_ports>`
+* :ref:`nxos_mtu <nxos_mtu>` use :ref:`nxos_system <nxos_system>` instead
+
+Noteworthy module changes
+-------------------------
+
+AWS lambda
+^^^^^^^^^^
+Previously ignored changes that only affected one parameter. Existing deployments may have outstanding changes that this bug fix will apply.
+
+
+Mount
+^^^^^
+
+Mount: Some fixes so bind mounts are not mounted each time the playbook runs.
+
+
+Plugins
+=======
+
+No major changes in this version.
+
+Porting custom scripts
+======================
+
+No major changes in this version.
+
+Networking
+==========
+
+There have been a number of changes to number of changes to how Networking Modules operate.
+
+Playbooks should still use ``connection: local``.
+
+The following changes apply to:
+
+* dellos6
+* dellos9
+* dellos10
+* eos
+* ios
+* iosxr
+* junos
+* sros
+* vyos
+
+Deprecation of top-level connection arguments
+---------------------------------------------
+
+**OLD** In Ansible 2.2:
+
+.. code-block:: yaml
+
+    - name: example of using top-level options for connection properties
+      ios_command:
+        commands: show version
+        host: "{{ inventory_hostname }}"
+        username: cisco
+        password: cisco
+        authorize: yes
+        auth_pass: cisco
+
+Will result in:
+
+.. code-block:: yaml
+
+   [WARNING]: argument username has been deprecated and will be removed in a future version
+   [WARNING]: argument host has been deprecated and will be removed in a future version
+   [WARNING]: argument password has been deprecated and will be removed in a future version
+
+
+**NEW** In Ansible 2.3:
+
+
+.. code-block:: yaml
+
+   - name: Gather facts
+     eos_facts:
+       gather_subset: all
+       provider:
+         username: myuser
+         password: "{{ networkpassword }}"
+         transport: cli
+         host: "{{ ansible_host }}"
+
+delegate_to vs ProxyCommand
+---------------------------
+
+The new connection framework for Network Modules in Ansible 2.3 no longer supports the use of the
+``delegate_to`` directive.  In order to use a bastion or intermediate jump host
+to connect to network devices, network modules now support the use of
+``ProxyCommand``.
+
+To use ``ProxyCommand`` configure the proxy settings in the Ansible inventory
+file to specify the proxy host.
+
+.. code-block:: ini
+
+    [nxos]
+    nxos01
+    nxos02
+
+    [nxos:vars]
+    ansible_ssh_common_args='-o ProxyCommand="ssh -W %h:%p -q bastion01"'
+
+
+With the configuration above, simply build and run the playbook as normal with
+no additional changes necessary.  The network module will now connect to the
+network device by first connecting to the host specified in
+``ansible_ssh_common_args`` which is ``bastion01`` in the above example.
+
+
+.. notes: Using ``ProxyCommand`` with passwords via variables
+
+   It is a feature that SSH doesn't support providing passwords via environment variables.
+   This is done to prevent secrets from leaking out, for example in ``ps`` output.
+
+   We recommend using SSH Keys, and if needed and ssh-agent, rather than passwords, where ever possible.
+

docs/docsite/rst/porting_guides.rst

@@ -0,0 +1,12 @@
+.. _porting_guides:
+
+**********************
+Ansible Porting Guides
+**********************
+
+This section lists porting guides that can help you playbooks, plugins and other parts of your Ansible infrastructure from one version of Ansible to the next. Please note that this is not a complete list. If you believe any extra information would be useful in these pages, you can edit by clicking `Edit on GitHub` on the top right, or raising an issue.
+
+
+* :ref:`Ansible 1.x to 2.0 porting guide <porting_2.0_guide>`
+* :ref:`Ansible 2.2 to 2.3 porting guide <porting_2.3_guide>`
+* :ref:`Ansible 2.3 to 2.4 porting guide <porting_2.4_guide>`

docs/docsite/rst/release_and_maintenance.rst

@@ -0,0 +1,137 @@
+Release and maintenance
+=======================
+
+.. contents:: Topics
+   :local:
+
+.. _schedule:
+
+Release cycle
+`````````````
+
+Ansible is developed and released on a flexible 4 months release cycle.
+This cycle can be extended in order to allow for larger changes to be properly
+implemented and tested before a new release is made available.
+
+Ansible supports the two most recent major stable releases.
+For more information, read about the
+`development and stable version maintenance workflow`_.
+
+If you are using a release of Ansible that is no longer supported, we strongly
+encourage you to upgrade as soon as possible in order to benefit from the
+latest features and security fixes.
+
+Older unsupported versions of Ansible can contain unfixed security
+vulnerabilities (*CVE*).
+
+You can refer to the `porting guide`_ for tips on updating your Ansible
+playbooks to run on newer versions.
+
+.. _porting guide: https://docs.ansible.com/ansible/porting_guide_2.0.html
+
+Release status
+``````````````
+
++-----------------+----------------------------+----------------------------------------+
+| Ansible release | Latest version             | Status                                 |
++=================+============================+========================================+
+| devel           | `2.4`_ (unreleased, trunk) | In development                         |
++-----------------+----------------------------+----------------------------------------+
+| 2.3             | `2.3.1`_ (2017-06-01)      | Supported (bug **and** security fixes) |
++-----------------+----------------------------+----------------------------------------+
+| 2.2             | `2.2.3`_ (2017-05-09)      | Supported (**only** security fixes)    |
++-----------------+----------------------------+----------------------------------------+
+| 2.1             | `2.1.6`_ (2017-06-01)      | Unsupported (end of life)              |
++-----------------+----------------------------+----------------------------------------+
+| 2.0             | `2.0.2`_ (2016-04-19)      | Unsupported (end of life)              |
++-----------------+----------------------------+----------------------------------------+
+| 1.9             | `1.9.6`_ (2016-04-15)      | Unsupported (end of life)              |
++-----------------+----------------------------+----------------------------------------+
+| <1.9            | n/a                        | Unsupported (end of life)              |
++-----------------+----------------------------+----------------------------------------+
+
+.. _2.4: https://github.com/ansible/ansible/blob/devel/CHANGELOG.md
+.. _2.3.1: https://github.com/ansible/ansible/blob/stable-2.3/CHANGELOG.md
+.. _2.2.3: https://github.com/ansible/ansible/blob/stable-2.2/CHANGELOG.md
+.. _2.1.6: https://github.com/ansible/ansible/blob/stable-2.1/CHANGELOG.md
+.. _2.0.2: https://github.com/ansible/ansible/blob/stable-2.0/CHANGELOG.md
+.. _1.9.6: https://github.com/ansible/ansible/blob/stable-1.9/CHANGELOG.md
+
+.. _support_life:
+.. _methods:
+
+Development and stable version maintenance workflow
+```````````````````````````````````````````````````
+
+The Ansible community develops and maintains Ansible on GitHub_.
+
+New modules, plugins, features and bugfixes will always be integrated in what
+will become the next major version of Ansible.
+This work is tracked on the ``devel`` git branch.
+
+Ansible provides bugfixes and security improvements for the most recent major
+release while the previous major release will only receive security patches.
+This work is tracked on the ``stable-<version>`` git branches.
+
+The fixes that land in supported stable branches will eventually be released
+as a new version when necessary.
+
+For more information on the changes included in each new version, you can refer
+to the changelog_, available on GitHub.
+
+Note that while there are no guarantees for providing fixes for unsupported
+releases of Ansible, there can sometimes be exceptions for critical issues.
+
+.. _GitHub: https://github.com/ansible/ansible
+.. _changelog: https://github.com/ansible/ansible/blob/devel/CHANGELOG.md
+
+Release candidates
+~~~~~~~~~~~~~~~~~~
+
+Before a new release or version of Ansible can be done, it will typically go
+through a release candidate process.
+
+This provides the Ansible community the opportunity to test Ansible and report
+bugs or issues they might come across.
+
+Ansible tags the first release candidate (``RC1``) which is usually scheduled
+to last five business days. The final release is done if no major bugs or
+issues are identified during this period.
+
+If there are major problems with the first candidate, a second candidate will
+be tagged (``RC2``) once the necessary fixes have landed.
+This second candidate lasts for a shorter duration than the first.
+If no problems have been reported after two business days, the final release is
+done.
+
+More release candidates can be tagged as required, so long as there are
+bugs that the Ansible core maintainers consider should be fixed before the
+final release.
+
+.. _freezing:
+
+Feature freeze
+~~~~~~~~~~~~~~
+
+While there is a pending release candidate, the focus of core developers and
+maintainers will on fixes towards the release candidate.
+
+Merging new features or fixes that are not related to the release candidate may
+be delayed in order to allow the new release to be shipped as soon as possible.
+
+.. seealso::
+
+   :doc:`committer_guidelines`
+       Guidelines for Ansible core contributors and maintainers
+   :doc:`test_strategies`
+       Testing strategies
+   :doc:`community`
+       Community information and contributing
+   `Ansible Changelog <https://github.com/ansible/ansible/blob/devel/CHANGELOG.md>`_
+       Documentation of the improvements for each version of Ansible
+   `Ansible release tarballs <https://releases.ansible.com/ansible/>`_
+       Ansible release tarballs
+   `Development Mailing List <http://groups.google.com/group/ansible-devel>`_
+       Mailing list for development topics
+   `irc.freenode.net <http://irc.freenode.net>`_
+       #ansible IRC chat channel

docs/docsite/rst/roadmap/ROADMAP_2_2.rst

@@ -91,7 +91,7 @@ Target: September 2016
   - **Deferred:** (stretch goal) Bird modules
   - **Deferred:** (stretch goal) GoBGP modules
 
-- **Implement ‘role revamp’ proposal to give users more control on role/task execution (Brian) **
+- **Implement ‘role revamp’ proposal to give users more control on role/task execution (Brian)**
 
   - **https://github.com/ansible/proposals/blob/master/roles_revamp.md**
 

docs/docsite/rst/roadmap/ROADMAP_2_3.rst

@@ -39,11 +39,11 @@ Target: Mid April 2017
 - **Windows platform** (nitzmahone)
 
   - Pipelining support **(done)**
-  - Become support **(done)**
+  - Become support **(done/experimental)**
   - Integrated kerberos ticket management (via ansible_user/ansible_password) **(done)**
   - Switch PS input encoding to BOM-less UTF8 **(done)**
-  - Server 2016 support/testing (now RTM’d) **(in progress)**
-  - Modularize Windows module_utils (allow N files) **(in progress)**
+  - Server 2016 support/testing (now RTM’d) **(partial)**
+  - Modularize Windows module_utils (allow N files) **(partial)**
   - Declarative argspec for PS / .NET **(bumped to 2.4)**
   - Kerberos encryption (via notting, pywinrm/requests_kerberos/pykerberos) **(in progress, available in pywinrm post 2.3 release)**
   - Fix plugin-specific connection var lookup/delegation (either registered explicitly by plugins or ansible_(plugin)_*) **(bumped to 2.4)**
@@ -132,7 +132,7 @@ Target: Mid April 2017
 - **Amazon resources** (ryansb)
 
   - Improve ec2.py integration tests **(partial, more to do in 2.4)**
-  - ELB version 2 **(pushed - needs_revision [PR](https://github.com/ansible/ansible/pull/19491))**
+  - ELB version 2 **(pushed - needs_revision)** `PR <https://github.com/ansible/ansible/pull/19491>`_
   - CloudFormation YAML, cross-stack reference, and roles support **(done)**
   - ECS module refactor **(done)**
   - AWS module unit testing w/ placebo (boto3 only) **(pushed 2.4)**

docs/man/man1/ansible-vault.1.asciidoc.in

@@ -12,7 +12,7 @@ ansible-vault - manage encrypted ansible vars files (YAML).
 
 SYNOPSIS
 --------
-ansible-vault [create|decrypt|edit|encrypt|rekey] [--help] [options] file_name
+ansible-vault [create|decrypt|edit|encrypt|encrypt_string|rekey] [--help] [options] file_name
 
 
 DESCRIPTION
@@ -116,6 +116,24 @@ Thus any of the following invocations can be used:
 Reading from stdin and writing only encrypted output is a good way to prevent
 sensitive data from ever hitting disk (either interactively or from a script).
 
+
+ENCRYPT_STRING
+--------------
+
+*$ ansible-vault encrypt_string [options] "string cheese"*
+
+The *encrypt_string* sub-command is used to encrypt a string supplied as an argument.
+
+The *encrypt_string* command works with *STDIN* and has the additional options
+*--name* and *--stdin-name*:
+
+*$ ansible-vault encrypt_string "munster" -name="cheese_du_jour"*
+*$ ansible-vault encrypt_string "$@" -stdin-name="cheese_list" < ./cheese_list.txt*
+
+The output will be on *STDOUT*. This command is useful for creating inline
+secrets introduced in version 2.3.
+
+
 DECRYPT
 -------
 

examples/ansible.cfg

@@ -181,6 +181,7 @@
 #vars_plugins       = /usr/share/ansible/plugins/vars
 #filter_plugins     = /usr/share/ansible/plugins/filter
 #test_plugins       = /usr/share/ansible/plugins/test
+#terminal_plugins   = /usr/share/ansible/plugins/terminal
 #strategy_plugins   = /usr/share/ansible/plugins/strategy
 
 
@@ -280,7 +281,7 @@
 # Controls showing custom stats at the end, off by default
 #show_custom_stats = True
 
-# Controlls which files to ignore when using a directory as inventory with
+# Controls which files to ignore when using a directory as inventory with
 # possibly multiple sources (both static and dynamic)
 #inventory_ignore_extensions = ~, .orig, .bak, .ini, .cfg, .retry, .pyc, .pyo
 
@@ -288,6 +289,12 @@
 # only update this setting if you know how this works, otherwise it can break module execution
 #network_group_modules=['eos', 'nxos', 'ios', 'iosxr', 'junos', 'vyos']
 
+# When enabled, this option allows lookups (via variables like {{lookup('foo')}} or when used as
+# a loop with `with_foo`) to return data that is not marked "unsafe". This means the data may contain
+# jinja2 templating language which will be run through the templating engine.
+# ENABLING THIS COULD BE A SECURITY RISK
+#allow_unsafe_lookups = False
+
 [privilege_escalation]
 #become=True
 #become_method=sudo
@@ -305,6 +312,19 @@
 # line to disable this behaviour.
 #pty=False
 
+# paramiko will default to looking for SSH keys initially when trying to
+# authenticate to remote devices.  This is a problem for some network devices
+# that close the connection after a key failure.  Uncomment this line to
+# disable the Paramiko look for keys function
+#look_for_keys = False
+
+# When using persistent connections with Paramiko, the connection runs in a
+# background process.  If the host doesn't already have a valid SSH key, by
+# default Ansible will prompt to add the host key.  This will cause connections
+# running in background processes to fail.  Uncomment this line to have
+# Paramiko automatically add host keys.
+#host_key_auto_add = True
+
 [ssh_connection]
 
 # ssh arguments to use
@@ -357,6 +377,26 @@
 # only be disabled if your sftp version has problems with batch mode
 #sftp_batch_mode = False
 
+[persistent_connection]
+
+# Configures the persistent connection timeout value in seconds.  This value is
+# how long the persistent connection will remain idle before it is destroyed.  
+# If the connection doesn't receive a request before the timeout value 
+# expires, the connection is shutdown.  The default value is 30 seconds.
+connect_timeout = 30
+
+# Configures the persistent connection retries.  This value configures the
+# number of attempts the ansible-connection will make when trying to connect
+# to the local domain socket.  The default value is 30.
+connect_retries = 30
+
+# Configures the amount of time in seconds to wait between connection attempts 
+# to the local unix domain socket.  This value works in conjunction with the
+# connect_retries value to define how long to try to connect to the local
+# domain socket when setting up a persistent connection.  The default value is
+# 1 second.
+connect_interval = 1
+
 [accelerate]
 #accelerate_port = 5099
 #accelerate_timeout = 30

examples/scripts/ConfigureRemotingForAnsible.ps1

@@ -50,7 +50,8 @@ Param (
     [switch]$SkipNetworkProfileCheck,
     $CreateSelfSignedCert = $true,
     [switch]$ForceNewSSLCert,
-    [switch]$EnableCredSSP
+    [switch]$EnableCredSSP,
+    [switch]$GlobalHttpFirewallAccess
 )
 
 Function Write-Log
@@ -69,7 +70,7 @@ Function Write-VerboseLog
 Function Write-HostLog
 {
     $Message = $args[0]
-    Write-Host $Message
+    Write-Output $Message
     Write-Log $Message
 }
 
@@ -119,6 +120,60 @@ Function New-LegacySelfSignedCert
     return $parsed_cert.Thumbprint
 }
 
+Function Enable-GlobalHttpFirewallAccess
+{
+    Write-Verbose "Forcing global HTTP firewall access"
+    # this is a fairly naive implementation; could be more sophisticated about rule matching/collapsing
+    $fw = New-Object -ComObject HNetCfg.FWPolicy2
+
+    # try to find/enable the default rule first
+    $add_rule = $false
+    $matching_rules = $fw.Rules | ? { $_.Name -eq "Windows Remote Management (HTTP-In)" }
+    $rule = $null
+    If ($matching_rules) {
+        If ($matching_rules -isnot [Array]) {
+            Write-Verbose "Editing existing single HTTP firewall rule"
+            $rule = $matching_rules
+        }
+        Else {
+            # try to find one with the All or Public profile first
+            Write-Verbose "Found multiple existing HTTP firewall rules..."
+            $rule = $matching_rules | % { $_.Profiles -band 4 }[0]
+
+            If (-not $rule -or $rule -is [Array]) {
+                Write-Verbose "Editing an arbitrary single HTTP firewall rule (multiple existed)"
+                # oh well, just pick the first one
+                $rule = $matching_rules[0]
+            }
+        }
+    }
+
+    If (-not $rule) {
+        Write-Verbose "Creating a new HTTP firewall rule"
+        $rule = New-Object -ComObject HNetCfg.FWRule
+        $rule.Name = "Windows Remote Management (HTTP-In)"
+        $rule.Description = "Inbound rule for Windows Remote Management via WS-Management. [TCP 5985]"
+        $add_rule = $true
+    }
+
+    $rule.Profiles = 0x7FFFFFFF
+    $rule.Protocol = 6
+    $rule.LocalPorts = 5985
+    $rule.RemotePorts = "*"
+    $rule.LocalAddresses = "*"
+    $rule.RemoteAddresses = "*"
+    $rule.Enabled = $true
+    $rule.Direction = 1
+    $rule.Action = 1
+    $rule.Grouping = "Windows Remote Management"
+
+    If ($add_rule) {
+        $fw.Rules.Add($rule)
+    }
+
+    Write-Verbose "HTTP firewall rule $($rule.Name) updated"
+}
+
 # Setup error handling.
 Trap
 {
@@ -137,8 +192,8 @@ $adminRole=[System.Security.Principal.WindowsBuiltInRole]::Administrator
 # Check to see if we are currently running "as Administrator"
 if (-Not $myWindowsPrincipal.IsInRole($adminRole))
 {
-    Write-Host "ERROR: You need elevated Administrator privileges in order to run this script."
-    Write-Host "       Start Windows PowerShell by using the Run as Administrator option."
+    Write-Output "ERROR: You need elevated Administrator privileges in order to run this script."
+    Write-Output "       Start Windows PowerShell by using the Run as Administrator option."
     Exit 2
 }
 
@@ -275,6 +330,10 @@ If ($EnableCredSSP)
     }
 }
 
+If ($GlobalHttpFirewallAccess) {
+    Enable-GlobalHttpFirewallAccess
+}
+
 # Configure firewall to allow WinRM HTTPS connections.
 $fwtest1 = netsh advfirewall firewall show rule name="Allow WinRM HTTPS"
 $fwtest2 = netsh advfirewall firewall show rule name="Allow WinRM HTTPS" profile=any

hacking/test-module

@@ -42,6 +42,7 @@ from ansible.parsing.utils.jsonify import jsonify
 from ansible.parsing.splitter import parse_kv
 import ansible.executor.module_common as module_common
 import ansible.constants as C
+from ansible.module_utils._text import to_text
 
 try:
     import json
@@ -159,7 +160,7 @@ def boilerplate_module(modfile, args, interpreters, check, destfile):
     print("* including generated source, if any, saving to: %s" % modfile2_path)
     if module_style not in ('ansiballz', 'old'):
         print("* this may offset any line numbers in tracebacks/debuggers!")
-    modfile2 = open(modfile2_path, 'w')
+    modfile2 = open(modfile2_path, 'wb')
     modfile2.write(module_data)
     modfile2.close()
     modfile = modfile2_path
@@ -178,6 +179,7 @@ def ansiballz_setup(modfile, modname, interpreters):
 
     cmd = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     out, err = cmd.communicate()
+    out, err = to_text(out, errors='surrogate_or_strict'), to_text(err)
     lines = out.splitlines()
     if len(lines) != 2 or 'Module expanded into' not in lines[0]:
         print("*" * 35)
@@ -209,6 +211,7 @@ def runtest(modfile, argspath, modname, module_style, interpreters):
 
     cmd = subprocess.Popen(invoke, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
     (out, err) = cmd.communicate()
+    out, err = to_text(out), to_text(err)
 
     try:
         print("*" * 35)
@@ -228,11 +231,11 @@ def runtest(modfile, argspath, modname, module_style, interpreters):
     print(jsonify(results,format=True))
 
 
-def rundebug(debugger, modfile, argspath, modname, module_style):
+def rundebug(debugger, modfile, argspath, modname, module_style, interpreters):
     """Run interactively with console debugger."""
 
     if module_style == 'ansiballz':
-        modfile, argspath = ansiballz_setup(modfile, modname)
+        modfile, argspath = ansiballz_setup(modfile, modname, interpreters)
 
     if argspath is not None:
         subprocess.call("%s %s %s" % (debugger, modfile, argspath), shell=True)
@@ -257,7 +260,7 @@ def main():
 
     if options.execute:
         if options.debugger:
-            rundebug(options.debugger, modfile, argspath, modname, module_style)
+            rundebug(options.debugger, modfile, argspath, modname, module_style, interpreters)
         else:
             runtest(modfile, argspath, modname, module_style, interpreters)
 

lib/ansible/cli/__init__.py

@@ -36,6 +36,7 @@ from ansible import constants as C
 from ansible.compat.six import with_metaclass
 from ansible.errors import AnsibleError, AnsibleOptionsError
 from ansible.module_utils._text import to_bytes, to_text
+from ansible.utils.path import unfrackpath
 
 try:
     from __main__ import display
@@ -265,6 +266,10 @@ class CLI(with_metaclass(ABCMeta, object)):
     def expand_tilde(option, opt, value, parser):
         setattr(parser.values, option.dest, os.path.expanduser(value))
 
+    @staticmethod
+    def unfrack_path(option, opt, value, parser):
+        setattr(parser.values, option.dest, unfrackpath(value))
+
     @staticmethod
     def expand_paths(option, opt, value, parser):
         """optparse action callback to convert a PATH style string arg to a list of path strings.
@@ -339,7 +344,8 @@ class CLI(with_metaclass(ABCMeta, object)):
             connect_group.add_option('-k', '--ask-pass', default=C.DEFAULT_ASK_PASS, dest='ask_pass', action='store_true',
                 help='ask for connection password')
             connect_group.add_option('--private-key','--key-file', default=C.DEFAULT_PRIVATE_KEY_FILE, dest='private_key_file',
-                help='use this file to authenticate the connection')
+                help='use this file to authenticate the connection',
+                action="callback", callback=CLI.unfrack_path, type=str)
             connect_group.add_option('-u', '--user', default=C.DEFAULT_REMOTE_USER, dest='remote_user',
                 help='connect as this user (default=%s)' % C.DEFAULT_REMOTE_USER)
             connect_group.add_option('-c', '--connection', dest='connection', default=C.DEFAULT_TRANSPORT,
@@ -574,10 +580,10 @@ class CLI(with_metaclass(ABCMeta, object)):
         ''' find reasonable way to display text '''
         # this is a much simpler form of what is in pydoc.py
         if not sys.stdout.isatty():
-            display.display(text)
+            display.display(text, screen_only=True)
         elif 'PAGER' in os.environ:
             if sys.platform == 'win32':
-                display.display(text)
+                display.display(text, screen_only=True)
             else:
                 self.pager_pipe(text, os.environ['PAGER'])
         else:
@@ -586,7 +592,7 @@ class CLI(with_metaclass(ABCMeta, object)):
             if p.returncode == 0:
                 self.pager_pipe(text, 'less')
             else:
-                display.display(text)
+                display.display(text, screen_only=True)
 
     @staticmethod
     def pager_pipe(text, cmd):

lib/ansible/cli/adhoc.py

@@ -174,6 +174,9 @@ class AdHocCLI(CLI):
             cb = self.callback
         elif self.options.one_line:
             cb = 'oneline'
+        # Respect custom 'stdout_callback' only with enabled 'bin_ansible_callbacks'
+        elif C.DEFAULT_LOAD_CALLBACK_PLUGINS and C.DEFAULT_STDOUT_CALLBACK != 'default':
+            cb = C.DEFAULT_STDOUT_CALLBACK
         else:
             cb = 'minimal'
 

lib/ansible/cli/galaxy.py

@@ -379,8 +379,9 @@ class GalaxyCLI(CLI):
                                         (role.name, role.install_info['version'], role.version or "unspecified"))
                         continue
                 else:
-                    display.display('- %s is already installed, skipping.' % str(role))
-                    continue
+                    if not force:
+                        display.display('- %s is already installed, skipping.' % str(role))
+                        continue
 
             try:
                 installed = role.install()

lib/ansible/compat/six/__init__.py

@@ -50,5 +50,12 @@ if _system_six:
     six = _system_six
 else:
     from . import _six as six
+
 six_py_file = '{0}.py'.format(os.path.splitext(six.__file__)[0])
+if not os.path.exists(six_py_file):
+    # Only the .pyc/.pyo version of six was installed but we need a .py file
+    # Fallback to the bundled copy
+    from . import _six as six
+    six_py_file = '{0}.py'.format(os.path.splitext(six.__file__)[0])
+
 exec(open(six_py_file, 'rb').read())

lib/ansible/constants.py

@@ -234,6 +234,7 @@ DEFAULT_FORCE_HANDLERS    = get_config(p, DEFAULTS, 'force_handlers', 'ANSIBLE_F
 DEFAULT_INVENTORY_IGNORE  = get_config(p, DEFAULTS, 'inventory_ignore_extensions', 'ANSIBLE_INVENTORY_IGNORE', ["~", ".orig", ".bak", ".ini", ".cfg", ".retry", ".pyc", ".pyo"], value_type='list')
 DEFAULT_VAR_COMPRESSION_LEVEL = get_config(p, DEFAULTS, 'var_compression_level', 'ANSIBLE_VAR_COMPRESSION_LEVEL', 0, value_type='integer')
 DEFAULT_INTERNAL_POLL_INTERVAL = get_config(p, DEFAULTS, 'internal_poll_interval', None, 0.001, value_type='float')
+DEFAULT_ALLOW_UNSAFE_LOOKUPS = get_config(p, DEFAULTS, 'allow_unsafe_lookups', None, False, value_type='boolean')
 ERROR_ON_MISSING_HANDLER  = get_config(p, DEFAULTS, 'error_on_missing_handler', 'ANSIBLE_ERROR_ON_MISSING_HANDLER', True, value_type='boolean')
 SHOW_CUSTOM_STATS = get_config(p, DEFAULTS, 'show_custom_stats', 'ANSIBLE_SHOW_CUSTOM_STATS', False, value_type='boolean')
 
@@ -299,7 +300,9 @@ DEFAULT_FILTER_PLUGIN_PATH     = get_config(p, DEFAULTS, 'filter_plugins',     '
 DEFAULT_TEST_PLUGIN_PATH       = get_config(p, DEFAULTS, 'test_plugins',       'ANSIBLE_TEST_PLUGINS', '~/.ansible/plugins/test:/usr/share/ansible/plugins/test', value_type='pathlist')
 DEFAULT_STRATEGY_PLUGIN_PATH   = get_config(p, DEFAULTS, 'strategy_plugins',   'ANSIBLE_STRATEGY_PLUGINS', '~/.ansible/plugins/strategy:/usr/share/ansible/plugins/strategy', value_type='pathlist')
 
-NETWORK_GROUP_MODULES          = get_config(p, DEFAULTS, 'network_group_modules','NETWORK_GROUP_MODULES', ['eos', 'nxos', 'ios', 'iosxr', 'junos', 'vyos'], value_type='list')
+NETWORK_GROUP_MODULES          = get_config(p, DEFAULTS, 'network_group_modules','NETWORK_GROUP_MODULES', ['eos', 'nxos', 'ios', 'iosxr', 'junos',
+                                                                                                           'vyos', 'sros', 'dellos9', 'dellos10', 'dellos6'],
+                                            value_type='list')
 
 DEFAULT_STRATEGY               = get_config(p, DEFAULTS, 'strategy',           'ANSIBLE_STRATEGY', 'linear')
 DEFAULT_STDOUT_CALLBACK        = get_config(p, DEFAULTS, 'stdout_callback',    'ANSIBLE_STDOUT_CALLBACK', 'default')

lib/ansible/errors/__init__.py

@@ -201,3 +201,9 @@ class AnsibleModuleExit(Exception):
     ''' local module exit '''
     def __init__(self, result):
         self.result = result
+class AnsibleActionSkip(AnsibleRuntimeError):
+    ''' an action runtime skip'''
+    pass
+class AnsibleActionFail(AnsibleRuntimeError):
+    ''' an action runtime failure'''
+    pass

lib/ansible/executor/module_common.py

@@ -37,7 +37,7 @@ from ansible import constants as C
 from ansible.errors import AnsibleError
 from ansible.module_utils._text import to_bytes, to_text
 from ansible.plugins import module_utils_loader
-from ansible.plugins.shell.powershell import async_watchdog, async_wrapper, become_wrapper, leaf_exec
+from ansible.plugins.shell.powershell import async_watchdog, async_wrapper, become_wrapper, leaf_exec, exec_wrapper
 # Must import strategy and use write_locks from there
 # If we import write_locks directly then we end up binding a
 # variable to the object and then it never gets updated.
@@ -121,11 +121,11 @@ import __main__
 # stdlib module
 scriptdir = None
 try:
-    scriptdir = os.path.dirname(os.path.abspath(__main__.__file__))
+    scriptdir = os.path.dirname(os.path.realpath(__main__.__file__))
 except (AttributeError, OSError):
     # Some platforms don't set __file__ when reading from stdin
     # OSX raises OSError if using abspath() in a directory we don't have
-    # permission to read.
+    # permission to read (realpath calls abspath)
     pass
 if scriptdir is not None:
     sys.path = [p for p in sys.path if p != scriptdir]
@@ -345,7 +345,7 @@ if __name__ == '__main__':
     finally:
         try:
             shutil.rmtree(temp_path)
-        except OSError:
+        except (NameError, OSError):
             # tempdir creation probably failed
             pass
     sys.exit(exitcode)
@@ -582,7 +582,8 @@ def _is_binary(b_module_data):
     return bool(start.translate(None, textchars))
 
 
-def _find_module_utils(module_name, b_module_data, module_path, module_args, task_vars, module_compression):
+def _find_module_utils(module_name, b_module_data, module_path, module_args, task_vars, module_compression, async_timeout, become,
+                       become_method, become_user, become_password, environment):
     """
     Given the source of the module, convert it to a Jinja2 template to insert
     module code and return whether it's a new or old style module.
@@ -742,8 +743,55 @@ def _find_module_utils(module_name, b_module_data, module_path, module_args, tas
         # Windows text editors
         shebang = u'#!powershell'
 
-        # powershell wrapper build is currently handled in build_windows_module_payload, called in action
-        # _configure_module after this function returns.
+        exec_manifest = dict(
+            module_entry=to_text(base64.b64encode(b_module_data)),
+            powershell_modules=dict(),
+            module_args=module_args,
+            actions=['exec'],
+            environment=environment
+        )
+
+        exec_manifest['exec'] = to_text(base64.b64encode(to_bytes(leaf_exec)))
+
+        if async_timeout > 0:
+            exec_manifest["actions"].insert(0, 'async_watchdog')
+            exec_manifest["async_watchdog"] = to_text(base64.b64encode(to_bytes(async_watchdog)))
+            exec_manifest["actions"].insert(0, 'async_wrapper')
+            exec_manifest["async_wrapper"] = to_text(base64.b64encode(to_bytes(async_wrapper)))
+            exec_manifest["async_jid"] = str(random.randint(0, 999999999999))
+            exec_manifest["async_timeout_sec"] = async_timeout
+
+        if become and become_method == 'runas':
+            exec_manifest["actions"].insert(0, 'become')
+            exec_manifest["become_user"] = become_user
+            exec_manifest["become_password"] = become_password
+            exec_manifest["become"] = to_text(base64.b64encode(to_bytes(become_wrapper)))
+
+        lines = b_module_data.split(b'\n')
+        module_names = set()
+
+        requires_module_list = re.compile(r'(?i)^#requires \-module(?:s?) (.+)')
+
+        for line in lines:
+            # legacy, equivalent to #Requires -Modules powershell
+            if REPLACER_WINDOWS in line:
+                module_names.add(b'powershell')
+                # TODO: add #Requires checks for Ansible.ModuleUtils.X
+
+        for m in module_names:
+            m = to_text(m)
+            exec_manifest["powershell_modules"][m] = to_text(
+                base64.b64encode(
+                    to_bytes(
+                        _slurp(os.path.join(_MODULE_UTILS_PATH, m + ".ps1"))
+                    )
+                )
+            )
+
+        # FUTURE: smuggle this back as a dict instead of serializing here; the connection plugin may need to modify it
+        module_json = json.dumps(exec_manifest)
+
+        b_module_data = exec_wrapper.replace(b"$json_raw = ''", b"$json_raw = @'\r\n%s\r\n'@" % to_bytes(module_json))
 
     elif module_substyle == 'jsonargs':
         module_args_json = to_bytes(json.dumps(module_args))
@@ -767,7 +815,8 @@ def _find_module_utils(module_name, b_module_data, module_path, module_args, tas
     return (b_module_data, module_style, shebang)
 
 
-def modify_module(module_name, module_path, module_args, task_vars=dict(), module_compression='ZIP_STORED'):
+def modify_module(module_name, module_path, module_args, task_vars=dict(), module_compression='ZIP_STORED', async_timeout=0, become=False,
+                  become_method=None, become_user=None, become_password=None, environment=dict()):
     """
     Used to insert chunks of code into modules before transfer rather than
     doing regular python imports.  This allows for more efficient transfer in
@@ -793,7 +842,10 @@ def modify_module(module_name, module_path, module_args, task_vars=dict(), modul
         # read in the module source
         b_module_data = f.read()
 
-    (b_module_data, module_style, shebang) = _find_module_utils(module_name, b_module_data, module_path, module_args, task_vars, module_compression)
+    (b_module_data, module_style, shebang) = _find_module_utils(module_name, b_module_data, module_path, module_args, task_vars, module_compression,
+                                                                async_timeout=async_timeout, become=become, become_method=become_method,
+                                                                become_user=become_user, become_password=become_password,
+                                                                environment=environment)
 
     if module_style == 'binary':
         return (b_module_data, module_style, to_text(shebang, nonstring='passthru'))
@@ -820,47 +872,3 @@ def modify_module(module_name, module_path, module_args, task_vars=dict(), modul
         shebang = to_bytes(shebang, errors='surrogate_or_strict')
 
     return (b_module_data, module_style, to_text(shebang, nonstring='passthru'))
-
-def build_windows_module_payload(module_name, module_path, b_module_data, module_args, task_vars, task, play_context):
-    exec_manifest = dict(
-        module_entry=base64.b64encode(b_module_data),
-        powershell_modules=dict(),
-        module_args=module_args,
-        actions=['exec']
-    )
-
-    exec_manifest['exec'] = base64.b64encode(to_bytes(leaf_exec))
-
-    if task.async > 0:
-        exec_manifest["actions"].insert(0, 'async_watchdog')
-        exec_manifest["async_watchdog"] = base64.b64encode(to_bytes(async_watchdog))
-        exec_manifest["actions"].insert(0, 'async_wrapper')
-        exec_manifest["async_wrapper"] = base64.b64encode(to_bytes(async_wrapper))
-        exec_manifest["async_jid"] = str(random.randint(0, 999999999999))
-        exec_manifest["async_timeout_sec"] = task.async
-
-    if play_context.become and play_context.become_method=='runas':
-        exec_manifest["actions"].insert(0, 'become')
-        exec_manifest["become_user"] = play_context.become_user
-        exec_manifest["become_password"] = play_context.become_pass
-        exec_manifest["become"] = base64.b64encode(to_bytes(become_wrapper))
-
-    lines = b_module_data.split(b'\n')
-    module_names = set()
-
-    requires_module_list = re.compile(r'(?i)^#requires \-module(?:s?) (.+)')
-
-    for line in lines:
-        # legacy, equivalent to #Requires -Modules powershell
-        if REPLACER_WINDOWS in line:
-            module_names.add(b'powershell')
-            # TODO: add #Requires checks for Ansible.ModuleUtils.X
-
-    for m in module_names:
-        exec_manifest["powershell_modules"][m] = base64.b64encode(
-            to_bytes(_slurp(os.path.join(_MODULE_UTILS_PATH, m + ".ps1"))))
-
-    # FUTURE: smuggle this back as a dict instead of serializing here; the connection plugin may need to modify it
-    b_module_data = json.dumps(exec_manifest)
-
-    return b_module_data

lib/ansible/executor/play_iterator.py

@@ -24,7 +24,6 @@ import fnmatch
 from ansible.compat.six import iteritems
 from ansible import constants as C
 from ansible.errors import AnsibleError
-from ansible.module_utils.six import cmp
 from ansible.playbook.block import Block
 from ansible.playbook.task import Task
 from ansible.playbook.role_include import IncludeRole

lib/ansible/executor/process/worker.py

@@ -68,21 +68,25 @@ class WorkerProcess(multiprocessing.Process):
         self._variable_manager  = variable_manager
         self._shared_loader_obj = shared_loader_obj
 
-        # dupe stdin, if we have one
-        self._new_stdin = sys.stdin
-        try:
-            fileno = sys.stdin.fileno()
-            if fileno is not None:
-                try:
-                    self._new_stdin = os.fdopen(os.dup(fileno))
-                except OSError:
-                    # couldn't dupe stdin, most likely because it's
-                    # not a valid file descriptor, so we just rely on
-                    # using the one that was passed in
-                    pass
-        except (AttributeError, ValueError):
-            # couldn't get stdin's fileno, so we just carry on
-            pass
+        if sys.stdin.isatty():
+            # dupe stdin, if we have one
+            self._new_stdin = sys.stdin
+            try:
+                fileno = sys.stdin.fileno()
+                if fileno is not None:
+                    try:
+                        self._new_stdin = os.fdopen(os.dup(fileno))
+                    except OSError:
+                        # couldn't dupe stdin, most likely because it's
+                        # not a valid file descriptor, so we just rely on
+                        # using the one that was passed in
+                        pass
+            except (AttributeError, ValueError):
+                # couldn't get stdin's fileno, so we just carry on
+                pass
+        else:
+            # set to /dev/null
+            self._new_stdin = os.devnull
 
     def run(self):
         '''
@@ -112,7 +116,7 @@ class WorkerProcess(multiprocessing.Process):
                 self._rslt_q
             ).run()
 
-            display.debug("done running TaskExecutor() for %s/%s" % (self._host, self._task))
+            display.debug("done running TaskExecutor() for %s/%s [%s]" % (self._host, self._task, self._task._uuid))
             self._host.vars = dict()
             self._host.groups = []
             task_result = TaskResult(
@@ -123,9 +127,9 @@ class WorkerProcess(multiprocessing.Process):
             )
 
             # put the result on the result queue
-            display.debug("sending task result")
+            display.debug("sending task result for task %s" % self._task._uuid)
             self._rslt_q.put(task_result)
-            display.debug("done sending task result")
+            display.debug("done sending task result for task %s" % self._task._uuid)
 
         except AnsibleConnectionFailure:
             self._host.vars = dict()

lib/ansible/executor/task_executor.py

@@ -20,18 +20,18 @@ from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type
 
 import base64
-import sys
 import time
 import traceback
 
 from ansible.compat.six import iteritems, string_types, binary_type
 
 from ansible import constants as C
-from ansible.errors import AnsibleError, AnsibleParserError, AnsibleUndefinedVariable, AnsibleConnectionFailure
+from ansible.errors import AnsibleError, AnsibleParserError, AnsibleUndefinedVariable, AnsibleConnectionFailure, AnsibleActionFail, AnsibleActionSkip
 from ansible.executor.task_result import TaskResult
 from ansible.module_utils._text import to_text
 from ansible.playbook.conditional import Conditional
 from ansible.playbook.task import Task
+from ansible.plugins.connection import ConnectionBase
 from ansible.template import Templar
 from ansible.utils.encrypt import key_for_hostname
 from ansible.utils.listify import listify_lookup_plugin_terms
@@ -82,7 +82,7 @@ class TaskExecutor:
         returned as a dict.
         '''
 
-        display.debug("in run()")
+        display.debug("in run() - task %s" % self._task._uuid)
 
         try:
             try:
@@ -471,16 +471,17 @@ class TaskExecutor:
         if not self._connection or not getattr(self._connection, 'connected', False) or self._play_context.remote_addr != self._connection._play_context.remote_addr:
             self._connection = self._get_connection(variables=variables, templar=templar)
             hostvars = variables.get('hostvars', None)
-            if hostvars:
+            # only template the vars if the connection actually implements set_host_overrides
+            # NB: this is expensive, and should be removed once connection-specific vars are being handled by play_context
+            sho_impl = getattr(type(self._connection), 'set_host_overrides', None)
+            if hostvars and sho_impl and sho_impl != ConnectionBase.set_host_overrides:
                 try:
-                    target_hostvars = hostvars.raw_get(self._host.name)
+                    target_hostvars = hostvars.get(self._host.name)
                 except:
                     # FIXME: this should catch the j2undefined error here
                     #        specifically instead of all exceptions
                     target_hostvars = dict()
-            else:
-                target_hostvars = dict()
-            self._connection.set_host_overrides(host=self._host, hostvars=target_hostvars)
+                self._connection.set_host_overrides(host=self._host, hostvars=target_hostvars)
         else:
             # if connection is reused, its _play_context is no longer valid and needs
             # to be replaced with the one templated above, in case other data changed
@@ -519,6 +520,10 @@ class TaskExecutor:
             display.debug("running the handler")
             try:
                 result = self._handler.run(task_vars=variables)
+            except AnsibleActionSkip as e:
+                return dict(skipped=True, msg=to_text(e))
+            except AnsibleActionFail as e:
+                return dict(failed=True, msg=to_text(e))
             except AnsibleConnectionFailure as e:
                 return dict(unreachable=True, msg=to_text(e))
             display.debug("handler run complete")
@@ -532,7 +537,7 @@ class TaskExecutor:
                 vars_copy[self._task.register] = wrap_var(result.copy())
 
             if self._task.async > 0:
-                if self._task.poll > 0 and not result.get('skipped'):
+                if self._task.poll > 0 and not result.get('skipped') and not result.get('failed'):
                     result = self._poll_async_result(result=result, templar=templar, task_vars=vars_copy)
                     #FIXME callback 'v2_runner_on_async_poll' here
 
@@ -611,7 +616,7 @@ class TaskExecutor:
         #        there is another source of truth we can use
         delegated_vars = variables.get('ansible_delegated_vars', dict()).get(self._task.delegate_to, dict()).copy()
         if len(delegated_vars) > 0:
-            result["_ansible_delegated_vars"] = dict()
+            result["_ansible_delegated_vars"] = {'ansible_delegated_host': self._task.delegate_to}
             for k in ('ansible_host', ):
                 result["_ansible_delegated_vars"][k] = delegated_vars.get(k)
 
@@ -704,27 +709,12 @@ class TaskExecutor:
                     if isinstance(i, string_types) and i.startswith("ansible_") and i.endswith("_interpreter"):
                         variables[i] = delegated_vars[i]
 
-        conn_type = self._play_context.connection
-        if conn_type == 'smart':
-            conn_type = 'ssh'
-            if sys.platform.startswith('darwin') and self._play_context.password:
-                # due to a current bug in sshpass on OSX, which can trigger
-                # a kernel panic even for non-privileged users, we revert to
-                # paramiko on that OS when a SSH password is specified
-                conn_type = "paramiko"
-            else:
-                # see if SSH can support ControlPersist if not use paramiko
-                if not check_for_controlpersist(self._play_context.ssh_executable):
-                    conn_type = "paramiko"
-
-        # if someone did `connection: persistent`, default it to using a persistent paramiko connection to avoid problems
-        if conn_type == 'persistent':
-            self._play_context.connection = 'paramiko'
-
-        # if using persistent connections (or the action has set the FORCE_PERSISTENT_CONNECTION attribute to True),
+        # if using persistent paramiko connections (or the action has set the FORCE_PERSISTENT_CONNECTION attribute to True),
         # then we use the persistent connection plugion. Otherwise load the requested connection plugin
-        elif C.USE_PERSISTENT_CONNECTIONS or getattr(self, 'FORCE_PERSISTENT_CONNECTION', False):
+        if C.USE_PERSISTENT_CONNECTIONS or getattr(self, 'FORCE_PERSISTENT_CONNECTION', False):
             conn_type == 'persistent'
+        else:
+            conn_type = self._play_context.connection
 
         connection = self._shared_loader_obj.connection_loader.get(conn_type, self._play_context, self._new_stdin)
 

lib/ansible/executor/task_queue_manager.py

@@ -136,10 +136,13 @@ class TaskQueueManager:
         handler_list = []
         for handler_block in play.handlers:
             handler_list.extend(_process_block(handler_block))
-
         # then initialize it with the given handler list
+        self.update_handler_list(handler_list)
+
+    def update_handler_list(self, handler_list):
         for handler in handler_list:
             if handler._uuid not in self._notified_handlers:
+                display.debug("Adding handler %s to notified list" % handler.name)
                 self._notified_handlers[handler._uuid] = []
             if handler.listen:
                 listeners = handler.listen
@@ -148,6 +151,7 @@ class TaskQueueManager:
                 for listener in listeners:
                     if listener not in self._listening_handlers:
                         self._listening_handlers[listener] = []
+                    display.debug("Adding handler %s to listening list" % handler.name)
                     self._listening_handlers[listener].append(handler._uuid)
 
     def load_callbacks(self):
@@ -333,7 +337,7 @@ class TaskQueueManager:
         defunct = False
         for idx,x in enumerate(self._workers):
             if hasattr(x[0], 'exitcode'):
-                if x[0].exitcode in [-9, -15]:
+                if x[0].exitcode in [-9, -11, -15]:
                     defunct = True
         return defunct
 

lib/ansible/galaxy/role.py

@@ -316,7 +316,7 @@ class GalaxyRole(object):
                             # bits that might be in the file for security purposes
                             # and drop any containing directory, as mentioned above
                             if member.isreg() or member.issym():
-                                parts = member.name.replace(archive_parent_dir, "").split(os.sep)
+                                parts = member.name.replace(archive_parent_dir, "", 1).split(os.sep)
                                 final_parts = []
                                 for part in parts:
                                     if part != '..' and '~' not in part and '$' not in part:

lib/ansible/inventory/__init__.py

@@ -162,6 +162,7 @@ class Inventory(object):
 
         ### POST PROCESS groups and hosts after specific parser was invoked
 
+        hosts = []
         group_names = set()
         # set group vars from group_vars/ files and vars plugins
         for g in self.groups:
@@ -169,10 +170,11 @@ class Inventory(object):
             group.vars = combine_vars(group.vars, self.get_group_variables(group.name))
             self.get_group_vars(group)
             group_names.add(group.name)
+            hosts.extend(group.get_hosts())
 
         host_names = set()
         # get host vars from host_vars/ files and vars plugins
-        for host in self.get_hosts(ignore_limits=True, ignore_restrictions=True):
+        for host in hosts:
             host.vars = combine_vars(host.vars, self.get_host_variables(host.name))
             self.get_host_vars(host)
             host_names.add(host.name)
@@ -473,30 +475,20 @@ class Inventory(object):
         """
 
         results = []
-        hostnames = set()
 
         def __append_host_to_results(host):
-            if host.name not in hostnames:
-                hostnames.add(host.name)
+            if not host.implicit:
                 results.append(host)
 
         groups = self.get_groups()
         for group in groups.values():
-            if pattern == 'all':
+            if self._match(group.name, pattern):
                 for host in group.get_hosts():
-                    if host.implicit:
-                        continue
                     __append_host_to_results(host)
             else:
-                if self._match(group.name, pattern) and group.name not in ('all', 'ungrouped'):
-                    for host in group.get_hosts():
-                        if host.implicit:
-                            continue
-                        __append_host_to_results(host)
-                else:
-                    matching_hosts = self._match_list(group.get_hosts(), 'name', pattern)
-                    for host in matching_hosts:
-                        __append_host_to_results(host)
+                matching_hosts = self._match_list(group.get_hosts(), 'name', pattern)
+                for host in matching_hosts:
+                    __append_host_to_results(host)
 
         if pattern in C.LOCALHOST and len(results) == 0:
             new_host = self._create_implicit_localhost(pattern)

lib/ansible/inventory/group.py

@@ -96,6 +96,8 @@ class Group:
             # isn't already a group with the same name
             if self.name not in [g.name for g in group.parent_groups]:
                 group.parent_groups.append(self)
+                for h in group.get_hosts():
+                    h.populate_ancestors()
 
             self.clear_hosts_cache()
 

lib/ansible/inventory/host.py

@@ -106,27 +106,42 @@ class Host:
     def set_gathered_facts(self, gathered):
         self._gathered_facts = gathered
 
+    def populate_ancestors(self):
+
+        # populate ancestors
+        for group in self.groups:
+            self.add_group(group)
+
     def add_group(self, group):
 
-        self.groups.append(group)
+        # populate ancestors
+        for oldg in group.get_ancestors():
+            if oldg not in self.groups:
+                self.add_group(oldg)
+
+        if group not in self.groups:
+            self.groups.append(group)
 
     def remove_group(self, group):
 
-        self.groups.remove(group)
+        if group in self.groups:
+            self.groups.remove(group)
+
+            # remove exclusive ancestors, xcept all!
+            for oldg in group.get_ancestors():
+                if oldg.name != 'all':
+                    for childg in self.groups:
+                        if oldg in childg.get_ancestors():
+                            break
+                    else:
+                        self.remove_group(oldg)
 
     def set_variable(self, key, value):
 
         self.vars[key]=value
 
     def get_groups(self):
-
-        groups = {}
-        for g in self.groups:
-            groups[g.name] = g
-            ancestors = g.get_ancestors()
-            for a in ancestors:
-                groups[a.name] = a
-        return groups.values()
+        return self.groups
 
     def get_vars(self):
 

lib/ansible/inventory/ini.py

@@ -103,7 +103,6 @@ class InventoryParser(object):
             self.lineno += 1
 
             line = line.strip()
-
             # Skip empty lines and comments
             if not line or line[0] in self._COMMENT_MARKERS:
                 continue
@@ -156,6 +155,9 @@ class InventoryParser(object):
                 for h in hosts:
                     self.groups[groupname].add_host(h)
 
+                #FIXME: needed to save hosts to group, find out why
+                self.groups[groupname].get_hosts()
+
             # [groupname:vars] contains variable definitions that must be
             # applied to the current group.
             elif state == 'vars':
@@ -202,7 +204,7 @@ class InventoryParser(object):
         # 'all' at the time it was created.
 
         for group in self.groups.values():
-            if group.depth == 0 and group.name not in ('all', 'ungrouped'):
+            if group.depth == 0 and group.name != 'all':
                 self.groups['all'].add_child_group(group)
 
     def _parse_group_name(self, line):

lib/ansible/inventory/yaml.py

@@ -70,7 +70,7 @@ class InventoryParser(object):
         # 'all' at the time it was created.
         for group in self.groups.values():
             if group.depth == 0 and group.name not in ('all', 'ungrouped'):
-                self.groups['all'].add_child_group(Group(group_name))
+                self.groups['all'].add_child_group(group)
 
     def _parse_groups(self, group, group_data):
 
@@ -83,19 +83,19 @@ class InventoryParser(object):
                 if section in group_data and isinstance(group_data[section], string_types):
                     group_data[section] = { group_data[section]: None}
 
-            if 'vars' in group_data:
+            if group_data.get('vars', False):
                 for var in group_data['vars']:
                     if var != 'ansible_group_priority':
                         self.groups[group].set_variable(var, group_data['vars'][var])
                     else:
                         self.groups[group].set_priority(group_data['vars'][var])
 
-            if 'children' in group_data:
+            if group_data.get('children', False):
                 for subgroup in group_data['children']:
                     self._parse_groups(subgroup, group_data['children'][subgroup])
                     self.groups[group].add_child_group(self.groups[subgroup])
 
-            if 'hosts' in group_data:
+            if group_data.get('hosts', False):
                 for host_pattern in group_data['hosts']:
                     hosts = self._parse_host(host_pattern, group_data['hosts'][host_pattern])
                     for h in hosts:

lib/ansible/module_utils/_text.py

@@ -44,9 +44,9 @@ except LookupError:
     HAS_SURROGATEESCAPE = False
 
 
-_COMPOSED_ERROR_HANDLERS = frozenset((None, 'surrogate_or_escape',
-                                     'surrogate_or_strict',
-                                     'surrogate_then_replace'))
+_COMPOSED_ERROR_HANDLERS = frozenset((None, 'surrogate_or_replace',
+                                      'surrogate_or_strict',
+                                      'surrogate_then_replace'))
 
 
 def to_bytes(obj, encoding='utf-8', errors=None, nonstring='simplerepr'):
@@ -133,6 +133,9 @@ def to_bytes(obj, encoding='utf-8', errors=None, nonstring='simplerepr'):
             return obj.encode(encoding, errors)
         except UnicodeEncodeError:
             if original_errors in (None, 'surrogate_then_replace'):
+                # We should only reach this if encoding was non-utf8 original_errors was
+                # surrogate_then_escape and errors was surrogateescape
+
                 # Slow but works
                 return_string = obj.encode('utf-8', 'surrogateescape')
                 return_string = return_string.decode('utf-8', 'replace')

lib/ansible/module_utils/ansible_tower.py

@@ -62,7 +62,7 @@ def tower_auth_config(module):
         if password:
             auth_config['password'] = password
         verify_ssl = module.params.get('tower_verify_ssl')
-        if verify_ssl:
+        if verify_ssl is not None:
             auth_config['verify_ssl'] = verify_ssl
         return auth_config
 

lib/ansible/module_utils/aos.py

@@ -135,11 +135,11 @@ def content_to_dict(module, content):
         content_dict = yaml.safe_load(content)
 
         if not isinstance(content_dict, dict):
-            raise
+            raise Exception()
 
         # Check if dict is empty and return an error if it's
         if not content_dict:
-            raise
+            raise Exception()
 
     except:
         module.fail_json(msg="Unable to convert 'content' to a dict, please check if valid")

lib/ansible/module_utils/asa.py

@@ -53,7 +53,9 @@ class Cli(CliBase):
     NET_PASSWD_RE = re.compile(r"[\r\n]?password: $", re.I)
 
     def __init__(self, *args, **kwargs):
+
         super(Cli, self).__init__(*args, **kwargs)
+        self.default_output = 'text'
 
     def connect(self, params, **kwargs):
         super(Cli, self).connect(params, kickstart=False, **kwargs)

lib/ansible/module_utils/basic.py

@@ -903,7 +903,7 @@ class AnsibleModule(object):
         return context
 
     def user_and_group(self, path, expand=True):
-        b_path = to_bytes(path, errors='surrogate_then_strict')
+        b_path = to_bytes(path, errors='surrogate_or_strict')
         if expand:
             b_path = os.path.expanduser(os.path.expandvars(b_path))
         st = os.lstat(b_path)
@@ -989,22 +989,22 @@ class AnsibleModule(object):
         return changed
 
     def set_owner_if_different(self, path, owner, changed, diff=None, expand=True):
-        b_path = to_bytes(path, errors='surrogate_then_strict')
+        b_path = to_bytes(path, errors='surrogate_or_strict')
         if expand:
             b_path = os.path.expanduser(os.path.expandvars(b_path))
-        path = to_text(b_path, errors='surrogate_then_strict')
         if owner is None:
             return changed
-        orig_uid, orig_gid = self.user_and_group(path, expand)
+        orig_uid, orig_gid = self.user_and_group(b_path, expand)
         try:
             uid = int(owner)
         except ValueError:
             try:
                 uid = pwd.getpwnam(owner).pw_uid
             except KeyError:
+                path = to_text(b_path)
                 self.fail_json(path=path, msg='chown failed: failed to look up user %s' % owner)
-        if orig_uid != uid:
 
+        if orig_uid != uid:
             if diff is not None:
                 if 'before' not in diff:
                     diff['before'] = {}
@@ -1018,15 +1018,15 @@ class AnsibleModule(object):
             try:
                 os.lchown(b_path, uid, -1)
             except OSError:
+                path = to_text(b_path)
                 self.fail_json(path=path, msg='chown failed')
             changed = True
         return changed
 
     def set_group_if_different(self, path, group, changed, diff=None, expand=True):
-        b_path = to_bytes(path, errors='surrogate_then_strict')
+        b_path = to_bytes(path, errors='surrogate_or_strict')
         if expand:
             b_path = os.path.expanduser(os.path.expandvars(b_path))
-        path = to_text(b_path, errors='surrogate_then_strict')
         if group is None:
             return changed
         orig_uid, orig_gid = self.user_and_group(b_path, expand)
@@ -1036,9 +1036,10 @@ class AnsibleModule(object):
             try:
                 gid = grp.getgrnam(group).gr_gid
             except KeyError:
+                path = to_text(b_path)
                 self.fail_json(path=path, msg='chgrp failed: failed to look up group %s' % group)
-        if orig_gid != gid:
 
+        if orig_gid != gid:
             if diff is not None:
                 if 'before' not in diff:
                     diff['before'] = {}
@@ -1052,15 +1053,15 @@ class AnsibleModule(object):
             try:
                 os.lchown(b_path, -1, gid)
             except OSError:
+                path = to_text(b_path)
                 self.fail_json(path=path, msg='chgrp failed')
             changed = True
         return changed
 
     def set_mode_if_different(self, path, mode, changed, diff=None, expand=True):
-        b_path = to_bytes(path, errors='surrogate_then_strict')
+        b_path = to_bytes(path, errors='surrogate_or_strict')
         if expand:
             b_path = os.path.expanduser(os.path.expandvars(b_path))
-        path = to_text(b_path, errors='surrogate_then_strict')
         path_stat = os.lstat(b_path)
 
         if mode is None:
@@ -1074,12 +1075,14 @@ class AnsibleModule(object):
                     mode = self._symbolic_mode_to_octal(path_stat, mode)
                 except Exception:
                     e = get_exception()
+                    path = to_text(b_path)
                     self.fail_json(path=path,
                                    msg="mode must be in octal or symbolic form",
                                    details=str(e))
 
                 if mode != stat.S_IMODE(mode):
                     # prevent mode from having extra info orbeing invalid long number
+                    path = to_text(b_path)
                     self.fail_json(path=path, msg="Invalid mode supplied, only permission info is allowed", details=mode)
 
         prev_mode = stat.S_IMODE(path_stat.st_mode)
@@ -1123,6 +1126,7 @@ class AnsibleModule(object):
                     raise e
             except Exception:
                 e = get_exception()
+                path = to_text(b_path)
                 self.fail_json(path=path, msg='chmod failed', details=str(e))
 
             path_stat = os.lstat(b_path)
@@ -1137,10 +1141,11 @@ class AnsibleModule(object):
         if attributes is None:
             return changed
 
-        b_path = to_bytes(path, errors='surrogate_then_strict')
+        b_path = to_bytes(path, errors='surrogate_or_strict')
         if expand:
             b_path = os.path.expanduser(os.path.expandvars(b_path))
-        path = to_text(b_path, errors='surrogate_then_strict')
+
+        existing = self.get_file_attributes(b_path)
 
         if existing.get('attr_flags','') != attributes:
             attrcmd = self.get_bin_path('chattr')
@@ -1163,6 +1168,7 @@ class AnsibleModule(object):
                             raise Exception("Error while setting attributes: %s" % (out + err))
                     except:
                         e = get_exception()
+                        path = to_text(b_path)
                         self.fail_json(path=path, msg='chattr failed', details=str(e))
         return changed
 
@@ -1853,7 +1859,7 @@ class AnsibleModule(object):
         try:
             cwd = os.getcwd()
             if not os.access(cwd, os.F_OK|os.R_OK):
-                raise
+                raise Exception()
             return cwd
         except:
             # we don't have access to the cwd, probably because of sudo.
@@ -1963,7 +1969,7 @@ class AnsibleModule(object):
                     else:
                        self.deprecate(d)
             else:
-                self.deprecate(d)
+                self.deprecate(kwargs['deprecations'])
 
         if self._deprecations:
             kwargs['deprecations'] = self._deprecations
@@ -2022,7 +2028,7 @@ class AnsibleModule(object):
                                    (filename, algorithm, ', '.join(AVAILABLE_HASH_ALGORITHMS)))
 
         blocksize = 64 * 1024
-        infile = open(filename, 'rb')
+        infile = open(os.path.realpath(filename), 'rb')
         block = infile.read(blocksize)
         while block:
             digest_method.update(block)
@@ -2358,7 +2364,7 @@ class AnsibleModule(object):
 
         clean_args = []
         is_passwd = False
-        for arg in to_clean_args:
+        for arg in (to_native(a) for a in to_clean_args):
             if is_passwd:
                 is_passwd = False
                 clean_args.append('********')

lib/ansible/module_utils/dellos10.py

@@ -1,5 +1,6 @@
 #
 # (c) 2015 Peter Sprygada, <psprygada@ansible.com>
+# (c) 2017 Red Hat, Inc
 #
 # Copyright (c) 2016 Dell Inc.
 #
@@ -28,28 +29,103 @@
 # LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
 # USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 #
-
 import re
 
-from ansible.module_utils.network import register_transport, to_list
-from ansible.module_utils.shell import CliBase
-from ansible.module_utils.netcfg import NetworkConfig, ConfigLine
+from ansible.module_utils.basic import env_fallback, return_values
+from ansible.module_utils.network_common import to_list, ComplexList
+from ansible.module_utils.connection import exec_command
+from ansible.module_utils.netcfg import NetworkConfig,ConfigLine
+
+_DEVICE_CONFIGS = {}
+
+WARNING_PROMPTS_RE = [
+    r"[\r\n]?\[confirm yes/no\]:\s?$",
+    r"[\r\n]?\[y/n\]:\s?$",
+    r"[\r\n]?\[yes/no\]:\s?$"
+]
+
+dellos10_argument_spec = {
+    'host': dict(),
+    'port': dict(type='int'),
+    'username': dict(fallback=(env_fallback, ['ANSIBLE_NET_USERNAME'])),
+    'password': dict(fallback=(env_fallback, ['ANSIBLE_NET_PASSWORD']), no_log=True),
+    'ssh_keyfile': dict(fallback=(env_fallback, ['ANSIBLE_NET_SSH_KEYFILE']), type='path'),
+    'authorize': dict(fallback=(env_fallback, ['ANSIBLE_NET_AUTHORIZE']), type='bool'),
+    'auth_pass': dict(fallback=(env_fallback, ['ANSIBLE_NET_AUTH_PASS']), no_log=True),
+    'timeout': dict(type='int'),
+    'provider': dict(type='dict'),
+}
+
+def check_args(module, warnings):
+    provider = module.params['provider'] or {}
+    for key in dellos10_argument_spec:
+        if key != 'provider' and module.params[key]:
+            warnings.append('argument %s has been deprecated and will be '
+                            'removed in a future version' % key)
+
+    if provider:
+        for param in ('auth_pass', 'password'):
+            if provider.get(param):
+                module.no_log_values.update(return_values(provider[param]))
 
 
-def get_config(module):
-    contents = module.params['config']
+def get_config(module, flags=[]):
+    cmd = 'show running-config '
+    cmd += ' '.join(flags)
+    cmd = cmd.strip()
 
-    if not contents:
-        contents = module.config.get_config()
-        module.params['config'] = contents
-        return NetworkConfig(indent=1, contents=contents[0])
-    else:
-        return NetworkConfig(indent=1, contents=contents)
+    try:
+        return _DEVICE_CONFIGS[cmd]
+    except KeyError:
+        rc, out, err = exec_command(module, cmd)
+        if rc != 0:
+            module.fail_json(msg='unable to retrieve current config', stderr=err)
+        cfg = str(out).strip()
+        _DEVICE_CONFIGS[cmd] = cfg
+        return cfg
 
 
+def to_commands(module, commands):
+    spec = {
+        'command': dict(key=True),
+        'prompt': dict(),
+        'answer': dict()
+    }
+    transform = ComplexList(spec, module)
+    return transform(commands)
+
+
+def run_commands(module, commands, check_rc=True):
+    responses = list()
+    commands = to_commands(module, to_list(commands))
+    for cmd in commands:
+        cmd = module.jsonify(cmd)
+        rc, out, err = exec_command(module, cmd)
+        if check_rc and rc != 0:
+            module.fail_json(msg=err, rc=rc)
+        responses.append(out)
+    return responses
+
+def load_config(module, commands):
+    rc, out, err = exec_command(module, 'configure terminal')
+    if rc != 0:
+        module.fail_json(msg='unable to enter configuration mode', err=err)
+
+    commands.append('commit')
+    for command in to_list(commands):
+        if command == 'end':
+            continue
+        cmd = {'command': command, 'prompt': WARNING_PROMPTS_RE, 'answer': 'yes'}
+        rc, out, err = exec_command(module, module.jsonify(cmd))
+        if rc != 0:
+            module.fail_json(msg=err, command=command, rc=rc)
+
+    exec_command(module, 'end')
+
 def get_sublevel_config(running_config, module):
     contents = list()
     current_config_contents = list()
+    running_config = NetworkConfig(contents=running_config, indent=1)
     obj = running_config.get_object(module.params['parents'])
     if obj:
         contents = obj.children
@@ -61,67 +137,8 @@ def get_sublevel_config(running_config, module):
             current_config_contents.append(c.rjust(len(c) + indent, ' '))
         if isinstance(c, ConfigLine):
             current_config_contents.append(c.raw)
-        indent = indent + 1
+        indent = 1
     sublevel_config = '\n'.join(current_config_contents)
 
     return sublevel_config
 
-
-class Cli(CliBase):
-
-    NET_PASSWD_RE = re.compile(r"[\r\n]?password:\s?$", re.I)
-
-    CLI_PROMPTS_RE = [
-        re.compile(r"[\r\n]?[\w+\-\.:\/\[\]]+(?:\([^\)]+\)){,3}(?:#) ?$"),
-        re.compile(r"\[\w+\@[\w\-\.]+(?: [^\]])\] ?[>#\$] ?$")
-    ]
-
-    CLI_ERRORS_RE = [
-        re.compile(r"% ?Error"),
-        re.compile(r"% ?Bad secret"),
-        re.compile(r"Syntax error:"),
-        re.compile(r"invalid input", re.I),
-        re.compile(r"(?:incomplete|ambiguous) command", re.I),
-        re.compile(r"connection timed out", re.I),
-        re.compile(r"[^\r\n]+ not found", re.I),
-        re.compile(r"'[^']' +returned error code: ?\d+"),
-    ]
-
-
-    def connect(self, params, **kwargs):
-        super(Cli, self).connect(params, kickstart=False, **kwargs)
-        self.shell.send('terminal length 0')
-
-
-    def configure(self, commands, **kwargs):
-        cmds = ['configure terminal']
-        cmds.extend(to_list(commands))
-        cmds.append('end')
-        cmds.append('commit')
-
-        responses = self.execute(cmds)
-        responses.pop(0)
-        return responses
-
-
-    def get_config(self, **kwargs):
-        return self.execute(['show running-configuration'])
-
-
-    def load_config(self, commands, **kwargs):
-        return self.configure(commands)
-
-
-    def commit_config(self, **kwargs):
-        self.execute(['commit'])
-
-
-    def abort_config(self, **kwargs):
-        self.execute(['discard'])
-
-
-    def save_config(self):
-        self.execute(['copy running-config startup-config'])
-
-
-Cli = register_transport('cli', default=True)(Cli)