wbrawner/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Backport/2.8/docs rst omnibus (#56310)

Browse source 
* Update windows_setup.rst (#55535): Wrong protocol and port in command.

(cherry picked from commit 6ea3eca8ff)

* Clarify the two targets of vault encryption, with notes about advantages and drawbacks of each

Co-Authored-By: tacatac <taca@kadisius.eu>
(cherry picked from commit 79198cad7a)

* Improve consistency of loop documentation (#55674)

(cherry picked from commit a5cb47d697)

* Add Microsoft Document URL for WinRM Memory Hotfix (#55680)

Co-Authored-By: hiyokotaisa <thel.vadam2485@gmail.com>
(cherry picked from commit 7b86208fcd)

* Clarify the documentation for `async` and `poll`; describe the behavior when `poll` = 0 and when it does not.

Co-Authored-By: tacatac <taca@kadisius.eu>
(cherry picked from commit dbc64ae64c)

* Add security group info and example to AWS guide (#55783): expand documentation on how to use lookup plugin aws_service_ip_ranges with ec2_group module

(cherry picked from commit bb5059f2c7)

* correct description of modules vs plugins (#55784)

(cherry picked from commit 9d5b5d7ddd)

* Fix var naming (#55795): Make vars match tasks in Google Compute guide.

(cherry picked from commit 943f7334c5)

* Clarifies how Ansible processes multiple `failed_when` conditions (#55941): multiple failed_when conditions join with AND not OR to counter third-party pages online incorrectly stating that it uses `OR`. ([example](https://groups.google.com/d/msg/ansible-project/cIaQTmY3ZLE/c5w8rlmdHWIJ)).

(cherry picked from commit 5439eb8bd8)

* Docs: edits & expands module_utils & search path info in dev guide (#55931)

(cherry picked from commit 8542459b95)

* Add faq note about ssh ServerAliveInterval (#55568)

(cherry picked from commit 76dba7aa4f)

* docsite: correct path, list requirements for testing module docs, etc. (#52008)

* dev_guide: correct path, list requirements, etc.; module HTML docs are in '_build/html/module' subdir

(cherry picked from commit b14f477bee)

* Developer documentation update involving module invocation (#55747)

* Update docs for the 2.7 change to AnsiballZ which invokes modules with one
  less Python interpreter

* Add a section on how module results are returned and on trust between modules, action plugins, and the executor.

* Update docs/docsite/rst/dev_guide/developing_program_flow_modules.rst

Co-Authored-By: abadger <a.badger@gmail.com>
(cherry picked from commit edafa71f42)

* add doc example of multiline failed_when with OR (#56007)
* add variety to multiple OR failed_when doc example

(cherry picked from commit 7d5ada7161)

* Note that by default the regex test is identical to match, but can do much more (#50205)

* Note that the regex test behaves like 'match', with default settings

(cherry picked from commit 86e98c5213)

* more info on how vaults work (#56183)

also add warning about what it covers.

(cherry picked from commit 8ff27c4e0c)

* Fix var naming in GCE guide

(cherry picked from commit dae5564e2b)

* dev_guide: Various small updates (#53273)

* Document the clarifications that I usually remark when doing reviews
* Update docs/docsite/rst/dev_guide/developing_modules_documenting.rst

Co-Authored-By: dagwieers <dag@wieers.com>
(cherry picked from commit eac7f1fb58)

* Lack of "--update" flag in older Ubuntu distros (#56283): when installing on older Ubuntu distributions be aware of the lack of ``-u`` or ``--update`` flag.

(cherry picked from commit dd0b0ae47b)

* should have gone into 52373 (#56306)

(cherry picked from commit 3c8d8b1509)
This commit is contained in:
Alicia Cozine 2019-05-13 08:22:20 -05:00 committed by GitHub
parent d69da31ceb
commit c05b61777c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
20 changed files with 595 additions and 364 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/docsite/rst/dev_guide/developing_locally.rst
View file

@ -5,7 +5,7 @@
Adding modules and plugins locally
**********************************
.. contents:: Topics
.. contents::
:local:
The easiest, quickest, and most popular way to extend Ansible is to copy or write a module or a plugin for local use. You can store local modules and plugins on your Ansible control node for use within your team or organization. You can also share a local plugin or module by embedding it in a role and publishing it on Ansible Galaxy. If you've been using roles off Galaxy, you may have been using local modules and plugins without even realizing it. If you're using a local module or plugin that already exists, this page is all you need.
@ -26,8 +26,8 @@ Modules and plugins: what's the difference?
===========================================
If you're looking to add local functionality to Ansible, you may be wondering whether you need a module or a plugin. Here's a quick overview of the differences:
* Modules are reusable, standalone scripts that can be used by the Ansible API, the :command:`ansible` command, or the :command:`ansible-playbook` command. Modules provide a defined interface, accepting arguments and returning information to Ansible by printing a JSON string to stdout before exiting.
* Plugins are shared code that can be used by any module. They provide abilities like caching information or copying files that are useful for many modules.
* Modules are reusable, standalone scripts that can be used by the Ansible API, the :command:`ansible` command, or the :command:`ansible-playbook` command. Modules provide a defined interface, accepting arguments and returning information to Ansible by printing a JSON string to stdout before exiting. Modules execute on the target system (usually that means on a remote system) in separate processes.
* :ref:`Plugins <plugins_lookup>` augment Ansible's core functionality and execute on the control node within the ``/usr/bin/ansible`` process. Plugins offer options and extensions for the core features of Ansible - transforming data, logging output, connecting to inventory, and more.
.. _local_modules:

140
docs/docsite/rst/dev_guide/developing_module_utilities.rst
View file

@ -1,87 +1,77 @@
.. _appendix_module_utilities:
.. _developing_module_utilities:
**************************
Appendix: Module Utilities
**************************
*************************************
Using and Developing Module Utilities
*************************************
Ansible provides a number of module utilities that provide helper functions that you can use when developing your own modules. The ``basic.py`` module utility provides the main entry point for accessing the Ansible library, and all Python Ansible modules must, at minimum, import ``AnsibleModule``::
Ansible provides a number of module utilities, or snippets of shared code, that
provide helper functions you can use when developing your own modules. The
``basic.py`` module utility provides the main entry point for accessing the
Ansible library, and all Python Ansible modules must import something from
``ansible.module_utils``. A common option is to import ``AnsibleModule``::
from ansible.module_utils.basic import AnsibleModule
If you need to share Python code between some of your own local modules, you can use Ansible's ``module_utils`` directories for this. When you run ``ansible-playbook``, Ansible will merge any files in the local ``module_utils`` directory into the ``ansible.module_utils`` namespace. For example, if you have your own custom modules that import a ``my_shared_code`` library, you can place that into a ``./module_utils/my_shared_code.py`` file in the root location where your playbook lives, and then import it in your modules like so::
The ``ansible.module_utils`` namespace is not a plain Python package: it is
constructed dynamically for each task invocation, by extracting imports and
resolving those matching the namespace against a :ref:`search path <ansible_search_path>` derived from the
active configuration.
To reduce the maintenance burden on your own local modules, you can extract
duplicated code into one or more module utilities and import them into your modules. For example, if you have your own custom modules that import a ``my_shared_code`` library, you can place that into a ``./module_utils/my_shared_code.py`` file like this::
from ansible.module_utils.my_shared_code import MySharedCodeClient
Your custom ``module_utils`` directories can live in the root directory of your playbook, or in the individual role directories, or in the directories specified by the ``ANSIBLE_MODULE_UTILS`` configuration setting.
When you run ``ansible-playbook``, Ansible will merge any files in your local ``module_utils`` directories into the ``ansible.module_utils`` namespace in the order defined by the :ref:`Ansible search path <ansible_search_path>`.
Ansible ships with the following list of ``module_utils`` files. The module utility source code lives in the ``./lib/ansible/module_utils`` directory under your main Ansible path - for more details on any specific module utility, please see the source code.
Naming and finding module utilities
===================================
You can generally tell what a module utility does from its name and/or its location. For example, ``openstack.py`` contains utilities for modules that work with OpenStack instances.
Generic utilities (shared code used by many different kinds of modules) live in the ``common`` subdirectory or in the root directory. Utilities
used by a particular set of modules generally live in a sub-directory that mirrors
the directory for those modules. For example:
* ``lib/ansible/module_utils/urls.py`` contains shared code for parsing URLs
* ``lib/ansible/module_utils/storage/emc/`` contains shared code related to EMC
* ``lib/ansible/modules/storage/emc/`` contains modules related to EMC
Following this pattern with your own module utilities makes everything easy to find and use.
.. _standard_mod_utils:
Standard module utilities
=========================
Ansible ships with an extensive library of ``module_utils`` files.
You can find the module
utility source code in the ``lib/ansible/module_utils`` directory under
your main Ansible path. We've described the most widely used utilities below. For more details on any specific module utility,
please see the `source code for module_utils <https://github.com/ansible/ansible/tree/devel/lib/ansible/module_utils>`_.
.. include:: shared_snippets/licensing.txt
- alicloud_ecs.py - Definitions and utilities for modules working with Alibaba Cloud ECS.
- api.py - Adds shared support for generic API modules.
- azure_rm_common.py - Definitions and utilities for Microsoft Azure Resource Manager template deployments.
- basic.py - General definitions and helper utilities for Ansible modules.
- cloudstack.py - Utilities for CloudStack modules.
- database.py - Miscellaneous helper functions for PostGRES and MySQL
- docker_common.py - Definitions and helper utilities for modules working with Docker.
- ec2.py - Definitions and utilities for modules working with Amazon EC2
- facts/- Folder containing helper functions for modules that return facts. See https://github.com/ansible/ansible/pull/23012 for more information.
- gce.py - Definitions and helper functions for modules that work with Google Compute Engine resources.
- ismount.py - Contains single helper function that fixes os.path.ismount
- keycloak.py - Definitions and helper functions for modules working with the Keycloak API
- known_hosts.py - utilities for working with known_hosts file
- manageiq.py - Functions and utilities for modules that work with ManageIQ platform and its resources.
- memset.py - Helper functions and utilities for interacting with Memset's API.
- mysql.py - Allows modules to connect to a MySQL instance
- netapp.py - Functions and utilities for modules that work with the NetApp storage platforms.
- network/a10/a10.py - Utilities used by the a10_server module to manage A10 Networks devices.
- network/aci/aci.py - Definitions and helper functions for modules that manage Cisco ACI Fabrics.
- network/aireos/aireos.py - Definitions and helper functions for modules that manage Cisco WLC devices.
- network/aos/aos.py - Module support utilities for managing Apstra AOS Server.
- network/aruba/aruba.py - Helper functions for modules working with Aruba networking devices.
- network/asa/asa.py - Module support utilities for managing Cisco ASA network devices.
- network/avi/avi.py - Helper functions for modules working with AVI networking devices.
- network/bigswitch/bigswitch_utils.py - Utilities used by the bigswitch module to manage Big Switch Networks devices.
- network/cloudengine/ce.py - Module support utilities for managing Huawei Cloudengine switch.
- network/cnos/cnos.py - Helper functions for modules working on devices running Lenovo CNOS.
- network/common/config.py - Configuration utility functions for use by networking modules
- network/common/netconf.py - Definitions and helper functions for modules that use Netconf transport.
- network/common/parsing.py - Definitions and helper functions for Network modules.
- network/common/network.py - Functions for running commands on networking devices
- network/common/utils.py - Defines commands and comparison operators and other utilises for use in networking modules
- network/dellos6/dellos6.py - Module support utilities for managing device running Dell OS6.
- network/dellos9/dellos9.py - Module support utilities for managing device running Dell OS9.
- network/dellos10/dellos10.py - Module support utilities for managing device running Dell OS10.
- network/enos/enos.py - Helper functions for modules working with Lenovo ENOS devices.
- network/eos/eos.py - Helper functions for modules working with EOS networking devices.
- network/fortios/fortios.py - Module support utilities for managing FortiOS devices.
- network/ios/ios.py - Definitions and helper functions for modules that manage Cisco IOS networking devices
- network/iosxr/iosxr.py - Definitions and helper functions for modules that manage Cisco IOS-XR networking devices.
- network/ironware/ironware.py - Module support utilities for managing Brocade IronWare devices.
- network/junos/junos.py - Definitions and helper functions for modules that manage Junos networking devices.
- network/meraki/meraki.py - Utilities specifically for the Meraki network modules.
- network/netscaler/netscaler.py - Utilities specifically for the netscaler network modules.
- network/nso/nso.py - Utilities for modules that work with Cisco NSO.
- network/nxos/nxos.py - Contains definitions and helper functions specific to Cisco NXOS networking devices.
- network/onyx/onyx.py - Definitions and helper functions for modules that manage Mellanox ONYX networking devices.
- network/ordance/ordance.py - Module support utilities for managing Ordnance devices.
- network/sros/sros.py - Helper functions for modules working with Open vSwitch bridges.
- network/vyos/vyos.py - Definitions and functions for working with VyOS networking
- openstack.py - Utilities for modules that work with Openstack instances.
- openswitch.py - Definitions and helper functions for modules that manage OpenSwitch devices
- powershell.ps1 - Utilities for working with Microsoft Windows clients
- pure.py - Functions and utilities for modules that work with the Pure Storage storage platforms.
- pycompat24.py - Exception workaround for Python 2.4.
- rax.py - Definitions and helper functions for modules that work with Rackspace resources.
- redhat.py - Functions for modules that manage Red Hat Network registration and subscriptions
- service.py - Contains utilities to enable modules to work with Linux services (placeholder, not in use).
- shell.py - Functions to allow modules to create shells and work with shell commands
- six/__init__.py - Bundled copy of the `Six Python library <https://pythonhosted.org/six/>`_ to aid in writing code compatible with both Python 2 and Python 3.
- splitter.py - String splitting and manipulation utilities for working with Jinja2 templates
- urls.py - Utilities for working with http and https requests
- utm_utils.py - Contains base class for creating new Sophos UTM Modules and helper functions for handling the rest interface of Sophos UTM
- vca.py - Contains utilities for modules that work with VMware vCloud Air
- vexata.py - Utilities for modules that work with Vexata storage platforms.
- vmware.py - Contains utilities for modules that work with VMware vSphere VMs
- xenserver.py - Contains utilities for modules that work with XenServer.
- ``api.py`` - Supports generic API modules
- ``basic.py`` - General definitions and helper utilities for Ansible modules
- ``common/dict_transformations.py`` - Helper functions for dictionary transformations
- ``common/file.py`` - Helper functions for working with files
- ``common/text/`` - Helper functions for converting and formatting text.
- ``common/parameters.py`` - Helper functions for dealing with module parameters
- ``common/sys_info.py`` - Functions for getting distribution and platform information
- ``common/validation.py`` - Helper functions for validating module parameters against a module argument spec
- ``facts/`` - Directory of utilities for modules that return facts. See `PR 23012 <https://github.com/ansible/ansible/pull/23012>`_ for more information
- ``ismount.py`` - Single helper function that fixes os.path.ismount
- ``json_utils.py`` - Utilities for filtering unrelated output around module JSON output, like leading and trailing lines
- ``known_hosts.py`` - utilities for working with known_hosts file
- ``network/common/config.py`` - Configuration utility functions for use by networking modules
- ``network/common/netconf.py`` - Definitions and helper functions for modules that use Netconf transport
- ``network/common/parsing.py`` - Definitions and helper functions for Network modules
- ``network/common/network.py`` - Functions for running commands on networking devices
- ``network/common/utils.py`` - Defines commands and comparison operators and other utilises for use in networking modules
- ``powershell/`` - Directory of definitions and helper functions for Windows PowerShell modules
- ``pycompat24.py`` - Exception workaround for Python 2.4
- ``service.py`` - Utilities to enable modules to work with Linux services (placeholder, not in use)
- ``shell.py`` - Functions to allow modules to create shells and work with shell commands
- ``six/__init__.py`` - Bundled copy of the `Six Python library <https://pythonhosted.org/six/>`_ to aid in writing code compatible with both Python 2 and Python 3
- ``splitter.py`` - String splitting and manipulation utilities for working with Jinja2 templates
- ``urls.py`` - Utilities for working with http and https requests

2
docs/docsite/rst/dev_guide/developing_modules_best_practices.rst
View file

@ -67,7 +67,7 @@ Python tips
Importing and using shared code
===============================
* Use shared code whenever possible - don't reinvent the wheel. Ansible offers the ``AnsibleModule`` common Python code, plus :ref:`utilities <appendix_module_utilities>` for many common use cases and patterns.
* Use shared code whenever possible - don't reinvent the wheel. Ansible offers the ``AnsibleModule`` common Python code, plus :ref:`utilities <developing_module_utilities>` for many common use cases and patterns. You can also create documentation fragments for docs that apply to multiple modules.
* Import ``ansible.module_utils`` code in the same place as you import other libraries.
* Do NOT use wildcards (*) for importing other python modules; instead, list the function(s) you are importing (for example, ``from some.other_python_module.basic import otherFunction``).
* Import custom packages in ``try``/``except``, capture any import errors, and handle them with ``fail_json()`` in ``main()``. For example:

95
docs/docsite/rst/dev_guide/developing_modules_documenting.rst
View file

@ -23,21 +23,23 @@ Every Ansible module written in Python must begin with seven standard sections i
.. _shebang:
Python shebang
==============
Python shebang & UTF-8 coding
===============================
Every Ansible module must begin with ``#!/usr/bin/python`` - this "shebang" allows ``ansible_python_interpreter`` to work.
This is immediately followed by ``# -*- coding: utf-8 -*-`` to clarify that the file is UTF-8 encoded.
.. _copyright:
Copyright and license
=====================
After the shebang, there should be a `copyright line <https://www.gnu.org/licenses/gpl-howto.en.html>`_ with the original copyright holder and a license declaration. The license declaration should be ONLY one line, not the full GPL prefix.:
After the shebang and UTF-8 coding, there should be a `copyright line <https://www.gnu.org/licenses/gpl-howto.en.html>`_ with the original copyright holder and a license declaration. The license declaration should be ONLY one line, not the full GPL prefix.:
.. code-block:: python
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright: (c) 2018, Terry Jones <terry.jones@example.org>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
@ -47,6 +49,7 @@ Major additions to the module (for instance, rewrites) may add additional copyri
.. code-block:: python
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright: (c) 2017, [New Contributor(s)]
# Copyright: (c) 2015, [Original Contributor(s)]
@ -57,7 +60,7 @@ Major additions to the module (for instance, rewrites) may add additional copyri
ANSIBLE_METADATA block
======================
After the shebang, the copyright, and the license, your module file should contain an ``ANSIBLE_METADATA`` section. This section provides information about the module for use by other tools. For new modules, the following block can be simply added into your module:
After the shebang, the UTF-8 coding, the copyright, and the license, your module file should contain an ``ANSIBLE_METADATA`` section. This section provides information about the module for use by other tools. For new modules, the following block can be simply added into your module:
.. code-block:: python
@ -96,10 +99,10 @@ Ansible metadata fields
The default value is a single element list ["preview"]. The following strings are valid
statuses and have the following meanings:
:stableinterface: The module's parameters are stable. Every effort will be made not to remove parameters or change
:stableinterface: The module's options (the parameters or arguments it accepts) are stable. Every effort will be made not to remove options or change
their meaning. **Not** a rating of the module's code quality.
:preview: The module is in tech preview. It may be
unstable, the parameters may change, or it may require libraries or
unstable, the options may change, or it may require libraries or
web services that are themselves subject to incompatible changes.
:deprecated: The module is deprecated and will be removed in a future release.
:removed: The module is not present in the release. A stub is
@ -111,13 +114,13 @@ Ansible metadata fields
DOCUMENTATION block
===================
After the shebang, the copyright line, the license, and the ``ANSIBLE_METADATA`` section comes the ``DOCUMENTATION`` block. Ansible's online module documentation is generated from the ``DOCUMENTATION`` blocks in each module's source code. The ``DOCUMENTATION`` block must be valid YAML. You may find it easier to start writing your ``DOCUMENTATION`` string in an :ref:`editor with YAML syntax highlighting <other_tools_and_programs>` before you include it in your Python file. You can start by copying our `example documentation string <https://github.com/ansible/ansible/blob/devel/examples/DOCUMENTATION.yml>`_ into your module file and modifying it. If you run into syntax issues in your YAML, you can validate it on the `YAML Lint <http://www.yamllint.com/>`_ website.
After the shebang, the UTF-8 coding, the copyright line, the license, and the ``ANSIBLE_METADATA`` section comes the ``DOCUMENTATION`` block. Ansible's online module documentation is generated from the ``DOCUMENTATION`` blocks in each module's source code. The ``DOCUMENTATION`` block must be valid YAML. You may find it easier to start writing your ``DOCUMENTATION`` string in an :ref:`editor with YAML syntax highlighting <other_tools_and_programs>` before you include it in your Python file. You can start by copying our `example documentation string <https://github.com/ansible/ansible/blob/devel/examples/DOCUMENTATION.yml>`_ into your module file and modifying it. If you run into syntax issues in your YAML, you can validate it on the `YAML Lint <http://www.yamllint.com/>`_ website.
Module documentation should briefly and accurately define what each module and option does, and how it works with others in the underlying system. Documentation should be written for broad audience--readable both by experts and non-experts.
* Descriptions should always start with a capital letter and end with a full stop. Consistency always helps.
* Verify that arguments in doc and module spec dict are identical.
* For password / secret arguments no_log=True should be set.
* If an optional parameter is sometimes required, reflect this fact in the documentation, e.g. "Required when I(state=present)."
* If an option is only sometimes required, describe the conditions. For example, "Required when I(state=present)."
* If your module allows ``check_mode``, reflect this fact in the documentation.
Each documentation field is described below. Before committing your module documentation, please test it at the command line and as HTML:
@ -147,16 +150,19 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
* A detailed description (generally two or more sentences).
* Must be written in full sentences, i.e. with capital letters and periods/full stops.
* Shouldn't mention the module name.
* Make use of multiple entries rather than using one long paragraph.
* Don't quote complete values unless it is required by YAML.
:version_added:
* The version of Ansible when the module was added.
* This is a string, and not a float, i.e. ``version_added: "2.1"``
* This is a string, and not a float, i.e. ``version_added: '2.1'``
:author:
* Name of the module author in the form ``First Last (@GitHubID)``.
* Use a multi-line list if there is more than one author.
* Don't use quotes as it should not be required by YAML.
:deprecated:
@ -164,19 +170,22 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
:options:
* Options are often called `parameters` or `arguments`. Because the documentation field is called `options`, we will use that term.
* If the module has no options (for example, it's a ``_facts`` module), all you need is one line: ``options: {}``.
* If your module has options (in other words, accepts arguments), each option should be documented thoroughly. For each module argument/option, include:
* If your module has options (in other words, accepts arguments), each option should be documented thoroughly. For each module option, include:
:option-name:
* Declarative operation (not CRUD), to focus on the final state, for example `online:`, rather than `is_online:`.
* The name of the option should be consistent with the rest of the module, as well as other modules in the same category.
* When in doubt, look for other modules to find option names that are used for the same purpose, we like to offer consistency to our users.
:description:
* Detailed explanation of what this option does. It should be written in full sentences.
* Should not list the possible values (that's what ``choices:`` is for, though it should explain `what` the values do if they aren't obvious).
* If an optional parameter is sometimes required this need to be reflected in the documentation, e.g. "Required when I(state=present)."
* The first entry is a description of the option itself; subsequent entries detail its use, dependencies, or format of possible values.
* Should not list the possible values (that's what ``choices:`` is for, though it should explain what the values do if they aren't obvious).
* If an option is only sometimes required, describe the conditions. For example, "Required when I(state=present)."
* Mutually exclusive options must be documented as the final sentence on each of the options.
:required:
@ -187,8 +196,8 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
:default:
* If ``required`` is false/missing, ``default`` may be specified (assumed 'null' if missing).
* Ensure that the default parameter in the docs matches the default parameter in the code.
* The default option must not be listed as part of the description.
* Ensure that the default value in the docs matches the default value in the code.
* The default field must not be listed as part of the description, unless it requires additional information or conditions.
* If the option is a boolean value, you can use any of the boolean values recognized by Ansible:
(such as true/false or yes/no). Choose the one that reads better in the context of the option.
@ -209,11 +218,11 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
:version_added:
* Only needed if this option was extended after initial Ansible release, i.e. this is greater than the top level `version_added` field.
* This is a string, and not a float, i.e. ``version_added: "2.3"``.
* This is a string, and not a float, i.e. ``version_added: '2.3'``.
:suboptions:
* If this option takes a dict, you can define it here.
* If this option takes a dict, you can define its structure here.
* See :ref:`azure_rm_securitygroup_module`, :ref:`os_ironic_node_module` for examples.
:requirements:
@ -221,11 +230,6 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
* List of requirements (if applicable).
* Include minimum versions.
:notes:
* Details of any important information that doesn't fit in one of the above sections.
* For example, whether ``check_mode`` is or is not supported.
:seealso:
* A list of references to other modules, documentation or Internet resources
@ -252,6 +256,11 @@ All fields in the ``DOCUMENTATION`` block are lower-case. All fields are require
description: Complete reference of the APIC object model.
link: https://developer.cisco.com/docs/apic-mim-ref/
:notes:
* Details of any important information that doesn't fit in one of the above sections.
* For example, whether ``check_mode`` is or is not supported.
Linking within module documentation
-----------------------------------
@ -272,22 +281,22 @@ You can link from your module documentation to other module docs, other resource
Documentation fragments
-----------------------
If you're writing multiple related modules, they may share common documentation, such as authentication details or file mode settings. Rather than duplicate that information in each module's ``DOCUMENTATION`` block, you can save it once as a doc_fragment plugin and use it in each module's documentation. In Ansible, shared documentation fragments are contained in a ``ModuleDocFragment`` class in `lib/ansible/plugins/doc_fragments/ <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/doc_fragments>`_. To include a documentation fragment, add ``extends_documentation_fragment: FRAGMENT_NAME`` in your module's documentation.
If you're writing multiple related modules, they may share common documentation, such as authentication details, file mode settings, ``notes:`` or ``seealso:`` entries. Rather than duplicate that information in each module's ``DOCUMENTATION`` block, you can save it once as a doc_fragment plugin and use it in each module's documentation. In Ansible, shared documentation fragments are contained in a ``ModuleDocFragment`` class in `lib/ansible/plugins/doc_fragments/ <https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins/doc_fragments>`_. To include a documentation fragment, add ``extends_documentation_fragment: FRAGMENT_NAME`` in your module's documentation.
.. _note:
* in 2.8 the Ansible directories for doc fragments changed, see documentation of previous versions to find the old locations.
* Prior to Ansible 2.8, documentation fragments were kept in ``lib/ansible/utils/module_docs_fragments``.
.. versionadded:: 2.8
Since version 2.8, you can have user supplied doc_fragments by using a ``doc_fragments`` directory adjacent to play or role, just like any other plugin.
Since Ansible 2.8, you can have user-supplied doc_fragments by using a ``doc_fragments`` directory adjacent to play or role, just like any other plugin.
For example, all AWS modules should include:
.. code-block:: yaml+jinja
extends_documentation_fragment:
- aws
- ec2
- aws
- ec2
You can find more examples by searching for ``extends_documentation_fragment`` under the Ansible source tree.
@ -297,17 +306,21 @@ You can find more examples by searching for ``extends_documentation_fragment`` u
EXAMPLES block
==============
After the shebang, the copyright line, the license, the ``ANSIBLE_METADATA`` section, and the ``DOCUMENTATION`` block comes the ``EXAMPLES`` block. Here you show users how your module works with real-world examples in multi-line plain-text YAML format. The best examples are ready for the user to copy and paste into a playbook. Review and update your examples with every change to your module.
After the shebang, the UTF-8 coding, the copyright line, the license, the ``ANSIBLE_METADATA`` section, and the ``DOCUMENTATION`` block comes the ``EXAMPLES`` block. Here you show users how your module works with real-world examples in multi-line plain-text YAML format. The best examples are ready for the user to copy and paste into a playbook. Review and update your examples with every change to your module.
Per playbook best practices, each example should include a ``name:`` line::
EXAMPLES = '''
EXAMPLES = r'''
- name: Ensure foo is installed
modulename:
name: foo
state: present
'''
The ``name:`` line should be capitalized and not include a trailing dot.
If your examples use boolean options, use yes/no values. Since the documentation generates boolean values as yes/no, having the examples use these values as well makes the module documentation more consistent.
If your module returns facts that are often needed, an example of how to use them can be helpful.
.. _return_block:
@ -315,16 +328,16 @@ If your module returns facts that are often needed, an example of how to use the
RETURN block
============
After the shebang, the copyright line, the license, the ``ANSIBLE_METADATA`` section, ``DOCUMENTATION`` and ``EXAMPLES`` blocks comes the ``RETURN`` block. This section documents the information the module returns for use by other modules.
After the shebang, the UTF-8 coding, the copyright line, the license, the ``ANSIBLE_METADATA`` section, ``DOCUMENTATION`` and ``EXAMPLES`` blocks comes the ``RETURN`` block. This section documents the information the module returns for use by other modules.
If your module doesn't return anything (apart from the standard returns), this section of your module should read: ``RETURN = ''' # '''``
If your module doesn't return anything (apart from the standard returns), this section of your module should read: ``RETURN = r''' # '''``
Otherwise, for each value returned, provide the following fields. All fields are required unless specified otherwise.
:return name:
Name of the returned field.
:description:
Detailed description of what this value represents.
Detailed description of what this value represents. Capitalized and with trailing dot.
:returned:
When this value is returned, such as ``always``, or ``on success``.
:type:
@ -333,31 +346,31 @@ Otherwise, for each value returned, provide the following fields. All fields are
One or more examples.
:version_added:
Only needed if this return was extended after initial Ansible release, i.e. this is greater than the top level `version_added` field.
This is a string, and not a float, i.e. ``version_added: "2.3"``.
This is a string, and not a float, i.e. ``version_added: '2.3'``.
:contains:
Optional. To describe nested return values, set ``type: complex`` and repeat the elements above for each sub-field.
Here are two example ``RETURN`` sections, one with three simple fields and one with a complex nested field::
RETURN = '''
RETURN = r'''
dest:
description: destination file/path
description: Destination file/path.
returned: success
type: string
type: str
sample: /path/to/file.txt
src:
description: source file used for the copy on the target machine
description: Source file used for the copy on the target machine.
returned: changed
type: string
type: str
sample: /home/httpd/.ansible/tmp/ansible-tmp-1423796390.97-147729857856000/source
md5sum:
description: md5 checksum of the file after running copy
description: MD5 checksum of the file after running copy.
returned: when supported
type: string
type: str
sample: 2a5aeecc61dc98c4d780b14b330e3282
'''
RETURN = '''
RETURN = r'''
packages:
description: Information about package requirements
returned: On success
@ -385,7 +398,7 @@ Here are two example ``RETURN`` sections, one with three simple fields and one w
Python imports
==============
After the shebang, the copyright line, the license, and the sections for ``ANSIBLE_METADATA``, ``DOCUMENTATION``, ``EXAMPLES``, and ``RETURN``, you can finally add the python imports. All modules must use Python imports in the form:
After the shebang, the UTF-8 coding, the copyright line, the license, and the sections for ``ANSIBLE_METADATA``, ``DOCUMENTATION``, ``EXAMPLES``, and ``RETURN``, you can finally add the python imports. All modules must use Python imports in the form:
.. code-block:: python

1
docs/docsite/rst/dev_guide/developing_modules_in_groups.rst
View file

@ -91,7 +91,6 @@ The first PR should include the following files:
* ``lib/ansible/modules/$category/$topic/$yourfirstmodule.py`` - A single module. *Required new file*
* ``lib/ansible/plugins/doc_fragments/$topic.py`` - Code documentation, such as details regarding common arguments. *Optional new file*
* ``lib/ansible/module_utils/$topic.py`` - Code shared between more than one module, such as common arguments. *Optional new file*
* ``docs/docsite/rst/dev_guide/developing_module_utilities.rst`` - Document your new `module_utils` file. *Optional update to existing file*
And that's it.

361
docs/docsite/rst/dev_guide/developing_program_flow_modules.rst
View file

@ -5,13 +5,9 @@
Ansible module architecture
***************************
This in-depth dive helps you understand Ansible's program flow to execute
modules. It is written for people working on the portions of the Core Ansible
Engine that execute a module. Those writing Ansible Modules may also find this
in-depth dive to be of interest, but individuals simply using Ansible Modules
will not likely find this to be helpful.
If you're working on Ansible's Core code, writing an Ansible module, or developing an action plugin, this deep dive helps you understand how Ansible's program flow executes. If you're just using Ansible Modules in playbooks, you can skip this section.
.. contents:: Topics
.. contents::
:local:
.. _flow_types_of_modules:
@ -19,7 +15,7 @@ will not likely find this to be helpful.
Types of modules
================
Ansible supports several different types of modules in its code base. Some of
Ansible supports several different types of modules in its code base. Some of
these are for backwards compatibility and others are to enable flexibility.
.. _flow_action_plugins:
@ -27,20 +23,18 @@ these are for backwards compatibility and others are to enable flexibility.
Action plugins
--------------
Action Plugins look like modules to end users who are writing :term:`playbooks` but
they're distinct entities for the purposes of this document. Action Plugins
always execute on the controller and are sometimes able to do all work there
(for instance, the ``debug`` Action Plugin which prints some text for the user to
see or the ``assert`` Action Plugin which can test whether several values in
a playbook satisfy certain criteria.)
Action plugins look like modules to anyone writing a playbook. Usage documentation for most action plugins lives inside a module of the same name. Some action plugins do all the work, with the module providing only documentation. Some action plugins execute modules. The ``normal`` action plugin executes modules that don't have special action plugins. Action plugins always execute on the controller.
More often, Action Plugins set up some values on the controller, then invoke an
actual module on the managed node that does something with these values. An
easy to understand version of this is the :ref:`template Action Plugin
<template_module>`. The :ref:`template Action Plugin <template_module>` takes values from
Some action plugins do all their work on the controller. For
example, the :ref:`debug <debug_module>` action plugin (which prints text for
the user to see) and the :ref:`assert <assert_module>` action plugin (which
tests whether values in a playbook satisfy certain criteria) execute entirely on the controller.
Most action plugins set up some values on the controller, then invoke an
actual module on the managed node that does something with these values. For example, the :ref:`template <template_module>` action plugin takes values from
the user to construct a file in a temporary location on the controller using
variables from the playbook environment. It then transfers the temporary file
to a temporary file on the remote system. After that, it invokes the
variables from the playbook environment. It then transfers the temporary file
to a temporary file on the remote system. After that, it invokes the
:ref:`copy module <copy_module>` which operates on the remote system to move the file
into its final location, sets file permissions, and so on.
@ -49,23 +43,20 @@ into its final location, sets file permissions, and so on.
New-style modules
-----------------
All of the modules that ship with Ansible fall into this category.
All of the modules that ship with Ansible fall into this category. While you can write modules in any language, all official modules (shipped with Ansible) use either Python or PowerShell.
New-style modules have the arguments to the module embedded inside of them in
some manner. Non-new-style modules must copy a separate file over to the
some manner. Old-style modules must copy a separate file over to the
managed node, which is less efficient as it requires two over-the-wire
connections instead of only one.
.. _flow_python_modules:
Python
------
^^^^^^
New-style Python modules use the :ref:`Ansiballz` framework for constructing
modules. All official modules (shipped with Ansible) use either this or the
:ref:`powershell module framework <flow_powershell_modules>`.
These modules use imports from :code:`ansible.module_utils` in order to pull in
modules. These modules use imports from :code:`ansible.module_utils` to pull in
boilerplate module code, such as argument parsing, formatting of return
values as :term:`JSON`, and various file operations.
@ -76,21 +67,21 @@ values as :term:`JSON`, and various file operations.
.. _flow_powershell_modules:
Powershell
----------
PowerShell
^^^^^^^^^^
New-style powershell modules use the :ref:`module_replacer` framework for
constructing modules. These modules get a library of powershell code embedded
New-style PowerShell modules use the :ref:`module_replacer` framework for
constructing modules. These modules get a library of PowerShell code embedded
in them before being sent to the managed node.
.. _flow_jsonargs_modules:
JSONARGS
--------
JSONARGS modules
----------------
Scripts can arrange for an argument string to be placed within them by placing
the string ``<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>`` somewhere inside of the
file. The module typically sets a variable to that value like this:
These modules are scripts that include the string
``<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>`` in their body.
This string is replaced with the JSON-formatted argument string. These modules typically set a variable to that value like this:
.. code-block:: python
@ -102,20 +93,20 @@ Which is expanded as:
json_arguments = """{"param1": "test's quotes", "param2": "\"To be or not to be\" - Hamlet"}"""
.. note:: Ansible outputs a :term:`JSON` string with bare quotes. Double quotes are
.. note:: Ansible outputs a :term:`JSON` string with bare quotes. Double quotes are
used to quote string values, double quotes inside of string values are
backslash escaped, and single quotes may appear unescaped inside of
a string value. To use JSONARGS, your scripting language must have a way
to handle this type of string. The example uses Python's triple quoted
strings to do this. Other scripting languages may have a similar quote
a string value. To use JSONARGS, your scripting language must have a way
to handle this type of string. The example uses Python's triple quoted
strings to do this. Other scripting languages may have a similar quote
character that won't be confused by any quotes in the JSON or it may
allow you to define your own start-of-quote and end-of-quote characters.
If the language doesn't give you any of these then you'll need to write
a :ref:`non-native JSON module <flow_want_json_modules>` or
:ref:`Old-style module <flow_old_style_modules>` instead.
The module typically parses the contents of ``json_arguments`` using a JSON
library and then use them as native variables throughout the rest of its code.
These modules typically parse the contents of ``json_arguments`` using a JSON
library and then use them as native variables throughout the code.
.. _flow_want_json_modules:
@ -124,12 +115,12 @@ Non-native want JSON modules
If a module has the string ``WANT_JSON`` in it anywhere, Ansible treats
it as a non-native module that accepts a filename as its only command line
parameter. The filename is for a temporary file containing a :term:`JSON`
string containing the module's parameters. The module needs to open the file,
parameter. The filename is for a temporary file containing a :term:`JSON`
string containing the module's parameters. The module needs to open the file,
read and parse the parameters, operate on the data, and print its return data
as a JSON encoded dictionary to stdout before exiting.
These types of modules are self-contained entities. As of Ansible 2.1, Ansible
These types of modules are self-contained entities. As of Ansible 2.1, Ansible
only modifies them to change a shebang line if present.
.. seealso:: Examples of Non-native modules written in ruby are in the `Ansible
@ -140,14 +131,14 @@ only modifies them to change a shebang line if present.
Binary modules
--------------
From Ansible 2.2 onwards, modules may also be small binary programs. Ansible
From Ansible 2.2 onwards, modules may also be small binary programs. Ansible
doesn't perform any magic to make these portable to different systems so they
may be specific to the system on which they were compiled or require other
binary runtime dependencies. Despite these drawbacks, a site may sometimes
have no choice but to compile a custom module against a specific binary
library if that's the only way they have to get access to certain resources.
binary runtime dependencies. Despite these drawbacks, you may have
to compile a custom module against a specific binary
library if that's the only way to get access to certain resources.
Binary modules take their arguments and will return data to Ansible in the same
Binary modules take their arguments and return data to Ansible in the same
way as :ref:`want JSON modules <flow_want_json_modules>`.
.. seealso:: One example of a `binary module
@ -162,10 +153,8 @@ Old-style modules
Old-style modules are similar to
:ref:`want JSON modules <flow_want_json_modules>`, except that the file that
they take contains ``key=value`` pairs for their parameters instead of
:term:`JSON`.
Ansible decides that a module is old-style when it doesn't have any of the
markers that would show that it is one of the other types.
:term:`JSON`. Ansible decides that a module is old-style when it doesn't have
any of the markers that would show that it is one of the other types.
.. _flow_how_modules_are_executed:
@ -173,8 +162,8 @@ How modules are executed
========================
When a user uses :program:`ansible` or :program:`ansible-playbook`, they
specify a task to execute. The task is usually the name of a module along
with several parameters to be passed to the module. Ansible takes these
specify a task to execute. The task is usually the name of a module along
with several parameters to be passed to the module. Ansible takes these
values and processes them in various ways before they are finally executed on
the remote machine.
@ -185,44 +174,43 @@ Executor/task_executor
The TaskExecutor receives the module name and parameters that were parsed from
the :term:`playbook <playbooks>` (or from the command line in the case of
:command:`/usr/bin/ansible`). It uses the name to decide whether it's looking
at a module or an :ref:`Action Plugin <flow_action_plugins>`. If it's
:command:`/usr/bin/ansible`). It uses the name to decide whether it's looking
at a module or an :ref:`Action Plugin <flow_action_plugins>`. If it's
a module, it loads the :ref:`Normal Action Plugin <flow_normal_action_plugin>`
and passes the name, variables, and other information about the task and play
to that Action Plugin for further processing.
.. _flow_normal_action_plugin:
Normal action plugin
--------------------
The ``normal`` action plugin
----------------------------
The ``normal`` action plugin executes the module on the remote host. It is
The ``normal`` action plugin executes the module on the remote host. It is
the primary coordinator of much of the work to actually execute the module on
the managed machine.
* It takes care of creating a connection to the managed machine by
instantiating a ``Connection`` class according to the inventory
configuration for that host.
* It adds any internal Ansible variables to the module's parameters (for
* It loads the appropriate connection plugin for the task, which then transfers
or executes as needed to create a connection to that host.
* It adds any internal Ansible properties to the module's parameters (for
instance, the ones that pass along ``no_log`` to the module).
* It takes care of creating any temporary files on the remote machine and
* It works with other plugins (connection, shell, become, other action plugins)
to create any temporary files on the remote machine and
cleans up afterwards.
* It does the actual work of pushing the module and module parameters to the
* It pushes the module and module parameters to the
remote host, although the :ref:`module_common <flow_executor_module_common>`
code described in the next section does the work of deciding which format
code described in the next section decides which format
those will take.
* It handles any special cases regarding modules (for instance, various
complications around Windows modules that must have the same names as Python
modules, so that internal calling of modules from other Action Plugins work.)
* It handles any special cases regarding modules (for instance, async
execution, or complications around Windows modules that must have the same names as Python modules, so that internal calling of modules from other Action Plugins work.)
Much of this functionality comes from the `BaseAction` class,
which lives in :file:`plugins/action/__init__.py`. It makes use of
which lives in :file:`plugins/action/__init__.py`. It uses the
``Connection`` and ``Shell`` objects to do its work.
.. note::
When :term:`tasks <tasks>` are run with the ``async:`` parameter, Ansible
uses the ``async`` Action Plugin instead of the ``normal`` Action Plugin
to invoke it. That program flow is currently not documented. Read the
to invoke it. That program flow is currently not documented. Read the
source for information on how that works.
.. _flow_executor_module_common:
@ -230,25 +218,27 @@ which lives in :file:`plugins/action/__init__.py`. It makes use of
Executor/module_common.py
-------------------------
Code in :file:`executor/module_common.py` takes care of assembling the module
to be shipped to the managed node. The module is first read in, then examined
to determine its type. :ref:`PowerShell <flow_powershell_modules>` and
:ref:`JSON-args modules <flow_jsonargs_modules>` are passed through
:ref:`Module Replacer <module_replacer>`. New-style
:ref:`Python modules <flow_python_modules>` are assembled by :ref:`Ansiballz`.
:ref:`Non-native-want-JSON <flow_want_json_modules>`,
:ref:`Binary modules <flow_binary_modules>`, and
:ref:`Old-Style modules <flow_old_style_modules>` aren't touched by either of
these and pass through unchanged. After the assembling step, one final
modification is made to all modules that have a shebang line. Ansible checks
Code in :file:`executor/module_common.py` assembles the module
to be shipped to the managed node. The module is first read in, then examined
to determine its type:
* :ref:`PowerShell <flow_powershell_modules>` and :ref:`JSON-args modules <flow_jsonargs_modules>` are passed through :ref:`Module Replacer <module_replacer>`.
* New-style :ref:`Python modules <flow_python_modules>` are assembled by :ref:`Ansiballz`.
* :ref:`Non-native-want-JSON <flow_want_json_modules>`, :ref:`Binary modules <flow_binary_modules>`, and :ref:`Old-Style modules <flow_old_style_modules>` aren't touched by either of these and pass through unchanged.
After the assembling step, one final
modification is made to all modules that have a shebang line. Ansible checks
whether the interpreter in the shebang line has a specific path configured via
an ``ansible_$X_interpreter`` inventory variable. If it does, Ansible
substitutes that path for the interpreter path given in the module. After
an ``ansible_$X_interpreter`` inventory variable. If it does, Ansible
substitutes that path for the interpreter path given in the module. After
this, Ansible returns the complete module data and the module type to the
:ref:`Normal Action <flow_normal_action_plugin>` which continues execution of
the module.
Next we'll go into some details of the two assembler frameworks.
Assembler frameworks
--------------------
Ansible supports two assembler frameworks: Ansiballz and the older Module Replacer.
.. _module_replacer:
@ -256,12 +246,12 @@ Module Replacer framework
^^^^^^^^^^^^^^^^^^^^^^^^^
The Module Replacer framework is the original framework implementing new-style
modules. It is essentially a preprocessor (like the C Preprocessor for those
familiar with that programming language). It does straight substitutions of
specific substring patterns in the module file. There are two types of
modules, and is still used for PowerShell modules. It is essentially a preprocessor (like the C Preprocessor for those
familiar with that programming language). It does straight substitutions of
specific substring patterns in the module file. There are two types of
substitutions:
* Replacements that only happen in the module file. These are public
* Replacements that only happen in the module file. These are public
replacement strings that modules can utilize to get helpful boilerplate or
access to arguments.
@ -272,12 +262,10 @@ substitutions:
:code:`from ansible.module_utils.basic import *` and should also only apply
to new-style Python modules.
- :code:`# POWERSHELL_COMMON` substitutes the contents of
:file:`ansible/module_utils/powershell.ps1`. It should only be used with
:file:`ansible/module_utils/powershell.ps1`. It should only be used with
:ref:`new-style Powershell modules <flow_powershell_modules>`.
* Replacements that are used by ``ansible.module_utils`` code. These are internal
replacement patterns. They may be used internally, in the above public
replacements, but shouldn't be used directly by modules.
* Replacements that are used by ``ansible.module_utils`` code. These are internal replacement patterns. They may be used internally, in the above public replacements, but shouldn't be used directly by modules.
- :code:`"<<ANSIBLE_VERSION>>"` is substituted with the Ansible version. In
:ref:`new-style Python modules <flow_python_modules>` under the
@ -286,29 +274,29 @@ substitutions:
:attr:``AnsibleModule.ansible_version``.
- :code:`"<<INCLUDE_ANSIBLE_MODULE_COMPLEX_ARGS>>"` is substituted with
a string which is the Python ``repr`` of the :term:`JSON` encoded module
parameters. Using ``repr`` on the JSON string makes it safe to embed in
a Python file. In new-style Python modules under the Ansiballz framework
parameters. Using ``repr`` on the JSON string makes it safe to embed in
a Python file. In new-style Python modules under the Ansiballz framework
this is better accessed by instantiating an `AnsibleModule` and
then using :attr:`AnsibleModule.params`.
- :code:`<<SELINUX_SPECIAL_FILESYSTEMS>>` substitutes a string which is
a comma separated list of file systems which have a file system dependent
security context in SELinux. In new-style Python modules, if you really
security context in SELinux. In new-style Python modules, if you really
need this you should instantiate an `AnsibleModule` and then use
:attr:`AnsibleModule._selinux_special_fs`. The variable has also changed
:attr:`AnsibleModule._selinux_special_fs`. The variable has also changed
from a comma separated string of file system names to an actual python
list of filesystem names.
- :code:`<<INCLUDE_ANSIBLE_MODULE_JSON_ARGS>>` substitutes the module
parameters as a JSON string. Care must be taken to properly quote the
string as JSON data may contain quotes. This pattern is not substituted
parameters as a JSON string. Care must be taken to properly quote the
string as JSON data may contain quotes. This pattern is not substituted
in new-style Python modules as they can get the module parameters another
way.
- The string :code:`syslog.LOG_USER` is replaced wherever it occurs with the
``syslog_facility`` which was named in :file:`ansible.cfg` or any
``ansible_syslog_facility`` inventory variable that applies to this host. In
new-style Python modules this has changed slightly. If you really need to
new-style Python modules this has changed slightly. If you really need to
access it, you should instantiate an `AnsibleModule` and then use
:attr:`AnsibleModule._syslog_facility` to access it. It is no longer the
actual syslog facility and is now the name of the syslog facility. See
:attr:`AnsibleModule._syslog_facility` to access it. It is no longer the
actual syslog facility and is now the name of the syslog facility. See
the :ref:`documentation on internal arguments <flow_internal_arguments>`
for details.
@ -317,32 +305,36 @@ substitutions:
Ansiballz framework
^^^^^^^^^^^^^^^^^^^
Ansible 2.1 switched from the :ref:`module_replacer` framework to the
Ansiballz framework for assembling modules. The Ansiballz framework differs
from module replacer in that it uses real Python imports of things in
:file:`ansible/module_utils` instead of merely preprocessing the module. It
The Ansiballz framework was adopted in Ansible 2.1 and is used for all new-style Python modules. Unlike the Module Replacer, Ansiballz uses real Python imports of things in
:file:`ansible/module_utils` instead of merely preprocessing the module. It
does this by constructing a zipfile -- which includes the module file, files
in :file:`ansible/module_utils` that are imported by the module, and some
boilerplate to pass in the module's parameters. The zipfile is then Base64
boilerplate to pass in the module's parameters. The zipfile is then Base64
encoded and wrapped in a small Python script which decodes the Base64 encoding
and places the zipfile into a temp directory on the managed node. It then
extracts just the ansible module script from the zip file and places that in
the temporary directory as well. Then it sets the PYTHONPATH to find python
modules inside of the zip file and invokes :command:`python` on the extracted
ansible module.
and places the zipfile into a temp directory on the managed node. It then
extracts just the Ansible module script from the zip file and places that in
the temporary directory as well. Then it sets the PYTHONPATH to find Python
modules inside of the zip file and imports the Ansible module as the special name, ``__main__``.
Importing it as ``__main__`` causes Python to think that it is executing a script rather than simply
importing a module. This lets Ansible run both the wrapper script and the module code in a single copy of Python on the remote machine.
.. note::
Ansible wraps the zipfile in the Python script for two reasons:
* Ansible wraps the zipfile in the Python script for two reasons:
* for compatibility with Python 2.6 which has a less
functional version of Python's ``-m`` command line switch.
* so that pipelining will function properly. Pipelining needs to pipe the
Python module into the Python interpreter on the remote node. Python
understands scripts on stdin but does not understand zip files.
* for compatibility with Python 2.6 which has a less
functional version of Python's ``-m`` command line switch.
* so that pipelining will function properly. Pipelining needs to pipe the
Python module into the Python interpreter on the remote node. Python
understands scripts on stdin but does not understand zip files.
* Prior to Ansible 2.7, the module was executed via a second Python interpreter instead of being
executed inside of the same process. This change was made once Python-2.4 support was dropped
to speed up module execution.
In Ansiballz, any imports of Python modules from the
:py:mod:`ansible.module_utils` package trigger inclusion of that Python file
into the zipfile. Instances of :code:`#<<INCLUDE_ANSIBLE_MODULE_COMMON>>` in
into the zipfile. Instances of :code:`#<<INCLUDE_ANSIBLE_MODULE_COMMON>>` in
the module are turned into :code:`from ansible.module_utils.basic import *`
and :file:`ansible/module-utils/basic.py` is then included in the zipfile.
Files that are included from :file:`module_utils` are themselves scanned for
@ -351,34 +343,34 @@ the zipfile as well.
.. warning::
At present, the Ansiballz Framework cannot determine whether an import
should be included if it is a relative import. Always use an absolute
should be included if it is a relative import. Always use an absolute
import that has :py:mod:`ansible.module_utils` in it to allow Ansiballz to
determine that the file should be included.
.. _flow_passing_module_args:
Passing args
------------
In :ref:`module_replacer`, module arguments are turned into a JSON-ified
string and substituted into the combined module file. In :ref:`Ansiballz`,
the JSON-ified string is passed into the module via stdin. When
a :class:`ansible.module_utils.basic.AnsibleModule` is instantiated,
it parses this string and places the args into
:attr:`AnsibleModule.params` where it can be accessed by the module's
other code.
Arguments are passed differently by the two frameworks:
* In :ref:`module_replacer`, module arguments are turned into a JSON-ified string and substituted into the combined module file.
* In :ref:`Ansiballz`, the JSON-ified string is part of the script which wraps the zipfile. Just before the wrapper script imports the Ansible module as ``__main__``, it monkey-patches the private, ``_ANSIBLE_ARGS`` variable in ``basic.py`` with the variable values. When a :class:`ansible.module_utils.basic.AnsibleModule` is instantiated, it parses this string and places the args into :attr:`AnsibleModule.params` where it can be accessed by the module's other code.
.. warning::
If you are writing modules, remember that the way we pass arguments is an internal implementation detail: it has changed in the past and will change again as soon as changes to the common module_utils
code allow Ansible modules to forgo using :class:`ansible.module_utils.basic.AnsibleModule`. Do not rely on the internal global ``_ANSIBLE_ARGS`` variable.
Very dynamic custom modules which need to parse arguments before they
instantiate an ``AnsibleModule`` may use ``_load_params`` to retrieve those parameters.
Although ``_load_params`` may change in breaking ways if necessary to support
changes in the code, it is likely to be more stable than either the way we pass parameters or the internal global variable.
.. note::
Internally, the `AnsibleModule` uses the helper function,
:py:func:`ansible.module_utils.basic._load_params`, to load the parameters
from stdin and save them into an internal global variable. Very dynamic
custom modules which need to parse the parameters prior to instantiating
an ``AnsibleModule`` may use ``_load_params`` to retrieve the
parameters. Be aware that ``_load_params`` is an internal function and
may change in breaking ways if necessary to support changes in the code.
However, we'll do our best not to break it gratuitously, which is not
something that can be said for either the way parameters are passed or
the internal global variable.
Prior to Ansible 2.7, the Ansible module was invoked in a second Python interpreter and the
arguments were then passed to the script over the script's stdin.
.. _flow_internal_arguments:
@ -386,93 +378,85 @@ Internal arguments
------------------
Both :ref:`module_replacer` and :ref:`Ansiballz` send additional arguments to
the module beyond those which the user specified in the playbook. These
the module beyond those which the user specified in the playbook. These
additional arguments are internal parameters that help implement global
Ansible features. Modules often do not need to know about these explicitly as
Ansible features. Modules often do not need to know about these explicitly as
the features are implemented in :py:mod:`ansible.module_utils.basic` but certain
features need support from the module so it's good to know about them.
The internal arguments listed here are global. If you need to add a local internal argument to a custom module, create an action plugin for that specific module - see ``_original_basename`` in the `copy action plugin <https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/action/copy.py#L329>`_ for an example.
_ansible_no_log
^^^^^^^^^^^^^^^
This is a boolean. If it's True then the playbook specified ``no_log`` (in
a task's parameters or as a play parameter). This automatically affects calls
to :py:meth:`AnsibleModule.log`. If a module implements its own logging then
it needs to check this value. The best way to look at this is for the module
to instantiate an `AnsibleModule` and then check the value of
:attr:`AnsibleModule.no_log`.
Boolean. Set to True whenever a parameter in a task or play specifies ``no_log``. Any module that calls :py:meth:`AnsibleModule.log` handles this automatically. If a module implements its own logging then
it needs to check this value. To access in a module, instantiate an
``AnsibleModule`` and then check the value of :attr:`AnsibleModule.no_log`.
.. note::
``no_log`` specified in a module's argument_spec are handled by a different mechanism.
``no_log`` specified in a module's argument_spec is handled by a different mechanism.
_ansible_debug
^^^^^^^^^^^^^^^
This is a boolean that turns on more verbose logging. If a module uses
Boolean. Turns more verbose logging on or off and turns on logging of
external commands that the module executes. If a module uses
:py:meth:`AnsibleModule.debug` rather than :py:meth:`AnsibleModule.log` then
the messages are only logged if this is True. This also turns on logging of
external commands that the module executes. This can be changed via
the ``debug`` setting in :file:`ansible.cfg` or the environment variable
:envvar:`ANSIBLE_DEBUG`. If, for some reason, a module must access this, it
should do so by instantiating an `AnsibleModule` and accessing
:attr:`AnsibleModule._debug`.
the messages are only logged if ``_ansible_debug`` is set to ``True``.
To set, add ``debug: True`` to :file:`ansible.cfg` or set the environment
variable :envvar:`ANSIBLE_DEBUG`. To access in a module, instantiate an
``AnsibleModule`` and access :attr:`AnsibleModule._debug`.
_ansible_diff
^^^^^^^^^^^^^^^
This boolean is turned on via the ``--diff`` command line option. If a module
supports it, it will tell the module to show a unified diff of changes to be
made to templated files. The proper way for a module to access this is by
instantiating an `AnsibleModule` and accessing
Boolean. If a module supports it, tells the module to show a unified diff of
changes to be made to templated files. To set, pass the ``--diff`` command line
option. To access in a module, instantiate an `AnsibleModule` and access
:attr:`AnsibleModule._diff`.
_ansible_verbosity
^^^^^^^^^^^^^^^^^^
This value could be used for finer grained control over logging. However, it
is currently unused.
Unused. This value could be used for finer grained control over logging.
_ansible_selinux_special_fs
^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is a list of names of filesystems which should have a special selinux
context. They are used by the `AnsibleModule` methods which operate on
files (changing attributes, moving, and copying). The list of names is set
via a comma separated string of filesystem names from :file:`ansible.cfg`::
List. Names of filesystems which should have a special SELinux
context. They are used by the `AnsibleModule` methods which operate on
files (changing attributes, moving, and copying). To set, add a comma separated string of filesystem names in :file:`ansible.cfg`::
# ansible.cfg
[selinux]
special_context_filesystems=nfs,vboxsf,fuse,ramfs
If a module cannot use the builtin ``AnsibleModule`` methods to manipulate
files and needs to know about these special context filesystems, it should
instantiate an ``AnsibleModule`` and then examine the list in
Most modules can use the built-in ``AnsibleModule`` methods to manipulate
files. To access in a module that needs to know about these special context filesystems, instantiate an ``AnsibleModule`` and examine the list in
:attr:`AnsibleModule._selinux_special_fs`.
This replaces :attr:`ansible.module_utils.basic.SELINUX_SPECIAL_FS` from
:ref:`module_replacer`. In module replacer it was a comma separated string of
filesystem names. Under Ansiballz it's an actual list.
:ref:`module_replacer`. In module replacer it was a comma separated string of
filesystem names. Under Ansiballz it's an actual list.
.. versionadded:: 2.1
_ansible_syslog_facility
^^^^^^^^^^^^^^^^^^^^^^^^
This parameter controls which syslog facility ansible module logs to. It may
be set by changing the ``syslog_facility`` value in :file:`ansible.cfg`. Most
This parameter controls which syslog facility Ansible module logs to. To set, change the ``syslog_facility`` value in :file:`ansible.cfg`. Most
modules should just use :meth:`AnsibleModule.log` which will then make use of
this. If a module has to use this on its own, it should instantiate an
this. If a module has to use this on its own, it should instantiate an
`AnsibleModule` and then retrieve the name of the syslog facility from
:attr:`AnsibleModule._syslog_facility`. The code will look slightly different
than it did under :ref:`module_replacer` due to how hacky the old way was
:attr:`AnsibleModule._syslog_facility`. The Ansiballz code is less hacky than the old :ref:`module_replacer` code:
.. code-block:: python
# Old way
# Old module_replacer way
import syslog
syslog.openlog(NAME, 0, syslog.LOG_USER)
# New way
# New Ansiballz way
import syslog
facility_name = module._syslog_facility
facility = getattr(syslog, facility_name, syslog.LOG_USER)
@ -483,14 +467,29 @@ than it did under :ref:`module_replacer` due to how hacky the old way was
_ansible_version
^^^^^^^^^^^^^^^^
This parameter passes the version of ansible that runs the module. To access
This parameter passes the version of Ansible that runs the module. To access
it, a module should instantiate an `AnsibleModule` and then retrieve it
from :attr:`AnsibleModule.ansible_version`. This replaces
from :attr:`AnsibleModule.ansible_version`. This replaces
:attr:`ansible.module_utils.basic.ANSIBLE_VERSION` from
:ref:`module_replacer`.
.. versionadded:: 2.1
.. _flow_module_return_values:
Module return values & Unsafe strings
-------------------------------------
At the end of a module's execution, it formats the data that it wants to return as a JSON string and prints the string to its stdout. The normal action plugin receives the JSON string, parses it into a Python dictionary, and returns it to the executor.
If Ansible templated every string return value, it would be vulnerable to an attack from users with access to managed nodes. If an unscrupulous user disguised malicious code as Ansible return value strings, and if those strings were then templated on the controller, Ansible could execute arbitrary code. To prevent this scenario, Ansible marks all strings inside returned data as ``Unsafe``, emitting any Jinja2 templates in the strings verbatim, not expanded by Jinja2.
Strings returned by invoking a module through ``ActionPlugin._execute_module()`` are automatically marked as ``Unsafe`` by the normal action plugin. If another action plugin retrieves information from a module through some other means, it must mark its return data as ``Unsafe`` on its own.
In case a poorly-coded action plugin fails to mark its results as "Unsafe," Ansible audits the results again when they are returned to the executor,
marking all strings as ``Unsafe``. The normal action plugin protects itself and any other code that it calls with the result data as a parameter. The check inside the executor protects the output of all other action plugins, ensuring that subsequent tasks run by Ansible will not template anything from those results either.
.. _flow_special_considerations:
Special considerations
@ -510,7 +509,7 @@ Ansible can transfer a module to a remote machine in one of two ways:
into the remote interpreter's stdin.
Pipelining only works with modules written in Python at this time because
Ansible only knows that Python supports this mode of operation. Supporting
Ansible only knows that Python supports this mode of operation. Supporting
pipelining means that whatever format the module payload takes before being
sent over the wire must be executable by Python via stdin.
@ -522,13 +521,13 @@ Why pass args over stdin?
Passing arguments via stdin was chosen for the following reasons:
* When combined with :ref:`ANSIBLE_PIPELINING`, this keeps the module's arguments from
temporarily being saved onto disk on the remote machine. This makes it
temporarily being saved onto disk on the remote machine. This makes it
harder (but not impossible) for a malicious user on the remote machine to
steal any sensitive information that may be present in the arguments.
* Command line arguments would be insecure as most systems allow unprivileged
users to read the full commandline of a process.
* Environment variables are usually more secure than the commandline but some
systems limit the total size of the environment. This could lead to
systems limit the total size of the environment. This could lead to
truncation of the parameters if we hit that limit.

93
docs/docsite/rst/dev_guide/overview_architecture.rst
View file

@ -1,5 +1,5 @@
********************
Ansible Architecture
Ansible architecture
********************
Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs.
@ -10,26 +10,35 @@ It uses no agents and no additional custom security infrastructure, so it's easy
In this section, we'll give you a really quick overview of how Ansible works so you can see how the pieces fit together.
.. contents::
:local:
Modules
=======
Ansible works by connecting to your nodes and pushing out small programs, called "Ansible Modules" to them. These programs are written to be resource models of the desired state of the system. Ansible then executes these modules (over SSH by default), and removes them when finished.
Ansible works by connecting to your nodes and pushing out scripts called "Ansible modules" to them. Most modules accept parameters that describe the desired state of the system.
Ansible then executes these modules (over SSH by default), and removes them when finished. Your library of modules can reside on any machine, and there are no servers, daemons, or databases required.
Your library of modules can reside on any machine, and there are no servers, daemons, or databases required. Typically you'll work with your favorite terminal program, a text editor, and probably a version control system to keep track of changes to your content.
You can :ref:`write your own modules <developing_modules_general>`, though you should first consider :ref:`whether you should <developing_modules>`. Typically you'll work with your favorite terminal program, a text editor, and probably a version control system to keep track of changes to your content. You may write specialized modules in any language that can return JSON (Ruby, Python, bash, etc).
Module utilities
================
When multiple modules use the same code, Ansible stores those functions as module utilities to minimize duplication and maintenance. For example, the code that parses URLs is ``lib/ansible/module_utils/url.py``. You can :ref:`write your own module utilities <developing_module_utilities>` as well. Module utilities may only be written in Python or in PowerShell.
Plugins
=======
Plugins are pieces of code that augment Ansible's core functionality. Ansible ships with a number of handy plugins, and you can easily write your own.
:ref:`Plugins <plugins_lookup>` augment Ansible's core functionality. While modules execute on the target system in separate processes (usually that means on a remote system), plugins execute on the control node within the ``/usr/bin/ansible`` process. Plugins offer options and extensions for the core features of Ansible - transforming data, logging output, connecting to inventory, and more. Ansible ships with a number of handy plugins, and you can easily :ref:`write your own <developing_plugins>`. For example, you can write an :ref:`inventory plugin <developing_inventory>` to connect to any datasource that returns JSON. Plugins must be written in Python.
Inventory
=========
By default, Ansible represents what machines it manages using a very simple INI file that puts all of your managed machines in groups of your own choosing.
By default, Ansible represents the machines it manages in a file (INI, YAML, etc.) that puts all of your managed machines in groups of your own choosing.
To add new machines, there is no additional SSL signing server involved, so there's never any hassle deciding why a particular machine didn't get linked up due to obscure NTP or DNS issues.
If there's another source of truth in your infrastructure, Ansible can also plugin to that, such as drawing inventory, group, and variable information from sources like EC2, Rackspace, OpenStack, and more.
If there's another source of truth in your infrastructure, Ansible can also connect to that. Ansible can draw inventory, group, and variable information from sources like EC2, Rackspace, OpenStack, and more.
Here's what a plain text inventory file looks like::
@ -67,8 +76,74 @@ Here's what a simple playbook looks like::
- common
- content
.. _ansible_search_path:
Extending Ansible with plug-ins and the API
===========================================
The Ansible search path
=======================
Should you want to write your own, Ansible modules can be written in any language that can return JSON (Ruby, Python, bash, etc). Inventory can also plug in to any datasource by writing a program that speaks to that datasource and returns JSON. There's also various Python APIs for extending Ansible's connection types (SSH is not the only transport possible), callbacks (how Ansible logs, etc), and even for adding new server side behaviors.
Modules, module utilities, plugins, playbooks, and roles can live in multiple locations. If you
write your own code to extend Ansible's core features, you may have multiple files with similar or the same names in different locations on your Ansible control node. The search path determines which of these files Ansible will discover and use on any given playbook run.
Ansible's search path grows incrementally over a run. As
Ansible finds each playbook and role included in a given run, it appends
any directories related to that playbook or role to the search path. Those
directories remain in scope for the duration of the run, even after the playbook or role
has finished executing. Ansible loads modules, module utilities, and plugins in this order:
1. Directories adjacent to a playbook specified on the command line. If you run Ansible with ``ansible-playbook /path/to/play.yml``, Ansible appends these directories if they exist:
.. code-block:: bash
/path/to/modules
/path/to/module_utils
/path/to/plugins
2. Directories adjacent to a playbook that is statically imported by a
playbook specified on the command line. If ``play.yml`` includes
``- import_playbook: /path/to/subdir/play1.yml``, Ansible appends these directories if they exist:
.. code-block:: bash
/path/to/subdir/modules
/path/to/subdir/module_utils
/path/to/subdir/plugins
3. Subdirectories of a role directory referenced by a playbook. If
``play.yml`` runs ``myrole``, Ansible appends these directories if they exist:
.. code-block:: bash
/path/to/roles/myrole/modules
/path/to/roles/myrole/module_utils
/path/to/roles/myrole/plugins
4. Directories specified as default paths in ``ansible.cfg`` or by the related
environment variables, including the paths for the various plugin types. See :ref:`ansible_configuration_settings` for more information.
Sample ``ansible.cfg`` fields:
.. code-block:: bash
DEFAULT_MODULE_PATH
DEFAULT_MODULE_UTILS_PATH
DEFAULT_CACHE_PLUGIN_PATH
DEFAULT_FILTER_PLUGIN_PATH
Sample environment variables:
.. code-block:: bash
ANSIBLE_LIBRARY
ANSIBLE_MODULE_UTILS
ANSIBLE_CACHE_PLUGINS
ANSIBLE_FILTER_PLUGINS
5. The standard directories that ship as part of the Ansible distribution.
.. caution::
Modules, module utilities, and plugins in user-specified directories will
override the standard versions. This includes some files with generic names.
For example, if you have a file named ``basic.py`` in a user-specified
directory, it will override the standard ``ansible.module_utils.basic``.
If you have more than one module, module utility, or plugin with the same name in different user-specified directories, the order of commands at the command line and the order of includes and roles in each play will affect which one is found and used on that particular play.

30
docs/docsite/rst/dev_guide/testing_documentation.rst
View file

@ -10,17 +10,27 @@ Before you submit a module for inclusion in the main Ansible repo, you must test
To check the HTML output of your module documentation:
#. Save your completed module file into the correct directory: ``lib/ansible/modules/$CATEGORY/my_code.py``.
#. Move to the docsite directory: ``cd /path/to/ansible/docs/docsite/``.
#. Run the command to build the docs for your module: ``MODULES=my_code make webdocs``.
#. View the HTML page at ``file:///path/to/ansible/docs/docsite/_build/html/my_code_module.html``.
#. Ensure working :ref:`development environment <environment_setup>`.
#. Install required Python packages (drop '--user' in venv/virtualenv):
To build the HTML documentation for multiple modules, use a comma-separated list of module names: ``MODULES=my_code,my_other_code make webdocs``.
.. code-block:: bash
To ensure that your documentation matches your ``argument_spec``, run the ``validate-modules`` test.
pip install --user -r requirements.txt
pip install --user -r docs/docsite/requirements.txt
.. code-block:: bash
#. Ensure your module is in the correct directory: ``lib/ansible/modules/$CATEGORY/mymodule.py``.
#. Build HTML from your module documentation: ``MODULES=mymodule make webdocs``.
#. To build the HTML documentation for multiple modules, use a comma-separated list of module names: ``MODULES=mymodule,mymodule2 make webdocs``.
#. View the HTML page at ``file:///path/to/docs/docsite/_build/html/modules/mymodule_module.html``.
# If you don't already, ensure you are using your local checkout
source hacking/env-setup
./test/sanity/validate-modules/validate-modules --arg-spec --warnings lib/ansible/modules/$CATEGORY/my_code.py
To ensure that your module documentation matches your ``argument_spec``:
#. Install required Python packages (drop '--user' in venv/virtualenv):
.. code-block:: bash
pip install --user -r test/runner/requirements/sanity.txt
#. run the ``validate-modules`` test::
./test/sanity/validate-modules/validate-modules --arg-spec --warnings lib/ansible/modules/$CATEGORY/mymodule.py

2
docs/docsite/rst/installation_guide/intro_installation.rst
View file

@ -150,7 +150,7 @@ To configure the PPA on your machine and install ansible run these commands:
$ sudo apt-add-repository --yes --update ppa:ansible/ansible
$ sudo apt install ansible
.. note:: On older Ubuntu distributions, "software-properties-common" is called "python-software-properties". You may want to use ``apt-get`` instead of ``apt`` in older versions.
.. note:: On older Ubuntu distributions, "software-properties-common" is called "python-software-properties". You may want to use ``apt-get`` instead of ``apt`` in older versions. Also, be aware that only newer distributions (i.e. 18.04, 18.10, etc.) have a ``-u`` or ``--update`` flag, so adjust your script accordingly.
Debian/Ubuntu packages can also be built from the source checkout, run:

4
docs/docsite/rst/porting_guides/porting_guide_2.5.rst
View file

@ -143,7 +143,7 @@ Ansible fact namespacing
Ansible facts, which have historically been written to names like ``ansible_*``
in the main facts namespace, have been placed in their own new namespace,
``ansible_facts.*`` For example, the fact ``ansible_distribution`` is now best
queried through the variable structure ``ansible_facts.distribution``.
queried through the variable structure ``ansible_facts.distribution``.
A new configuration variable, ``inject_facts_as_vars``, has been added to
ansible.cfg. Its default setting, 'True', keeps the 2.4 behavior of facts
@ -389,4 +389,4 @@ If your module uses shared module utilities, you must update all references. For
from ansible.module_utils.network.vyos.vyos import get_config, load_config
See the module utilities developer guide see :ref:`appendix_module_utilities` for more information.
See the module utilities developer guide see :ref:`developing_module_utilities` for more information.

7
docs/docsite/rst/reference_appendices/faq.rst
View file

@ -96,6 +96,13 @@ With earlier versions of Ansible, it was necessary to configure a
suitable `ProxyCommand` for one or more hosts in `~/.ssh/config`,
or globally by setting `ssh_args` in `ansible.cfg`.
.. _ssh_serveraliveinterval:
How do I get Ansible to notice a dead target in a timely manner?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
You can add ``-o ServerAliveInterval=NumberOfSeconds`` in ``ssh_args`` from ``ansible.cfg``. Without this option, SSH and therefore Ansible will wait until the TCP connection times out. Another solution is to add ``ServerAliveInterval`` into your global SSH configuration. A good value for ``ServerAliveInterval`` is up to you to decide; keep in mind that ``ServerAliveCountMax=3`` is the SSH default so any value you set will be tripled before terminating the SSH session.
.. _ec2_cloud_performance:
How do I speed up management inside EC2?

31
docs/docsite/rst/scenario_guides/guide_aws.rst
View file

@ -132,6 +132,37 @@ With the host group now created, a second play at the bottom of the same provisi
- name: Check NTP service
service: name=ntpd state=started
.. _aws_security_groups:
Security Groups
```````````````
Security groups on AWS are stateful. The response of a request from your instance is allowed to flow in regardless of inbound security group rules and vice-versa.
In case you only want allow traffic with AWS S3 service, you need to fetch the current IP ranges of AWS S3 for one region and apply them as an egress rule.::
- name: fetch raw ip ranges for aws s3
set_fact:
raw_s3_ranges: "{{ lookup('aws_service_ip_ranges', region='eu-central-1', service='S3', wantlist=True) }}"
- name: prepare list structure for ec2_group module
set_fact:
s3_ranges: "{{ s3_ranges | default([]) + [{'proto': 'all', 'cidr_ip': item, 'rule_desc': 'S3 Service IP range'}] }}"
with_items: "{{ raw_s3_ranges }}"
- name: set S3 IP ranges to egress rules
ec2_group:
name: aws_s3_ip_ranges
description: allow outgoing traffic to aws S3 service
region: eu-central-1
state: present
vpc_id: vpc-123456
purge_rules: true
purge_rules_egress: true
rules: []
rules_egress: "{{ s3_ranges }}"
tags:
Name: aws_s3_ip_ranges
.. _aws_host_inventory:
Host Inventory

16
docs/docsite/rst/scenario_guides/guide_gce.rst
View file

@ -29,7 +29,7 @@ used, but you may experience issues trying to use them together.
While the community GCP modules are not going away, Google is investing effort
into the new "gcp_*" modules. Google is committed to ensuring the Ansible
community has a great experience with GCP and therefore recommends adopting
community has a great experience with GCP and therefore recommends adopting
these new modules if possible.
@ -42,7 +42,7 @@ The Google Cloud Platform (GCP) modules require both the ``requests`` and the
$ pip install requests google-auth
Alternatively for RHEL / CentOS, the ``python-requests`` package is also
Alternatively for RHEL / CentOS, the ``python-requests`` package is also
available to satisfy ``requests`` libraries.
.. code-block:: bash
@ -169,9 +169,9 @@ rest.
hosts: localhost
gather_facts: no
vars:
project: my-project
auth_kind: serviceaccount
service_account_file: /home/my_account.json
gcp_project: my-project
gcp_cred_kind: serviceaccount
gcp_cred_file: /home/my_account.json
zone: "us-central1-a"
region: "us-central1"
@ -234,10 +234,10 @@ rest.
register: instance
- name: Wait for SSH to come up
wait_for: host={{ instance.address }} port=22 delay=10 timeout=60
wait_for: host={{ address.address }} port=22 delay=10 timeout=60
- name: Add host to groupname
add_host: hostname={{ instance.address }} groupname=new_instances
add_host: hostname={{ address.address }} groupname=new_instances
- name: Manage new instances
@ -268,7 +268,7 @@ module (and more!). Below is a mapping of ``gce`` fields over to
``gcp_compute_instance`` fields.
============================ ========================================== ======================
gce.py gcp_compute_instance.py Notes
gce.py gcp_compute_instance.py Notes
============================ ========================================== ======================
state state/status State on gce has multiple values: "present", "absent", "stopped", "started", "terminated". State on gcp_compute_instance is used to describe if the instance exists (present) or does not (absent). Status is used to describe if the instance is "started", "stopped" or "terminated".
image disks[].initialize_params.source_image You'll need to create a single disk using the disks[] parameter and set it to be the boot disk (disks[].boot = true)

29
docs/docsite/rst/user_guide/playbooks_async.rst
View file

@ -9,9 +9,19 @@ be running operations that take longer than the SSH timeout.
To avoid blocking or timeout issues, you can use asynchronous mode to run all of your tasks at once and then poll until they are done.
The behaviour of asynchronous mode depends on the value of `poll`.
Avoid connection timeouts: poll > 0
-----------------------------------
When ``poll`` is a positive value, the playbook will *still* block on the task until it either completes, fails or times out.
In this case, however, `async` explicitly sets the timeout you wish to apply to this task rather than being limited by the connection method timeout.
To launch a task asynchronously, specify its maximum runtime
and how frequently you would like to poll for status. The default
poll value is 10 seconds if you do not specify a value for `poll`::
poll value is 15 seconds if you do not specify a value for `poll`::
---
@ -35,8 +45,21 @@ poll value is 10 seconds if you do not specify a value for `poll`::
task when run in check mode. See :doc:`playbooks_checkmode` on how to
skip a task in check mode.
Alternatively, if you do not need to wait on the task to complete, you may
run the task asynchronously by specifying a poll value of 0::
Concurrent tasks: poll = 0
--------------------------
When ``poll`` is 0, Ansible will start the task and immediately move on to the next one without waiting for a result.
From the point of view of sequencing this is asynchronous programming: tasks may now run concurrently.
The playbook run will end without checking back on async tasks.
The async tasks will run until they either complete, fail or timeout according to their `async` value.
If you need a synchronization point with a task, register it to obtain its job ID and use the :ref:`async_status <async_status_module>` module to observe it.
You may run a task asynchronously by specifying a poll value of 0::
---

29
docs/docsite/rst/user_guide/playbooks_error_handling.rst
View file

@ -63,11 +63,9 @@ the handler from running, such as a host becoming unreachable.)
Controlling What Defines Failure
````````````````````````````````
Suppose the error code of a command is meaningless and to tell if there
is a failure what really matters is the output of the command, for instance
if the string "FAILED" is in the output.
Ansible lets you define what "failure" means in each task using the ``failed_when`` conditional. As with all conditionals in Ansible, lists of multiple ``failed_when`` conditions are joined with an implicit ``and``, meaning the task only fails when *all* conditions are met. If you want to trigger a failure when any of the conditions is met, you must define the conditions in a string with an explicit ``or`` operator.
Ansible provides a way to specify this behavior as follows::
You may check for failure by searching for a word or phrase in the output of a command::
- name: Fail task when the command error output prints FAILED
command: /usr/bin/example-command -x -y -z
@ -93,14 +91,29 @@ In previous version of Ansible, this can still be accomplished as follows::
msg: "the command failed"
when: "'FAILED' in command_result.stderr"
You can also combine multiple conditions to specify this behavior as follows::
You can also combine multiple conditions for failure. This task will fail if both conditions are true::
- name: Check if a file exists in temp and fail task if it does
command: ls /tmp/this_should_not_be_here
register: result
failed_when:
- '"No such" not in result.stdout'
- result.rc == 0
- '"No such" not in result.stdout'
If you want the task to fail when only one condition is satisfied, change the ``failed_when`` definition to::
failed_when: result.rc == 0 or "No such" not in result.stdout
If you have too many conditions to fit neatly into one line, you can split it into a multi-line yaml value with ``>``::
- name: example of many failed_when conditions with OR
shell: "./myBinary"
register: ret
failed_when: >
("No such file or directory" in ret.stdout) or
(ret.stderr != '') or
(ret.rc == 10)
.. _override_the_changed_result:
@ -168,7 +181,7 @@ Blocks only deal with 'failed' status of a task. A bad task definition or an unr
- debug:
msg: 'I caught an error, can do stuff here to fix it, :-)'
This will 'revert' the failed status of the outer ``block`` task for the run and the play will continue as if it had succeeded.
This will 'revert' the failed status of the outer ``block`` task for the run and the play will continue as if it had succeeded.
See :ref:`block_error_handling` for more examples.
.. seealso::
@ -185,5 +198,3 @@ See :ref:`block_error_handling` for more examples.
Have a question? Stop by the google group!
`irc.freenode.net <http://irc.freenode.net>`_
#ansible IRC chat channel

4
docs/docsite/rst/user_guide/playbooks_loops.rst
View file

@ -376,6 +376,8 @@ You can specify the name of the variable for each loop using ``loop_var`` with `
.. note:: If Ansible detects that the current loop is using a variable which has already been defined, it will raise an error to fail the task.
Extended loop variables
-----------------------
.. versionadded:: 2.8
As of Ansible 2.8 you can get extended loop information using the ``extended`` option to loop control. This option will expose the following information.
@ -400,6 +402,8 @@ Variable Description
loop_control:
extended: yes
Accessing the name of your loop_var
-----------------------------------
.. versionadded:: 2.8
As of Ansible 2.8 you can get the name of the value provided to ``loop_control.loop_var`` using the ``ansible_loop_var`` variable

7
docs/docsite/rst/user_guide/playbooks_tests.rst
View file

@ -59,8 +59,7 @@ To match strings against a substring or a regular expression, use the "match", "
msg: "matched pattern 4"
when: url is regex("example.com/\w+/foo")
'match' requires zero or more characters at the beginning of the string, while 'search' only requires matching a subset of the string.
'match' requires zero or more characters at the beginning of the string, while 'search' only requires matching a subset of the string. By default, 'regex' works like `search`, but `regex` can be configured to perform other tests as well.
.. _testing_versions:
@ -199,7 +198,7 @@ The following tests can provide information about a path on the controller::
- debug:
msg: "path is {{ (mypath is abs)|ternary('absolute','relative')}}"
- debug:
- debug:
msg: "path is the same file as path2"
when: mypath is same_file(path2)
@ -266,5 +265,3 @@ The following tasks are illustrative of the tests meant to check the status of t
Have a question? Stop by the google group!
`irc.freenode.net <http://irc.freenode.net>`_
#ansible IRC chat channel

21
docs/docsite/rst/user_guide/playbooks_vault.rst
View file

@ -5,13 +5,30 @@ Using Vault in playbooks
.. contents:: Topics
The "Vault" is a feature of Ansible that allows you to keep sensitive data such as passwords or keys in encrypted files, rather than as plaintext in playbooks or roles. These vault files can then be distributed or placed in source control.
The "Vault" is a feature of Ansible that allows you to keep sensitive data such as passwords or keys protected at rest, rather than as plaintext in playbooks or roles. These vaults can then be distributed or placed in source control.
There are 2 types of vaulted content and each has their own uses and limitations:
:Vaulted files:
* The full file is encrypted in the vault, this can contain Ansible variables or any other type of content.
* It will always be decrypted when loaded or referenced, Ansible cannot know if it needs the content unless it decrypts it.
* It can be used for inventory, anything that loads variables (i.e vars_files, group_vars, host_vars, include_vars, etc)
and some actions that deal with files (i.e M(copy), M(assemble), M(script), etc).
:Single encrypted variable:
* Only specific variables are encrypted inside a normal 'variable file'.
* Does not work for other content, only variables.
* Decrypted on demand, so you can have vaulted variables with different vault secrets and only provide those needed.
* You can mix vaulted and non vaulted variables in the same file, even inline in a play or role.
.. warning::
* Vault ONLY protects data 'at rest', once decrypted play and plugin authors are responsible of avoiding any secrets discolsure,
see ``no_log`` for details on hiding output.
To enable this feature, a command line tool, :ref:`ansible-vault` is used to edit files, and a command line flag :option:`--ask-vault-pass <ansible-vault --ask-vault-pass>`, :option:`--vault-password-file <ansible-vault --vault-password-file>` or :option:`--vault-id <ansible-playbook --vault-id>` is used. You can also modify your ``ansible.cfg`` file to specify the location of a password file or configure Ansible to always prompt for the password. These options require no command line flag usage.
For best practices advice, refer to :ref:`best_practices_for_variables_and_vaults`.
Running a Playbook With Vault
`````````````````````````````

77
docs/docsite/rst/user_guide/vault.rst
View file

@ -16,7 +16,12 @@ For best practices advice, refer to :ref:`best_practices_for_variables_and_vault
What Can Be Encrypted With Vault
````````````````````````````````
Ansible Vault can encrypt any structured data file used by Ansible. This can include "group_vars/" or "host_vars/" inventory variables, variables loaded by "include_vars" or "vars_files", or variable files passed on the ansible-playbook command line with ``-e @file.yml`` or ``-e @file.json``. Role variables and defaults are also included.
File-level encryption
^^^^^^^^^^^^^^^^^^^^^
Ansible Vault can encrypt any structured data file used by Ansible.
This can include "group_vars/" or "host_vars/" inventory variables, variables loaded by "include_vars" or "vars_files", or variable files passed on the ansible-playbook command line with ``-e @file.yml`` or ``-e @file.json``. Role variables and defaults are also included.
Ansible tasks, handlers, and so on are also data so these can be encrypted with vault as well. To hide the names of variables that you're using, you can encrypt the task files in their entirety.
@ -26,7 +31,19 @@ given as the ``src`` argument to the :ref:`copy <copy_module>`, :ref:`template <
<assemble_module>` modules, the file will be placed at the destination on the target host decrypted
(assuming a valid vault password is supplied when running the play).
As of version 2.3, Ansible supports encrypting single values inside a YAML file, using the `!vault` tag to let YAML and Ansible know it uses special processing. This feature is covered in more details below.
.. note::
The advantages of file-level encryption are that it is easy to use and that password rotation is straightforward with :ref:`rekeying <rekeying_files>`.
The drawback is that the contents of files are no longer easy to access and read. This may be problematic if it is a list of tasks (when encrypting a variables file, :ref:`best practice <best_practices_for_variables_and_vaults>` is to keep references to these variables in a non-encrypted file).
Variable-level encryption
^^^^^^^^^^^^^^^^^^^^^^^^^
Ansible also supports encrypting single values inside a YAML file, using the `!vault` tag to let YAML and Ansible know it uses special processing. This feature is covered in more detail :ref:`below <encrypt_string_for_use_in_yaml>`.
.. note::
The advantage of variable-level encryption is that files are still easily legible even if they mix plaintext and encrypted variables.
The drawback is that password rotation is not as simple as with file-level encryption: the :ref:`rekey <ansible_vault_rekey>` command does not work with this method.
.. _vault_ids:
@ -34,23 +51,28 @@ As of version 2.3, Ansible supports encrypting single values inside a YAML file,
Vault IDs and Multiple Vault Passwords
``````````````````````````````````````
*Available since Ansible 2.4*
A vault ID is an identifier for one or more vault secrets. Since Ansible 2.4,
Ansible supports multiple vault passwords. Vault IDs provide
labels for individual vault passwords.
A vault ID is an identifier for one or more vault secrets;
Ansible supports multiple vault passwords.
Vault IDs provide labels to distinguish between individual vault passwords.
To use vault IDs, you must provide an ID *label* of your choosing and a *source* to obtain its password (either ``prompt`` or a file path):
.. code-block:: bash
--vault-id label@source
This switch is available for all Ansible commands that can interact with vaults: :ref:`ansible-vault`, :ref:`ansible-playbook`, etc.
Vault-encrypted content can specify which vault ID it was encrypted with.
Prior to Ansible 2.4, only one vault password could be used at a time, So any
vault files or vars that needed to be decrypted all had to use the same password.
Since Ansible 2.4, vault files or vars that are encrypted with different
passwords can be used at the same time.
For example, a playbook can now include a vars file encrypted with a 'dev' vault
ID and a 'prod' vault ID.
.. note:
Older versions of Ansible, before 2.4, only supported using one single vault password at a time.
.. _creating_files:
@ -67,6 +89,12 @@ First you will be prompted for a password. After providing a password, the tool
The default cipher is AES (which is shared-secret based).
To create a new encrypted data file with the Vault ID 'password1' assigned to it and be prompted for the password, run:
.. code-block:: bash
ansible-vault create --vault-id password1@prompt foo.yml
.. _editing_encrypted_files:
@ -81,6 +109,12 @@ the file, saving it back when done and removing the temporary file:
ansible-vault edit foo.yml
To edit a file encrypted with the 'vault2' password file and assigned the 'pass2' vault ID:
.. code-block:: bash
ansible-vault edit --vault-id pass2@vault2 foo.yml
.. _rekeying_files:
@ -96,6 +130,13 @@ Should you wish to change your password on a vault-encrypted file or files, you
This command can rekey multiple data files at once and will ask for the original
password and also the new password.
To rekey files encrypted with the 'preprod2' vault ID and the 'ppold' file and be prompted for the new password:
.. code-block:: bash
ansible-vault rekey --vault-id preprod2@ppold --new-vault-id preprod2@prompt foo.yml bar.yml baz.yml
A different ID could have been set for the rekeyed files by passing it to ``--new-vault-id``.
.. _encrypting_files:
@ -109,6 +150,18 @@ the :ref:`ansible-vault encrypt <ansible_vault_encrypt>` command. This command
ansible-vault encrypt foo.yml bar.yml baz.yml
To encrypt existing files with the 'project' ID and be prompted for the password:
.. code-block:: bash
ansible-vault encrypt --vault-id project@prompt foo.yml bar.yml baz.yml
.. note::
It is technically possible to separately encrypt files or strings with the *same* vault ID but *different* passwords, if different password files or prompted passwords are provided each time.
This could be desirable if you use vault IDs as references to classes of passwords (rather than a single password) and you always know which specific password or file to use in context. However this may be an unnecessarily complex use-case.
If two files are encrypted with the same vault ID but different passwords by accident, you can use the :ref:`rekey <rekeying_files>` command to fix the issue.
.. _decrypting_files:

4
docs/docsite/rst/user_guide/windows_setup.rst
View file

@ -101,6 +101,8 @@ The following PowerShell command will install the hotfix:
(New-Object -TypeName System.Net.WebClient).DownloadFile($url, $file)
powershell.exe -ExecutionPolicy ByPass -File $file -Verbose
For more details, please refer to the `Hotfix document <https://support.microsoft.com/en-us/help/2842230/out-of-memory-error-on-a-computer-that-has-a-customized-maxmemorypersh>`_ from Microsoft.
WinRM Setup
```````````
Once Powershell has been upgraded to at least version 3.0, the final step is for the
@ -381,7 +383,7 @@ target Windows host:
winrs -r:http://server:5985/wsman -u:Username -p:Password ipconfig
# Test out HTTPS (will fail if the cert is not verifiable)
winrs -r:http://server:5985/wsman -u:Username -p:Password -ssl ipconfig
winrs -r:https://server:5986/wsman -u:Username -p:Password -ssl ipconfig
# Test out HTTPS, ignoring certificate verification
$username = "Username"