Commit graph

5875 commits

Author SHA1 Message Date
Nils Steinger
0129fb0a44 Remove duplicates from host list *before* caching it
Ansible previously added hosts to the host list multiple times for commands
like `ansible -i 'localhost,' -c local -m ping 'localhost,localhost'
--list-hosts`.
8d5f36a fixed the obvious error, but still added the un-deduplicated list to a
cache, so all future invocations of get_hosts() would retrieve a
non-deduplicated list.
This caused problems down the line: For some reason, Ansible only ever
schedules "flush_handlers" tasks (instead of scheduling any actual tasks from
the playbook) for hosts that are contained in the host lists multiple times.
This probably happens because the host states are stored in a dictionary
indexed by the hostnames, so duplicate hostname would cause the state to be
overwritten by subsequent invocations of … something.
2015-12-05 15:54:08 +01:00
Toshio Kuratomi
c8fd599e14 Merge pull request #13373 from no2a/boolean
BOOLEAN should contain boolean literals
2015-12-04 12:17:12 -08:00
Toshio Kuratomi
0434644d12 Transform exceptions into ansible messages via to_unicode instead of str to avoid tracebacks.
Fixes #13385
2015-12-04 11:50:39 -08:00
James Cammarata
84507aedd4 Adding a uuid field so we can track host equality across serialization too 2015-12-04 13:33:27 -05:00
James Cammarata
750adbaa27 Changing up how host (in)equality is checked
Fixes #13397
2015-12-04 12:57:46 -05:00
Toshio Kuratomi
e385c91fa5 Update submodule refs# 2015-12-04 09:57:06 -08:00
Brian Coca
d5446f9804 fixed ansible-pull broken options
* sudo was not working, now it supports full become
* now default checkout dir works, not only when specifying
* paths for checkout dir get expanded
* fixed limit options for playbook
* added verbose and debug info
2015-12-03 20:47:02 -08:00
Brian Coca
e1c62fb5af reverted to previous pull checkout dir behaviour
This fixes bugs with not finding plays when not specifying checkout dir
Also makes it backwards compatible
2015-12-03 19:42:05 -08:00
Brian Coca
8d5f36a6c2 return unique list of hosts 2015-12-03 19:39:57 -08:00
Brian Coca
a1f516824e corrected playbook path, reformated options help
the last just to make the help consistent and readable
2015-12-03 18:23:53 -08:00
Brian Coca
26520442bd Now and/or shell expressions depend on shell plugin
This should fix issues with fish shell users as && and || are
not valid syntax, fish uses actual 'and' and 'or' programs.
Also updated to allow for fish backticks pushed quotes to subshell,
fish seems to handle spaces w/o them.
Lastly, removed encompassing subshell () for fish compatibility.
fixes #13199
2015-12-03 16:38:34 -08:00
James Cammarata
f467f1770f Properly compare object references for Hosts when adding new ones
Fixes #13397
2015-12-03 15:25:54 -05:00
James Cammarata
4426b7f6e0 fix sorting of groups for host vars
Fixes #13371
2015-12-03 14:23:02 -05:00
Brian Coca
7d073bdd17 Merge pull request #13345 from amenonsen/extract
Add an 'extract' filter
2015-12-03 11:19:06 -08:00
Peter Sprygada
29f5c5db71 bugfix for ios.py shared module argument creation
This patch fixes a bug in module_utils/ios.py where the the wrong shared
module arguments are being generated.  This bug prevented the shared module
from operating correctly.  This patch should be generally applied.
2015-12-03 12:53:07 -05:00
James Cammarata
5f83a6aeda Properly default remote_user for delegated-to hosts
Fixes #13323
2015-12-03 11:31:33 -05:00
Toshio Kuratomi
e201a255d1 Revert "Make sudo+requiretty and ANSIBLE_PIPELINING work together"
This reverts commit f488de8599.

Reverting for now due to hard to pin down bugs: #13410  #13411
2015-12-03 08:01:05 -08:00
Toshio Kuratomi
9b81c35d06 Don't compare or merge str with unicode
Fixes #13387
2015-12-02 20:53:50 -08:00
James Cammarata
a183972477 Don't use play vars in HostVars
Fixes #13398
2015-12-02 14:16:47 -05:00
James Cammarata
8ff67e0494 Default msg param to AnsibleError to avoid serialization problems 2015-12-02 14:16:47 -05:00
James Cammarata
5f1f2acfea Merge pull request #13405 from sivel/v2-playbook-on-start
Get v2_playbook_on_start working
2015-12-02 12:40:24 -05:00
Matt Martz
384b2e0234 Get v2_playbook_on_start working
* Move self._tqm.load_callbacks() earlier to ensure that v2_on_playbook_start can fire
* Pass the playbook instance to v2_on_playbook_start
* Add a _file_name instance attribute to the playbook
2015-12-02 11:29:51 -06:00
Brian Coca
7708661270 Merge pull request #13402 from amenonsen/expandpath
Use CLI.expand_tilde also for the vault --output file
2015-12-02 09:23:46 -08:00
Brian Coca
94d2958e1f Merge pull request #13403 from amenonsen/module-lang
Make module_lang default to whatever LANG is set to on the control node
2015-12-02 09:06:57 -08:00
Abhijit Menon-Sen
0d16d16ab8 Make module_lang default to whatever LANG is set to on the control node 2015-12-02 22:25:10 +05:30
Abhijit Menon-Sen
fac7626230 Use CLI.expand_tilde also for the vault --output file 2015-12-02 22:08:37 +05:30
Matt Martz
ce54a59cdc Catch additional assertion errors for load_list_of_blocks 2015-12-02 10:35:20 -06:00
Brian Coca
53cd802251 better error on invalid task lists 2015-12-02 08:14:20 -08:00
James Cammarata
f7dc5a9515 Minor tweak and comment addition to 974a0ce3 2015-12-02 09:11:08 -05:00
James Cammarata
f1d53a87e5 Merge pull request #13394 from Christoph-D/issue-13370
Fix #13370
2015-12-02 08:57:09 -05:00
Abhijit Menon-Sen
8cf1815867 Add an 'extract' filter
At its most basic, this is nothing more than an array or hash lookup,
but when used in conjunction with map, it is very useful. For example,
while constructing an "ssh-keyscan …" command to update known_hosts on
all hosts in a group, one can get a list of IP addresses with:

    groups['x']|map('extract', hostvars, 'ec2_ip_address')|list

This returns hostvars[a].ec2_ip_address, hostvars[b].ec2_ip_address, and
so on. You can even specify an array of keys for a recursive lookup, and
mix string and integer keys depending on what you're looking up:

    ['localhost']|map('extract', hostvars, ['vars','group_names',0])|first
        == hostvars['localhost']['vars']['group_names'][0]
            == 'ungrouped'

Includes documentation and tests.
2015-12-02 18:16:38 +05:30
Abhijit Menon-Sen
22381be253 Bump version in devel now that stable-2.0 has been branched 2015-12-02 17:57:53 +05:30
Christoph Dittmann
267199fdd5 Update debug messages and comments
The comment was taken literally from lib/plugins/strategy/linear.py and
makes no sense in free.py where we have no noop tasks.

Also update the debug messages.
2015-12-02 11:17:24 +01:00
Christoph Dittmann
c6e400fbea Fix issue #13370
all_blocks is referenced after the loop over included_files, so it needs
to be initialized before this loop, not inside.
2015-12-02 11:17:24 +01:00
Christoph Dittmann
9a6ae1d62a Let PlayIterator.add_tasks accept empty task lists
PlayIterator.add_tasks raised an error when trying to add an empty task
list.  This was the root cause of ansible issue #13370.
2015-12-02 11:17:24 +01:00
Brian Coca
b92cb93539 unconditionally set vars on init to avoid issues with var precedence 2015-12-01 21:24:46 -08:00
Peter Sprygada
2491afc8b6 fixes a syntax issue with module_utils/eapi.py
This patch fixes an issue with the common args dict in the eapi shared
module.  This patch is required for the eapi shared module to be properly
imported and is therefore should be applied to all instances.
2015-12-01 23:34:59 -05:00
Brian Coca
8c630406c9 Merge pull request #13357 from privateip/feature_ssh_shared_module
initial add of ssh shared module.
2015-12-01 19:15:03 -08:00
Brian Coca
923b5467e3 Merge pull request #13354 from privateip/feature_eapi
adds device common argument to shared module
2015-12-01 19:14:07 -08:00
Brian Coca
574d7be8d5 Merge pull request #13358 from privateip/feature_ios_shared_module
Feature ios shared module
2015-12-01 19:04:24 -08:00
Peter Sprygada
93cd7a2a5f adds module create function for eapi.py shared module
This commit changes the way modules create an instance of AnsibleModule to
now use a common function, eapi_module.  This function will now automatically
append the common argument spec to the module argument_spec.  Module
arguments can override common module arguments
2015-12-01 20:00:41 -05:00
Peter Sprygada
8638e0ccf8 initial add of the ios shared module
This adds shared module support for building modules that connect to Cisco
IOS devices.  It builds on the module_utils/ssh.py shared module.
2015-12-01 19:55:41 -05:00
Brian Coca
3e01516783 fixed signature for init on callbacks
also removed passing display to base class which already handles this
2015-12-01 14:06:54 -08:00
nitzmahone
c94509f273 allow shell plugin to affect remote module filename
Fix for 13368, added get_remote_filename to shell plugins, powershell version appends .ps1 if necessary, base shell plugin no-ops
2015-12-01 13:39:02 -08:00
Abhijit Menon-Sen
f488de8599 Make sudo+requiretty and ANSIBLE_PIPELINING work together
Pipelining is a *significant* performance benefit, because each task can
be completed with a single SSH connection (vs. one ssh connection at the
start to mkdir, plus one sftp and one ssh per task).

Pipelining is disabled by default in Ansible because it conflicts with
the use of sudo if 'Defaults requiretty' is set in /etc/sudoers (as it
is on Red Hat) and su (which always requires a tty).

We can (and already do) make sudo/su happy by using "ssh -t" to allocate
a tty, but then the python interpreter goes into interactive mode and is
unhappy with module source being written to its stdin, per the following
comment from connections/ssh.py:

        # we can only use tty when we are not pipelining the modules.
        # piping data into /usr/bin/python inside a tty automatically
        # invokes the python interactive-mode but the modules are not
        # compatible with the interactive-mode ("unexpected indent"
        # mainly because of empty lines)

Instead of the (current) drastic solution of turning off pipelining when
we use a tty, we can instead use a tty but suppress the behaviour of the
Python interpreter to switch to interactive mode. The easiest way to do
this is to make its stdin *not* be a tty, e.g. with cat|python.

This works, but there's a problem: ssh will ignore -t if its input isn't
really a tty. So we could open a pseudo-tty and use that as ssh's stdin,
but if we then write Python source into it, it's all echoed back to us
(because we're a tty). So we have to use -tt to force tty allocation; in
that case, however, ssh puts the tty into "raw" mode (~ICANON), so there
is no good way for the process on the other end to detect EOF on stdin.
So if we do:

    echo -e "print('hello world')\n"|ssh -tt someho.st "cat|python"

…it hangs forever, because cat keeps on reading input even after we've
closed our pipe into ssh's stdin. We can get around this by writing a
special __EOF__ marker after writing in_data, and doing this:

    echo -e "print('hello world')\n__EOF__\n"|ssh -tt someho.st "sed -ne '/__EOF__/q' -e p|python"

This works fine, but in fact I use a clever python one-liner by mgedmin
to achieve the same effect without depending on sed (at the expense of a
much longer command line, alas; Python really isn't one-liner-friendly).

We also enable pipelining by default as a consequence.
2015-12-01 23:32:20 +05:30
Toshio Kuratomi
50553bc2ba _connect no longer takes a port argument 2015-12-01 09:12:25 -08:00
Brian Coca
f1fcab4610 ignore password flags in become conflict check
since all the --ask pass options end up triggering the same code
and are functionally equivalent, ignore them when it comes to checking
privilege escalation conflicts. This allows using -K when --become-method=su
and so on.
2015-12-01 08:18:31 -08:00
Brian Coca
a6f6a80caa avoid inheritance issues with default=dict declaration at class level
this should avoid the issue of subsequent plays not prompting for a var
prompted for in a previous play.
2015-12-01 08:18:31 -08:00
Toshio Kuratomi
30094912eb boto is expecting that we pass it unicode strings.
The secret_key parameter especially can contain non-ascii characters and
will throw an error if such a string is passed as a byte str.

Potential fix for #13303
2015-12-01 07:03:57 -08:00
Arata Notsu
6d6d4f0c8e BOOLEAN should contain boolean literals
It is natural that an argument_spec with choises=BOOLEAN accepts
boolean literal (True, False) though the current implementation
allows only string or int.
2015-12-01 23:51:39 +09:00
Toshio Kuratomi
ba4e571029 Update submodule refs to go along with the StandardError change in ec2 moudles 2015-11-30 19:05:33 -08:00
Toshio Kuratomi
19d5759771 raise AnsibleAWSError instead of StandardError.
* StandardError doesn't exist in python3
* because it is the root of builtin expections, we can't catch it
  separate from the builtin exceptions
* It doesn't tell us anything about the error being thrown as it's too
  generic
2015-11-30 19:05:33 -08:00
James Cammarata
dc0fae1af7 Also make sure remote_user is defaulted correctly for delegated hosts
Fixes #13323
2015-11-30 16:15:14 -05:00
Toshio Kuratomi
e2ddc2f6ab Call the function :-)
Fixes #13330
2015-11-30 12:34:20 -08:00
James Cammarata
cc36eedf76 Ensure port is (re)set for delegated-to hosts
Fixes #13265
2015-11-30 14:40:23 -05:00
Brian Coca
b5f2c3def2 fixed typo 2015-11-30 09:20:59 -08:00
Brian Coca
eefb4931dd allow for bad stdout return from make temp dir command
fixes #13359
2015-11-30 09:19:16 -08:00
Brian Coca
005b17afec corrected become_methods class variable in winrm
This should now correctly react when using become with winrm
fixes #13331
2015-11-30 08:35:08 -08:00
James Cammarata
80db6bacc4 Make sure run_once tasks properly set variables for all active hosts
Fixes #13267
2015-11-30 11:27:05 -05:00
James Cammarata
fbc9553bd4 Use text_type instead of unicode 2015-11-30 10:33:36 -05:00
James Cammarata
c6a30f7000 Make sure the uuid in vars is string 2015-11-30 10:26:12 -05:00
James Cammarata
f926e81782 Re-implement lookup wantlist
Fixes #13285
2015-11-29 23:45:54 -05:00
James Cammarata
dfa576b037 Merge pull request #13307 from Yannig/devel_fix_big_include_vars
Fix for https://github.com/ansible/ansible/issues/13221
2015-11-29 23:14:03 -05:00
Peter Sprygada
c087160652 initial add of ssh shared module.
This ssh shared module is used for building modules that require an
interactive shell environment such as those required for connecting
to network devices
2015-11-29 21:48:52 -05:00
James Cammarata
6671d78f95 Tweak location of stats callback execution and properly relocate stats output code 2015-11-28 14:03:21 -05:00
James Cammarata
ea23159be4 Merge pull request #13348 from emonty/bug/iterate-on-none
Put in trap for args being None
2015-11-28 13:44:09 -05:00
Monty Taylor
d20e67d708 Put in trap for args being None
_normalize_old_style_args can return None. If it does, the loop
"for args in args" blows up.
2015-11-28 13:38:11 -05:00
James Cammarata
800811a15f Trigger on_stats just once, not once for each play
Fixes #13271
2015-11-28 13:37:43 -05:00
James Cammarata
a7f7f8bd29 Merge pull request #13297 from amenonsen/ssh-escalation
Explicitly accept become_success in awaiting_prompt state
2015-11-28 10:03:00 -05:00
James Cammarata
5b6162a166 Re-adding role_name/role_uuid variables 2015-11-28 09:08:24 -05:00
James Cammarata
8d9835c40b Merge pull request #13342 from Yannig/devel_fix_bomb_shell
Devel fix bomb shell
2015-11-28 09:02:12 -05:00
Yannig Perré
5227c6bb52 Do not copy variable_manager each time. Instead, keep host and local variable_manager sync.
Fix https://github.com/ansible/ansible/issues/13221
2015-11-28 14:58:33 +01:00
Yannig Perré
2fc7c8b460 More restrictive test against variable name to allow setting variable starting with _. 2015-11-28 10:35:06 +01:00
Brian Coca
fa358d9d61 avoids prompting for vars during syntax check
fixes #13319
2015-11-27 11:41:00 -08:00
Yannig Perré
2c54fb1339 Switch parameters validation after parsing in order to be more consistent between old and new style. 2015-11-26 13:33:58 +01:00
Abhijit Menon-Sen
f20e2630b0 Explicitly accept become_success in awaiting_prompt state
If we request escalation with a password, we start in expecting_prompt
state. If the escalation then succeeds without the password, i.e., the
become_success response arrives, we must explicitly move into the next
state (awaiting_escalation, which immediately goes into ready_to_send),
so that we no longer try to apply the timeout.

Otherwise, we would leak the success notification and eventually
timeout. But if the module response did arrive before the timeout
expired, the "process has already exited" test would do the right
thing by accident (which is why it didn't fail more often).

Fixes #13289
2015-11-26 06:06:01 +05:30
Toshio Kuratomi
1b743436b9 Do not double transform to unicode 2015-11-25 07:48:52 -08:00
Chrrrles Paul
157230c3e8 Merge pull request #13257 from chrrrles/utf8
fixing error with using non-ascii values for ask-sudo-password
2015-11-25 11:52:17 +09:00
Charles Paul
1e5a205389 fixing errors with utf-8 values
removing utf-8 stanza

changing cast to binary_type instead

using to_unicode
2015-11-25 11:47:25 +09:00
Brian Coca
71fe49ee3a Merge pull request #13247 from bcoca/fetch_fixes
fixes to fetch action module
2015-11-24 10:55:23 -08:00
Brian Coca
7244b5ae49 added missing : 2015-11-25 10:52:42 -08:00
Brian Coca
0ec60ac09b added missing events to base class 2015-11-25 10:49:03 -08:00
James Cammarata
70de8bc96f Fix ssh state issues by simply assuming it's never connected 2015-11-24 12:00:37 -05:00
James Cammarata
65747285a4 Properly check for prompting state when re-using ssh connection
Fixes #13278
2015-11-24 09:09:54 -05:00
James Cammarata
96fcfe45d0 Merge pull request #13264 from Yannig/devel_debug_var_list_or_dict
Allow debug var parameter to accept a list or dict.
2015-11-23 16:47:45 -05:00
Yannig Perré
5fa49a9ad8 Use to_unicode instead of str() 2015-11-23 22:04:55 +01:00
James Cammarata
679488fdb5 Merge pull request #13239 from sysreq0/devel
We should give pipes.quote() a string every time
2015-11-23 14:27:26 -05:00
James Cammarata
53e86f3130 Merge pull request #13261 from cchurch/with_items_reuse_connection
Modify task executor to reuse connections inside a loop.
2015-11-23 13:57:12 -05:00
James Cammarata
e06b107d2d Template (and include vars) PlaybookInclude paths
Fixes #13249
2015-11-23 11:54:06 -05:00
Chris Church
272778f732 Modify task executor to reuse connection inside a loop. Fix WinRM connection to set _connected properly and display when remote shell is opened/closed. Add integration test using raw + with_items. 2015-11-23 11:53:05 -05:00
Toshio Kuratomi
adf2d53fa2 Update submodule refs 2015-11-23 08:41:19 -08:00
Yannig Perré
0480b44f50 Allow debug var parameter to accept a list or dict. Fix https://github.com/ansible/ansible/issues/13252 2015-11-23 16:58:24 +01:00
Brian Coca
478c6c756a marked spot that should send per item reulsts 2015-11-22 14:44:51 -08:00
Brian Coca
fb96748d7c fixes to fetch action module
* now only runs remote checksum when needed (fixes #12290)
 * unified return points to simplify program flow
2015-11-22 09:52:34 -08:00
Toshio Kuratomi
2631a8e6e4 Update extras to fix docs build 2015-11-20 14:14:12 -08:00
Toshio Kuratomi
8fd15ae2a3 Merge pull request #13217 from ansible/more-strict-plugin-lookup
Fix non-module plugins picking up files that did not end in .py.
2015-11-20 13:50:45 -08:00
Toshio Kuratomi
77c83fd520 Commit submodule refs to the devel HEAD 2015-11-20 12:46:22 -08:00
Toshio Kuratomi
664c7980a2 Update submodule refs 2015-11-20 12:42:44 -08:00
Sebastien Couture
087dbc1ed5 We should give pipes.quote() a string every time 2015-11-20 12:58:50 -05:00
Toshio Kuratomi
591c81e95f Docker cp sets file ownership to root:root so we can't use it.
Fixes #13219
2015-11-20 07:45:00 -08:00