Commit graph

4926 commits

Author SHA1 Message Date
Brian Coca
2b28cdc0dd be more tolerant with non list descriptions 2015-08-27 14:57:50 -04:00
Toshio Kuratomi
86b2982005 Merge pull request #12112 from amenonsen/vault-stdio
Implement cat-like filtering behaviour for encrypt/decrypt
2015-08-27 11:26:48 -07:00
Toshio Kuratomi
db4a96a8d6 Merge pull request #12078 from amenonsen/hash_merge
Add hash_merge and hash_merge_recursive filters with documentation
2015-08-27 11:12:49 -07:00
Abhijit Menon-Sen
b328bc023d Add a combine filter with documentation
This is based on some code from (closed) PR #7872, but reworked based on
suggestions by @abadger and the other core team members.

Closes #7872 by @darkk (hash_merge/hash_replace filters)
Closes #11153 by @telbizov (merged_dicts lookup plugin)
2015-08-27 23:29:12 +05:30
James Cammarata
2df6513f8d Version bump for v2.0.0-0.1.alpha1 and submodule updates 2015-08-27 13:43:37 -04:00
Brian Coca
6c9dc78d8c Merge pull request #12126 from amenonsen/vault-aes-deprecate
Remove deprecated and unused VaultAES encryption code
2015-08-27 13:09:28 -04:00
James Cammarata
1170a453c8 Merge pull request #12114 from ilya-epifanov/devel
fixed hostvars access in conjunction with --limit usage
2015-08-27 12:54:23 -04:00
Abhijit Menon-Sen
090cfc9e03 More helpful prompts from ansible-vault encrypt/decrypt
Now we issue a "Reading … from stdin" prompt if our input isatty(), as
gpg does. We also suppress the "x successful" confirmation message at
the end if we're part of a pipeline.

(The latter requires that we not close sys.stdout in VaultEditor, and
for symmetry we do the same for sys.stdin, though it doesn't matter in
that case.)
2015-08-27 22:04:18 +05:30
Abhijit Menon-Sen
b6de6e69a6 Also support output to stdout with no arguments
This allows "cat plaintext|ansible-vault encrypt > ciphertext".
2015-08-27 22:04:18 +05:30
Abhijit Menon-Sen
e7eebb6954 Implement cat-like filtering behaviour for encrypt/decrypt
This allows the following invocations:

    # Interactive use, like gpg
    ansible-vault encrypt --output x

    # Non-interactive, for scripting
    echo plaintext|ansible-vault encrypt --output x

    # Separate input and output files
    ansible-vault encrypt input.yml --output output.yml

    # Existing usage (in-place encryption) unchanged
    ansible-vault encrypt inout.yml

…and the analogous cases for ansible-vault decrypt as well.

In all cases, the input and output files can be '-' to read from stdin
or write to stdout. This permits sensitive data to be encrypted and
decrypted without ever hitting disk.
2015-08-27 22:04:18 +05:30
Abhijit Menon-Sen
32b38d4e29 Fix add_option indentation for consistency before adding another option 2015-08-27 22:04:18 +05:30
Abhijit Menon-Sen
8fc8bf9439 Simplify VaultEditor methods
We don't need to keep creating VaultLibs everywhere, and we don't need
to keep checking for errors because VaultLib does it already.
2015-08-27 22:04:18 +05:30
Abhijit Menon-Sen
e99395f0c0 Don't create a VaultLib in each method; do it in __init__ instead 2015-08-27 22:04:18 +05:30
Abhijit Menon-Sen
159887a6c9 Remove deprecated and unused VaultAES encryption code
Now that VaultLib always decides to use AES256 to encrypt, we don't need
this broken code any more. We need to be able to decrypt this format for
a while longer, but encryption support can be safely dropped.
2015-08-27 16:54:39 +05:30
maty0609
52e94468c9 Merge remote-tracking branch 'ansible/devel' into devel 2015-08-27 10:29:48 +01:00
maty0609
6f24e6f994 Adding support for Archlinux and Slackware in fallback
In some cases Archlinux and Slackware is not detected by
platform.dist(). This should solve the issue.
2015-08-27 10:22:02 +01:00
Brian Coca
b2bfe3502b make sure delimiter is basestring for cvsfile
fixes #12062
2015-08-26 18:38:39 -04:00
Ilya Epifanov
81bf88b6e0 fixed hostvars access in conjunction with --limit usage 2015-08-27 00:15:26 +03:00
James Cammarata
0441a7a217 Finishing off porting of chroot connection plugin 2015-08-26 16:43:06 -04:00
Toshio Kuratomi
017bd7b1cd Fix synchronize lookup of localhost info 2015-08-26 13:36:50 -07:00
James Cammarata
50448d68e1 Implement max_fail_percentage and any_errors_fatal support
Fixes #11997
2015-08-26 12:03:47 -04:00
James Cammarata
af41ba929c Add float and percent types for FieldAttributes
Also sets the max_fail_percentage value to the percent type.
2015-08-26 12:03:47 -04:00
Brian Coca
b2ae6945c4 always load vars plugins
fixes #12104
2015-08-26 11:40:53 -04:00
Travis Paul
604f825a8e Update "smart" transport to handle Sun_SSH_1.5 on SmartOS 2015-08-26 15:00:45 +00:00
Toshio Kuratomi
111c0cc204 Merge pull request #12106 from amenonsen/vault-cleanups
Vault cleanups, pass #1
2015-08-26 07:30:13 -07:00
Abhijit Menon-Sen
b84053019a Make the filename the first argument to rekey_file 2015-08-26 19:54:59 +05:30
Toshio Kuratomi
5df5a14edc Merge pull request #12101 from tquenolle/devel
Synchronize fix error
2015-08-26 07:03:13 -07:00
Abhijit Menon-Sen
20fd9224bb Pass the filename to the individual VaultEditor methods, not __init__
Now we don't have to recreate VaultEditor objects for each file, and so
on. It also paves the way towards specifying separate input and output
files later.
2015-08-26 19:17:37 +05:30
Brian Coca
82603bb2a0 avoids running abspath on None 2015-08-26 09:28:23 -04:00
Abhijit Menon-Sen
a27c5741a1 Remove inaccurate outdated comment 2015-08-26 18:31:45 +05:30
Abhijit Menon-Sen
f91ad3dabe Don't pass the cipher around so much
It's unused and unnecessary; VaultLib can decide for itself what cipher
to use when encrypting. There's no need (and no provision) for the user
to override the cipher via options, so there's no need for code to see
if that has been done either.
2015-08-26 18:31:45 +05:30
Abhijit Menon-Sen
017566a2d9 Use AES256 if the cipher is not write-whitelisted 2015-08-26 18:09:21 +05:30
Abhijit Menon-Sen
47bcdf5952 Remove incorrect copy-pasted comment 2015-08-26 18:09:21 +05:30
Thomas Quenolle
c948af3b1e Synchronize fix error
Fix the error:
 "RuntimeError: dictionary changed size during iteration"
2015-08-26 11:01:00 +02:00
James Cammarata
601a1cc6d9 Multiple fixes for include statements and blocks in general
Fixes #11981
Fixes #11995
Fixes #12039
Fixes #12077
2015-08-26 02:23:22 -04:00
Brian Coca
154754ae50 pushed module_loader to task_queue_manager so all cli's can benefit from it
also normalized -M option across all cli
fixes #12016
2015-08-25 18:14:03 -04:00
Toshio Kuratomi
d2c948dd6a Remove decrypted vault temp_file mistakenly left from patch making vault edit idempotent
This bug was introduced in commit f8bf2ba on July 27.  Hasn't gone out
in a release yet.
2015-08-25 14:51:32 -07:00
Toshio Kuratomi
56ae3a032f Merge pull request #12075 from ansible/fix-vault-unicode
Unicode and other fixes for vault
2015-08-25 14:49:15 -07:00
Toshio Kuratomi
a3fd4817ef Unicode and other fixes for vault 2015-08-25 12:43:09 -07:00
Toshio Kuratomi
16e8a7dd67 Merge pull request #11767 from amenonsen/vault-new-password-file
add option to ansible-vault to read new password from file for rekey
2015-08-25 10:15:27 -07:00
Toshio Kuratomi
156feec264 Merge pull request #11650 from objectified/feature-docker-connection
allow ansible to connect to docker containers (without using ssh)
2015-08-25 09:28:25 -07:00
Abhijit Menon-Sen
8bf0dbb7a9 Use [x:y] host ranges instead of [x-y]
This commit deprecates the earlier groupname[x-y] syntax in favour of
the inclusive groupname[x:y] syntax. It also makes the subscripting
code simpler and adds explanatory comments.

One problem addressed by the cleanup is that _enumeration_info used to
be called twice, and its results discarded the first time because of the
convoluted control flow.
2015-08-25 21:17:24 +05:30
Abhijit Menon-Sen
73f10de386 Document the behaviour of _match_one_pattern in some detail
The possibilities are complicated enough that I didn't want to make
changes without having a complete description of what it actually
accepts/matches. Note that this text documents current behaviour, not
necessarily the behaviour we want. Some of this is undocumented and may
not be intended.
2015-08-25 21:17:24 +05:30
Abhijit Menon-Sen
fa6ffa1dbd Remove & and ! pattern prefixes as early as possible
Now everything under _match_one_pattern can ignore them. This also means
that we can use the cache to return the same results for 'foo' and '!foo'.
2015-08-25 21:17:24 +05:30
Abhijit Menon-Sen
704c3815d3 Reorder functions into a logical sequence based on usage
There are no code changes; this is committed separately so as to make
the subsequent "real" diffs easier to read.
2015-08-25 21:17:24 +05:30
Richard Poole
3090a45891 add option to ansible-vault to read new password from file for rekey
The --new-vault-password-file option works the same as
--vault-password-file but applies only to rekeying (when
--vault-password-file sets the old password). Also update the manpage
to document these options more fully.
2015-08-25 21:14:49 +05:30
Brian Coca
ae91cdfc98 fixed environment inheritance 2015-08-25 10:15:32 -04:00
Brian Coca
8aa732e0a4 allow for lists, sets and dicts to default to None, now return empty type in post processing
remove defaults from inhertiable fieldattributes to allow for proper detection and override
2015-08-25 10:14:28 -04:00
Marius Gedminas
44c94328c8 Speed up execution
`if method in dir(self):` is very inefficient:

- it must construct a list object listing all the object attributes & methods
- it must then perform a O(N) linear scan of that list

Replace it with the idiomatic `if hasattr(self, method):`, which is a
O(1) expected time hash lookup.

Should fix #11981.
2015-08-25 16:07:21 +03:00
Rene Moser
3db4039ad1 cloudstack: implement general api_region support, update docs 2015-08-25 13:54:21 +02:00
objectified
b1785a0361 replace compare_versions() with distutils.version 2015-08-25 02:18:37 -04:00
objectified
c39fb43ad9 added Maintainer comment header 2015-08-25 02:06:01 -04:00
Toshio Kuratomi
1f7b0fee0a Fixes #12076 2015-08-24 19:07:04 -07:00
Brian Coca
17060f9849 remove +1 from size that was cutting off first char of copied files when a recursive dir
fixes #12055
2015-08-24 21:10:03 -04:00
Toshio Kuratomi
6e107d2f22 Comments pointing the way towards substituting cryptography for pycrypto 2015-08-24 15:50:37 -07:00
Brian Coca
16f3f8e244 now does not error out when notes are not included in module 2015-08-24 13:24:58 -04:00
objectified
d9723069c5 align exec_command() definition with local.py 2015-08-24 12:32:11 -04:00
objectified
3a5522a22c fake being connected for logging purposes 2015-08-24 12:32:11 -04:00
objectified
8f2a6a9fae use docker cp when docker >=1.8.0 2015-08-24 12:32:11 -04:00
objectified
2de773477f allow ansible to connect to docker containers 2015-08-24 12:32:11 -04:00
James Cammarata
21e421ce53 Validate required list items are not None or empty strings
Fixes #12011
2015-08-24 11:44:28 -04:00
James Cammarata
db65503778 Revert "Add PowerShell exception handling and turn on strict mode." 2015-08-23 21:09:16 -04:00
Abhijit Menon-Sen
3aedc0bca9 Don't insist on ansible-vault taking only one filename parameter
Apart from ansible-vault create, every vault subcommand is happy to deal
with multiple filenames, so we can check that there's at least one, and
make create check separately that there aren't any extra.
2015-08-23 17:52:51 -04:00
Brian Coca
f1b8323b62 fixed host/group var loading when inventory is a directory 2015-08-23 17:51:47 -04:00
Brian Coca
af06a97f17 kept vars as alias to hostvars[inventory_hostname] and avoid subtrees into themselves 2015-08-23 17:51:47 -04:00
Brian Coca
3ccfebc9f7 Merge pull request #12047 from cchurch/powershell_common_cleanup
Add PowerShell exception handling and turn on strict mode.
2015-08-23 14:08:18 -04:00
Brian Coca
a4ffa09414 Merge pull request #11880 from bcoca/configurable_squash
made squashable with_ plugin list configurable
2015-08-23 13:32:15 -04:00
Abhijit Menon-Sen
09e4eac2e5 Use rsplit(':',1) for clarity; no functional changes 2015-08-23 22:52:35 +05:30
Ryan Petrello
1886307845 Fix a parsing bug that prevents IPv6 addresses from being used with add_host
Closes #8682
2015-08-23 22:50:47 +05:30
Chris Church
4b2cdadc98 Add PowerShell exception handling and turn on strict mode.
* Add exception handling when running PowerShell modules to provide exception message and stack trace.
* Enable strict mode for all PowerShell modules and internal commands.
* Update common PowerShell code to fix strict mode errors.
* Fix an issue with Set-Attr where it would not replace an existing property if already set.
* Add tests for exception handling using modified win_ping modules.
2015-08-22 18:28:07 -04:00
Brian Coca
47d9e7ca93 Merge pull request #11984 from Alphadelta14/devel
Support any Mapping for with_dict lookup.
2015-08-22 12:39:42 -04:00
James Cammarata
0958edfc7c Submodule pointer update 2015-08-22 12:39:04 -04:00
Brian Coca
e8157eab19 now output works for both search and info 2015-08-22 02:42:21 -04:00
Brian Coca
a6c0661d21 made src more prominent 2015-08-22 02:33:17 -04:00
Brian Coca
6ffd9c3025 draft galaxy cli search
TODO: paging results
2015-08-22 02:28:27 -04:00
James Cammarata
e282309f6d Make sure the inventory restriction is not None 2015-08-21 16:08:21 -04:00
James Cammarata
635fa0757b Several var fixes
* Fixes hostvar serialization issue (#12005)
* Fixes regression in include_vars from within a role (#9498), where
  we had the precedence order for vars_cache (include_vars, set_fact)
  incorrectly before role vars.
* Fixes another bug in which vars loaded from files in the format of
  a list instead of dictionary would cause a failure.

Fixes #9498
Fixes #12005
2015-08-21 12:02:23 -04:00
Brian Coca
144da7e7d1 Merge pull request #11765 from ldx/vault_pbkdf2hmac
Use PBKDF2HMAC() from cryptography for vault keys.
2015-08-21 11:06:00 -04:00
Brian Coca
f150fe2c23 moved mandatory back to filters as it does not always return a boolean, it does an exception on failure 2015-08-21 09:54:56 -04:00
Brian Coca
2787b3acac ported consul_kv lookup from v1 2015-08-21 00:22:20 -04:00
Brian Coca
74079db8d2 ported dig lookup from v1 2015-08-21 00:22:01 -04:00
Brian Coca
9bb95b5235 Merge pull request #12031 from amenonsen/deadcode
Remove unused (copied) _before_comment method
2015-08-20 23:36:56 -04:00
Brian Coca
f6b6ed530b added file tests 2015-08-20 22:04:21 -04:00
Brian Coca
cd4a0c70b0 isnotanumber is not needed as jinja2 has builtin number test 2015-08-20 22:04:20 -04:00
Brian Coca
fe06577ac2 fixed mandatory test 2015-08-20 22:04:20 -04:00
Abhijit Menon-Sen
599ad9cb51 Remove unused (copied) _before_comment method
This was copied from inventory/ini.py, but the rewritten version doesn't
use it, and shows that it isn't needed.
2015-08-21 06:58:23 +05:30
Toshio Kuratomi
b5a078a385 update submodule refs 2015-08-20 15:41:05 -07:00
Abhijit Menon-Sen
745ecd4845 Sanitize IPv6 hostname/port handling
Now we accept IPv6 addresses _with port numbers_ only in the standard
[xxx]:NN notation (though bare IPv6 addresses may be given, as before,
and non-IPv6 addresses may also be placed in square brackets), and any
other host identifiers (IPv4/hostname/host pattern) as before, with an
optional :NN suffix.
2015-08-20 22:06:44 +05:30
Abhijit Menon-Sen
74aab6f726 Use a self._raise_error helper and avoid passing the lineno around
Based on a patch by @Richard2ndQuadrant.
2015-08-20 22:05:17 +05:30
Abhijit Menon-Sen
9133cd409c Make _parse take an array of input lines as an argument
(There's no compelling reason to do this right now, but should be parser
need to be called multiple times in future, this makes it easier.)
2015-08-20 22:05:17 +05:30
Abhijit Menon-Sen
98a1905796 Rename 'section' to 'groupname' to better reflect its purpose 2015-08-20 22:05:17 +05:30
Abhijit Menon-Sen
1284c49bd7 Rewrite the INI InventoryParser
The new code parses INI-format inventory files in a single pass using a
well-documented state machine that reports precise errors and eliminates
the duplications and inconsistencies and outright errors in the earlier
three-phase parsing code (e.g. three ways to skip comments). It is also
much easier now to follow what decisions are being taken on the basis of
the parsed data. The comments point out various potential improvements,
particularly in the area of consistent IPv6 handling.

On the ornate marble tombstone of the old code, the following
inscription is one last baffling memento from a bygone age:

-    def _before_comment(self, msg):
-        ''' what's the part of a string before a comment? '''
-        msg = msg.replace("\#","**NOT_A_COMMENT**")
-        msg = msg.split("#")[0]
-        msg = msg.replace("**NOT_A_COMMENT**","#")
-        return msg
2015-08-20 22:05:17 +05:30
Jon Hawkesworth
a46b500851 Add win_splitdrive filter for windows users 2015-08-20 13:54:54 +01:00
Toshio Kuratomi
4f32a61504 Merge pull request #10957 from feanil/feanil/retain_nonetypes
Don't convert nulls to strings.
2015-08-19 19:43:27 -07:00
Brian Coca
e8b86f448d fix issue with improper connection override in delegation 2015-08-19 20:36:08 -04:00
Feanil Patel
892e230514 Don't convert nulls to strings.
This change is similar to https://github.com/ansible/ansible/pull/10465

It extends the logic there to also support none types.  Right now if you have
a '!!null' in yaml, and that var gets passed around, it will get converted to
a string.

eg. defaults/main.yml
```
ENABLE_AWESOME_FEATURE: !!null # Yaml Null
OTHER_CONFIG:
  secret1: "so_secret"
  secret2: "even_more_secret"

CONFIG:
  hostname: "some_hostname"
  features:
    awesame_feature: "{{ ENABLE_AWESOME_FEATURE}}"
  secrets: "{{ OTHER_CONFIG }}"
```

If you output `CONFIG` to json or yaml, the feature flag would get represented in the output
as a string instead of as a null, but secrets would get represented as a dictionary.  This is
a mis-match in behaviour where some "types" are retained and others are not.  This change
should fix the issue.

I also updated the template test to test for this and made the changes to v2.

Added a changelog entry specifically for the change from empty string to null as the default.

Made the null representation configurable.

It still defaults to the python NoneType but can be overriden to be an emptystring by updating
the DEFAULT_NULL_REPRESENTATION config.
2015-08-19 18:35:07 -04:00
Brian Coca
22a69e2498 fixes to delegation code 2015-08-19 15:49:37 -04:00
Brian Coca
36cbd771a4 changed fixme to deprecated 2015-08-19 15:49:37 -04:00
Brian Coca
961bee00d5 centralized the definition of 'localhost' 2015-08-19 15:49:37 -04:00
Rene Moser
d16429b59c cloudstack: rename returns for consistency 2015-08-19 21:25:19 +02:00