Commit graph

1680 commits

Author SHA1 Message Date
Toshio Kuratomi
6a61b6d431 Add a whitelist for checking for six. Use it for digital_ocean.py 2016-10-06 10:59:02 -07:00
Toshio Kuratomi
1efe782b46 Refactor parsing of CLI args so that we can modify them in the base class
Implement tag and skip_tag handling in the CLI() class.  Change tag and
skip_tag command line options to be accepted multiple times on the CLI
and add them together rather than overwrite.

* Make it configurable whether to merge or overwrite multiple --tags arguments
* Make the base CLI class an abstractbaseclass so we can implement
  functionality in parse() but still make subclasses implement it.
* Deprecate the overwrite feature of --tags with a message that the
  default will change in 2.4 and go away in 2.5.

* Add documentation for merge_multiple_cli_flags
* Fix galaxy search so its tags argument does not conflict with generic tags
* Unit tests and more integration tests for tags
2016-10-06 10:46:58 -04:00
Toshio Kuratomi
23305540b4 Make ini parsing slightly more robust
Prior to this commit, the ini parser would fail if the inventory was
not 100% utf-8.  This commit makes this slightly more robust by
omitting full line comments from that requirement.

Fixes #17593
2016-10-04 11:24:50 -07:00
Toshio Kuratomi
08a58ae025 Fix for run_command tests now that it returns native strings 2016-10-03 18:45:28 -07:00
Toshio Kuratomi
5bc3cb278c Remove unicode escape (#17866)
* Remove unicode-escape which is not present on python3

Alternative fix for #17305

* Enable the assemble test on python3

* Fix other problems with assemble on python3
2016-10-02 22:12:51 -07:00
Matt Davis
aa0ad073b8 bugfixes to JSON junk filter, added unit/integration tests to exercise (#17834) 2016-10-02 08:03:42 -07:00
Matt Clay
21857e8618 Disable privileged mode on Shippable. (#17844) 2016-09-30 22:06:28 -07:00
Brian Coca
dfff608ceb dzdo prompt fix
fixed tests
dzdo has 2 paths now, with and w/o password
fixes #17838
2016-09-30 17:58:19 -04:00
jctanner
fff161f2f6 Smart mode for sftp+scp (#17813)
If the sftp fails, roll over to scp by default. This saves users
from having to know about the scp_if_ssh method when sftp is broken
on the remote host.
2016-09-29 17:44:54 -04:00
Adrian Likins
1f03801fcc mv test_git to 'destructive' integration tests (#17526)
The test_git role removes ~/.ssh/known_hosts currently
and that is destructive.
2016-09-29 13:46:34 -04:00
Toshio Kuratomi
4452ee86bd Turn mount test back on (#17797)
* Turn mount test back on

* Mount tests need PRIVILEGED so turn that back on

Revert "Revert "Set PRIVILEGED=true for non_destructive tests. (#17733)" (#17738)"

This reverts commit dc0fb1c212.

* Add a needs_privileged tag so that we can skip mount tests on centos6

Some containers timeout on shippable tests when run with privileged.
Unfortunately, some tests require that in order to run.  Tagging those
allows us to skip those tests on the platforms that timeout when we get
ready to run the integration test in shippable.

* Centos6 times out with PRIVILEGED set so remove that (will disable the mount tests on centos6)

* Remove false start
2016-09-28 10:52:33 -07:00
Toshio Kuratomi
efa78b4ff4 The mode that we fake from stat is checked more closely under python3.5 on El Capitan (#17794)
Python2 seems to allow any integer.  Python3.5 on Linux seems to allow
a 32 bit unsigned int.  Python3.5 on El Capitan seems to limit it to
a smaller size...  perhaps a 16 bit int.
2016-09-28 08:24:28 -07:00
Matt Clay
a265d2d77d Run test_template tests on OS X. (#17791) 2016-09-27 23:30:58 -07:00
Matt Clay
0dc3db9e75 Enable unarchive tests for OS X and FreeBSD. (#17789) 2016-09-27 21:37:24 -07:00
Adrian Likins
bba0fb3a42 Add mnt pnts with single quote to test (16855) (#17771)
This adds some test data to test_facts.py that
includes mnt points that have a single quote in
the path.

Ala, https://github.com/ansible/ansible/issues/16855

The bug was already fixed via other changes, but this is
for regression testing.
2016-09-27 20:21:25 -07:00
Matt Clay
6d78397b8b Update get_url test to use httptester. (#17787) 2016-09-27 18:11:09 -07:00
Matt Clay
d05df5fc61 Disable expect continue in ansible-core-ci. (#17737)
This will allow use of larger SSH keys.
2016-09-23 22:09:46 -07:00
Toshio Kuratomi
1f311f0739 Disable mount tests for now. Mount is buggy on too many platforms 2016-09-23 16:35:34 -07:00
Matt Clay
2e0bec293c Improve error handling for ansible-core-ci. (#17734) 2016-09-23 14:42:47 -07:00
Toshio Kuratomi
0bf92d2b5c Add tests for the mount module (#17718)
* Add tests for the mount module

* Switch from unmounted to absent...

the code for mounting always modifies fstab so we need to always modify
fstab to cleanup as well.

* Fix comments and copyright
2016-09-23 12:38:31 -07:00
Matt Clay
cf4d436e07 Remove run_tests.sh and update docs. (#17719)
Removed the obsolete test/utils/run_tests.sh script in favor of
using the newer test/utils/shippable/integration.sh instead.
2016-09-22 23:51:53 -07:00
Matt Clay
50b0d011fe Update integration.sh to copy source by default. (#17717)
Also add more documentation to the script.
2016-09-22 22:28:55 -07:00
Matt Clay
40ffd8269d Add shellcheck to code-smell checks. (#17715) 2016-09-22 16:12:10 -07:00
Adrian Likins
e11f7d2339 add a defaults for integration.sh (#17709)
* add a default test image (ansible/ansible:centos7)
* add a default test target (all)
2016-09-22 13:25:04 -07:00
Matt Clay
ada2432165 Add keep on failure option to integration.sh. (#17711) 2016-09-22 12:42:55 -07:00
Tom Melendez
97a1b7ded1 Integration Playbook for Google Cloud DNS module. (#17698) 2016-09-22 14:40:57 -04:00
Christoph
f9e49de2ef Add a test for int/float parameter type checking (#16741)
A parameter of type int should accept int and string, but not float.
A parameter of type float should accept float, int, and string.

Also reset the arguments in another test so that it runs cleanly.  This
agrees with what all the other tests are doing.
2016-09-21 20:36:14 -07:00
Toshio Kuratomi
8aa8e07d13 Remove _load_hosts() from Play initialization as it's no longer needed and it breaks using extra_vars defining a list for hosts (#17699)
Thanks to @jimi-c for the solution

Fixes #16583
2016-09-21 17:25:54 -07:00
Matt Clay
3874e653c1 Remove tests for deprecated su and sudo. (#17697) 2016-09-21 15:12:26 -07:00
Tom Melendez
0e85f88f8e test_gce_mig integration test playbook (#17249)
Includes:
* Tests to create, delete, resize, recreate and configure an Autoscaler
* Tests for parameters only.  Use TEST_FLAGS="--tags 'param-check'" to execute only these tests.

Requires:
* An instance template to be configured in your project.
2016-09-21 17:48:09 -04:00
Matt Clay
39c5c558f8 Switch tests from sudo to become. (#17694) 2016-09-21 13:58:35 -07:00
Toshio Kuratomi
7889e1ffc1 Additional tests for check mode idempotence (#17679) 2016-09-20 17:56:07 -07:00
Matt Clay
e0ad02c6dd Remove test for deprecated first_available_file. (#17674) 2016-09-20 13:45:57 -07:00
James Cammarata
1b54d3b6dc Merge branch 'threaded_receiver' into devel 2016-09-20 09:18:26 -05:00
Rob Cutmore
2716fe4362 Add test for checking pip package in check mode (#17360) 2016-09-20 07:07:27 -07:00
Toshio Kuratomi
5e9a8d9202 Fix password lookup py3 plus alikins unittest additions refactoring (#17626)
* Improve unit testing of 'password' lookup

The tests showed some UnicodeErrors for the
cases where the 'chars' param include unicode,
causing the 'getattr(string, c, c)' to fail.
So the candidate char generation code try/excepts
UnicodeErrors there now.

Some refactoring of the password.py module to make
it easier to test, and some new tests that cover more
of the password and salt generation.

* More refactoring and fixes.

* manual merge of text enc fixes from pr17475

* moving methods to module scope

* more refactoring

* A few more text encoding fixes/merges

* remove now unused code

* Add test cases and data for _gen_candidate_chars

* more test coverage for password lookup

* wip

* More text encoding fixes and test coverage

* cleanups

* reenable text_type assert

* Remove unneeded conditional in _random_password

* Add docstring for _gen_candidate_chars

* remove redundant to_text and list comphenesion

* Move set of 'chars' default in _random_password

on py2, C.DEFAULT_PASSWORD_CHARS is a regular str
type, so the assert here fails. Move setting the
default into the method and to_text(DEFAULT_PASSWORD_CHARS)
if it's needed.

* combine _random_password and _gen_password

* s/_create_password_file/_create_password_file_dir

* native strings for exception msgs

* move password to_text to _read_password_file

* move to_bytes(content) to _write_password_file

* add more test assertions about genned pw's

* Some cleanups to alikins and abadger's password lookup refactoring:

* Make DEFAULT_PASSWORD_CHARS into a text string in constants.py
  - Move this into the nonconfigurable section of constants.
* Make utils.encrypt.do_encrypt() return a text string because all the
  hashes in passlib should be returning ascii-only strings and they are
  text strings in python3.
* Make the split up of functions more sane:
   - Don't split such that conditionals have to occur in two separate functions.
   - Don't go overboard: Good to split file system manipulation from parsing
     but we don't need to do every file manipulation in a separate
     function.
  - Don't split so that creation of the password store happens in two
    parts.
  - Don't split in such a way that no decisions are made in run.
* Organize functions by when it gets called from run().
* Run all potential characters through the gen_candidate_chars function
  because it does both normalization and validation.
* docstrings for functions
* Change when we store salt slightly.  Store it whenever it was already
  present in the file as well as when encrypt is requested.  This will
  head of potential idempotence bugs where a user has two playbook tasks
  using the same password and in one they need it encrypted but in the
  other they need it plaintext.
* Reorganize tests to follow the order of the functions so it's easier
  to figure out if/where a function has been tested.
* Add tests for the functions that read and write the password file.
* Add tests of run() when the password has already been created.
* Test coverage currently at 100%
2016-09-19 11:37:57 -07:00
cinerama70
02cec7dca9 Add os_family to test_distribution_version (#17620)
As suggested in feedback on
https://github.com/ansible/ansible/pull/17575, add
os_family to test_distribution_version. Add the
correct os_family to the existing testcase data
entries.

Also add os_family to the output of
gen_distribution_version_testcase.py so any new
generated entries will contain this data.
2016-09-17 23:27:35 +02:00
James Cammarata
5a57c66e3c Moving result reading to a background thread 2016-09-17 08:12:52 -05:00
Steve Gargan
a06da86d52 update consul tests to reflect fixes to the core and session modules (#17451) 2016-09-16 14:58:17 -07:00
Tom Melendez
91a30e59e4 Changes to enable make gce to run. Added sys import so libcloud error is displayed; renamed credentials keys in template file so they work properly with gce_credentials.py. (#16607) 2016-09-16 10:02:18 -04:00
Toshio Kuratomi
8af8eec789 Merge pull request #17590 from abadger/vault-fixes
Vault fixes
2016-09-15 18:39:34 -07:00
Matt Clay
355b29573c Use default on undefined vars in tests. (#17601) 2016-09-15 16:07:55 -07:00
Toshio Kuratomi
e70066a6f7 Many Cleanups to vault
* Make is_encrypted_file handle both files opened in text and binary mode
  On python3, by default files are opened in text mode.  Since we know
  the encoding of vault files (and especially the header which is the
  first set of bytes) we can decide whether the file is an encrypted
  vault file in either case.
* Fix is_encrypted_file not resetting the file position
* Update is_encrypted_file to check that all the data in the file is ascii
* For is_encrypted_file(), add start_pos and count parameters
  This allows callers to specify reading vaulttext from the middle of
  a file if necessary.
* Combine VaultLib.encrypt() and VaultLib.encrypt_bytestring()
* Change vault's is_encrypted() to take either text or byte strings and to return False if any part of the data is non-ascii.
* Remove unnecessary use of six.b
* Vault Cipher: mark a few methods as private.
* VaultAES256._is_equal throws a TypeError if given non byte strings
* Make VaultAES256 methods that don't need self staticmethods and classmethods
* Mark VaultAES and is_encrypted as deprecated
* Get rid of VaultFile (unused and feature implemented in a different way)
* Normalize variable and parameter names on plaintext, ciphertext, vaulttext
* Normalize variable and parameter names on "b_" prefix when dealing with bytes
* Test changes:
  * Remove redundant tests( both checking the same byte string)
  * Fix use of format string without format operator
  * Enable vault editor tests on python3
  * Initialize the vault_cipher for VaultAES256 testing in setUp()
  * Make assertTrue and assertFalse take the actual method calls for
    better error messages.
  * Test that non-ascii byte strings compare correctly.
  * Test that unicode strings and ints raise TypeError

* Test-specific:
  * Removed test_methods_exist().  We only have one VaultLib so the
    implementation is the assurance that the methods exist. (Can use an abc for
    this if it changes).
  * Add tests for both byte string and text string input where the API takes either.
  * Convert "assert" to unittest assert functions or add a custom message where
    that will make failures easier to debug.
  * Move instantiating the VaultLib into setUp().
2016-09-15 15:22:06 -07:00
Toshio Kuratomi
0139298786 Have unittests import mock from ansible.compat so they work on py3 without mock installed from pypi 2016-09-15 15:21:17 -07:00
James Cammarata
dfb1c0647e Revert "Move queuing tasks to a background thread"
This reverts commit b71957d6e6.
2016-09-15 17:00:06 -05:00
Matt Clay
16baef1f72 Do not use bare vars in integration tests. (#17599) 2016-09-15 17:41:54 -04:00
Matt Davis
f497d771c8 win_shell/win_command changes + tests (#17557) 2016-09-15 11:25:56 -07:00
Matt Clay
7b3cbafbe9 Add python 3 awareness to apt module tests. (#17574)
Also remove unnecessary dependency from ubuntu1604py3 Dockerfile.
2016-09-14 15:38:27 -07:00
Matt Clay
bcb49fecb5 Add tool for downloading Shippable logs. 2016-09-14 15:30:46 -07:00
Toshio Kuratomi
f4cd1c6321 Fix galaxy's parsing of the command line. (#17569)
Also make the parsing of the action in both galaxy and vault more
robust.

Fixes #17534
May Fix #17563
2016-09-14 11:49:54 -07:00