Commit graph

29 commits

Author SHA1 Message Date
Matt Clay
ca13e678ae Backport test infra fixes and updates to stable-2.5. (#46992)
* Fix unit test parametrize order on Python 3.5.

(cherry picked from commit 53b230ca74)

* Fix ansible-test unit test execution. (#45772)

* Fix ansible-test units requirements install.
* Run unit tests as unprivileged user under Docker.

(cherry picked from commit 379a7f4f5a)

* Run unit tests in parallel. (#45812)

(cherry picked from commit abe8e4c9e8)

* Minor fixes for unit test delegation.

(cherry picked from commit be199cfe90)

* add support for opening shell on remote Windows host (#43919)

* add support for opening shell on remote Windows host

* added arg completion and fix sanity check

* remove uneeded arg

(cherry picked from commit 6ca4ea0c1f)

* Block network access for unit tests in docker.

(cherry picked from commit 99cac99cbc)

* Make ansible-test available in the bin directory. (#45876)

(cherry picked from commit f3d1f9544b)

* Support comments in ansible-test flat files.

(cherry picked from commit 5a3000af19)

* Fix incorrect use of subprocess.CalledProcessError (#45890)

(cherry picked from commit 24dd87bd0a)

* Improve ansible-test match error handling.

(cherry picked from commit 2056c981ae)

* Improve error handling for docs-build test.

(cherry picked from commit 2148999048)

* Bug fixes and cleanup for ansible-test. (#45991)

* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.

(cherry picked from commit ac492476e5)

* Fix integration test library search path.

This prevents tests from loading modules outside the source tree,
which could result in testing the wrong module if a system-wide
install is present, or custom modules exist.

(cherry picked from commit d603cd41fe)

* Update default container to version 1.2.0.

(cherry picked from commit d478a4c3f6)
(cherry picked from commit 21c4eb8db5)

* Fix ansible-test docker python version handling.

This removes the old name based version detection behavior and
uses versions defined in the docker completion file instead, as
the new containers do not follow the old naming scheme.

(cherry picked from commit 54937ba784)

* Reduce noise in docs-build test failures.

(cherry picked from commit 4085d01617)

* Fix ansible-test encoding issues for exceptions.

(cherry picked from commit 0d7a156319)

* Fix ansible-test multi-group smoke test handling. (#46363)

* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.

(cherry picked from commit e53390b3b1)

* Use default-test-container version 1.3.0.

(cherry picked from commit 6d9be66418)

* Add file exists check in integration-aliases test.

(cherry picked from commit 33a8be9109)

* Improve ansible-test environment checking between tests. (#46459)

* Add unified diff output to environment validation.

This makes it easier to see where the environment changed.

* Compare Python interpreters by version to pip shebangs.

This helps expose cases where pip executables use a different
Python interpreter than is expected.

* Query `pip.__version__` instead of using `pip --version`.

This is a much faster way to query the pip version. It also more
closely matches how we invoke pip within ansible-test.

* Remove redundant environment scan between tests.

This reuses the environment scan from the end of the previous test
as the basis for comparison during the next test.

(cherry picked from commit 0dc7f38787)

* Add symlinks sanity test. (#46467)

* Add symlinks sanity test.
* Replace legacy test symlinks with actual content.
* Remove dir symlink from template_jinja2_latest.
* Update import test to use generated library dir.
* Fix copy test symlink setup.

(cherry picked from commit e2b6047514)

* Fix parametrize warning in unit tests.

(cherry picked from commit 1a28898a00)

* Update MANIFEST.in (#46502)

* Update MANIFEST.in:

- Remove unnecessary prune.
- Include files needed by tests.
- Exclude botmeta sanity test.

These changes permit sanity tests to pass on sdist output.
(cherry picked from commit cbb49f66ec)

* Fix unit tests which modify the source tree. (#45763)

* Fix CNOS unit test log usage.
* Use temp dir for Galaxy unit tests.
* Write to temp files in interfaces_file unit test.
* Fix log placement in netapp_e_ldap unit test.

(cherry picked from commit 0686450cae)

* Fix ansible-test custom docker image traceback.

(cherry picked from commit 712ad9ed64)

* ansible-test: Create public key creating Windows targets (#43760)

* ansible-test: Create public key creating Windows targets

* Changed to always set SSH Key for Windows hosts

(cherry picked from commit adc0efe10c)

* Fix and re-enable sts_assume_role integration tests (#46026)

* Fix the STS assume role error message assertion when the role to assume does not exist.

(cherry picked from commit 18dc928e28)

* Fix ACI unit test on Python 3.7.0.

The previous logic was only needed for pre-release versions of 3.7.

(cherry picked from commit c0bf9815c9)

* Remove placeboify from unit tests that are not calling AWS (i.e. creating a recording) (#45754)

(cherry picked from commit 2167ce6cb6)

* Update sanity test ignore entries.
2018-10-13 10:44:11 -07:00
Matt Clay
bf8f228fd5 Update Shippable integration test groups. (#43118)
* Update Shippable integration test groups.
* Update integration test group aliases.
* Rebalance AWS and Azure tests with extra group.
* Rebalance Windows tests with another group.

(cherry picked from commit 4e489d1be8)
2018-07-24 10:02:00 -07:00
Toshio Kuratomi
7b325baa22 Bkprt recursive copy fix (#40268)
* Fixes #34893 (#40166)

Fixes several bugs exposed in #34893
* Fixes relative path handling in copy so that it splits directories and
  reconstructs the correct file path
* Return failed in the proper circumstances

(cherry picked from commit ca4147f2cc)

* Add changelog for recursive copy fix

(cherry picked from commit cab0f21564)
2018-07-03 11:49:45 -07:00
Matt Clay
20ab1d2994 Move requirements into tests. (#35885)
(cherry picked from commit e7b793c8c6)
2018-05-01 09:55:56 -07:00
Toshio Kuratomi
cd7042857c Fix copy to only follow symlinks for files in the non-recursive case
Revert "**Temporary**"

This reverts commit 28b86b1148.

We don't need this now that copy has been fixed

(cherry picked from commit f332151f59)
2018-04-20 12:48:32 -07:00
Toshio Kuratomi
8a22b51755
Fix copy for implicit relative paths (#35016)
copy currently fails if you specify a destination without any directory
component.  This is because we take the dirname of the destination for
some processing and no dirname causes issues.

This corrects that by prepending "./" if there is no directory component
in dest.
2018-02-08 00:12:32 -08:00
Matt Clay
69132007c0 Revert "Move requirements into tests. (#35197)"
This reverts commit 4f6017dc16.
2018-01-23 10:54:25 -08:00
Matt Clay
4f6017dc16
Move requirements into tests. (#35197) 2018-01-23 10:08:16 -08:00
Jordan Borean
9106284c1c
copy tests: only create recursive symlink in tests (#35073) 2018-01-19 11:42:20 +10:00
Brian Coca
bbd6b8bb42 Temporary (#31677)
* allow shells to have per host options, remote_tmp

added language to shell
removed module lang setting from general as  plugins have it now
use get to avoid bad powershell plugin
more resilient tmp discovery, fall back to `pwd`
add shell to docs
fixed options for when frags are only options
added shell set ops in t_e and fixed option frags
normalize tmp dir usag4e

- pass tmpdir/tmp/temp options as env var to commands, making it default for tempfile
- adjusted ansiballz tmpdir
- default local tempfile usage to the configured local tmp
- set env temp in action

add options to powershell
shift temporary to internal envvar/params
ensure tempdir is set if we pass var
ensure basic and url use expected tempdir
ensure localhost uses local tmp
give /var/tmp priority, less perms issues
more consistent tempfile mgmt for ansiballz
made async_dir configurable
better action handling, allow for finally rm tmp
fixed tmp issue and no more tempdir in ballz
hostvarize world readable and admin users
always set shell tempdir
added comment to discourage use of exception/flow control

* Mostly revert expand_user as it's not quite working.

This was an additional feature anyhow.

Kept the use of pwd as a fallback but moved it to a second ssh
connection.  This is not optimal but getting that to work in a single
ssh connection was part of the problem holding this up.

(cherry picked from commit 395b714120522f15e4c90a346f5e8e8d79213aca)

* fixed script and other action plugins

ensure tmpdir deletion
allow for connections that don't support new options (legacy, 3rd party)
fixed tests
2018-01-15 21:15:04 -08:00
Jordan Borean
8386201242 add deprecation for stat get_md5 (#33002) 2017-11-27 18:58:55 -05:00
Matt Martz
4fe08441be Deprecate tests used as filters (#32361)
* Warn on tests used as filters

* Update docs, add aliases for tests that fit more gramatically with test syntax

* Fix rst formatting

* Add successful filter, alias of success

* Remove renamed_deprecation, it was overkill

* Make directory alias for is_dir

* Update tests to use proper jinja test syntax

* Update additional documentation, living outside of YAML files, to reflect proper jinja test syntax

* Add conversion script, porting guide updates, and changelog updates

* Update newly added uses of tests as filters

* No underscore variable

* Convert recent tests as filter changes to win_stat

* Fix some changes related to rebasing a few integration tests

* Make tests_as_filters_warning explicitly accept the name of the test, instead of inferring the name

* Add test for tests_as_filters_warning

* Update tests as filters in newly added/modified tests

* Address recent changes to several integration tests

* Address recent changes in cs_vpc
2017-11-27 17:58:08 -05:00
Brian Coca
ac9278ff0f remove action plugin only fields from 'file' calls (#31047)
* remove action plugin only fields from 'file' calls

fixes #30556

* Add a test for #30556
2017-09-29 17:13:32 -07:00
Toshio Kuratomi
bc66faa328 Add more tests for copy/file/template with harlinks 2017-08-17 18:16:17 -07:00
Mehran Kholdi
ecfa7f696d Enforce state='file' in copy module
This was causing wrong behaviour when `prev_state` was `hard`-link,
since the `file` module tried to apply the same `state` on the new
file, causing unexpected errors.

Particularly, both `overlay` and `devicemapper` storage drivers in
docker use hardlinks to share files between layers. This causes
most ansible playbooks to fail when working with files from layers
below.
2017-08-17 18:16:17 -07:00
Sam Doran
a8e4c9be7a Cause copy module to fail on empty string as source (#27975)
* Fail if an empty string is set as src for copy module

Fixes #27363

* Cleanup task formatting on copy tests

Use multi-line YAML
Add debug statements with verbosity: 1 rather than leave them in there commented out.

* Add test for empty string as source

* Do more checks in order to add more specific errors messages

Add more integration tests for the various failure scenarios.
Cleanup some syntax on existing integration test tasks.
2017-08-15 12:41:16 -04:00
Pierre-Louis Bonicoli
022d45cb52 copy: clearer parameter values (tests) 2017-08-13 08:22:22 -07:00
Pierre-Louis Bonicoli
efd17915f1 copy: clearer comments, filenames, task names (tests) 2017-08-13 08:22:22 -07:00
Pierre-Louis Bonicoli
6f47b3e9e8 copy: check non-existing one level directory too 2017-08-13 08:22:22 -07:00
Pierre-Louis Bonicoli
470989bff9 copy: check behavior related to dest creation when src is a file 2017-08-13 08:22:22 -07:00
Pilou
ade593da52 Copy module: improve tests allowing to use a managed host which isn't the controller host (#25672)
* set output_dir_expanded using module result

'path' values are expanded using 'expandvars' too

* foo.txt is located in 'files' directory

* Use 'role_path' and 'connection: local' for local paths

'{{ role_path }}/tmp' is used for generated paths

* Use local connection with local paths

/tmp/ansible-test-abs-link and /tmp/ansible-test-abs-link-dir are
defined by targets/copy/files/subdir/subdir1/ansible-test-abs-link
and targets/copy/files/subdir/subdir1/ansible-test-abs-link-dir links.

* task names: add a suffix when same name is reused

* Check that item exists before checking file mode

then error message is more explicit when item doesn't exist

* Use output_dir_expanded only when necessary

* Enforce remote_user when root is required

* Fix remote path

* Use different local & remote user

this is useful when controller and managed hosts are identical

* Checks must not expect output of tested module to be right

* Use a temporary directory on the controller

* Use sha1 & md5 filters instead of hardcoded values

* Use 'remote_dir' for directory on managed host

* Workaround tempfile error on OS X

Error was:
temp_path = tempfile.mkdtemp(prefix='ansible_')
AttributeError: 'module' object has no attribute 'mkdtemp'"
2017-07-31 22:07:22 -07:00
Toshio Kuratomi
0a2cdb2585 New tests for copy recursive with absolute paths
Absolute path trailing slash handling in absolute directories

find_needle() isn't passing a trailing slash through verbatim.  Since
copy uses that to determine if it should copy a directory or just the
files inside of it, we have to detect that and restore it after calling
find_needle()

Fixes #27439
2017-07-28 21:00:51 -07:00
Toshio Kuratomi
f86ce0975d Add a directory walker to copy
* We need a directory walker that can handle symlinks, empty directories,
  and some other odd needs.  This commit contains a directory walker that
  can do all that.  The walker returns information about the files in the
  directories that we can then use to implement different strategies for
  copying the files to the remote machines.
* Add local_follow parameter to copy that follows local symlinks (follow
  is for remote symlinks)
* Refactor the copying of files out of run into its own method
* Add new integration tests for copy

Fixes #24949
Fixes #21513
2017-07-20 08:01:29 -07:00
Toshio Kuratomi
753a3a03d0 Revert "Fix for recursive copy slowness"
This reverts commit 78ced5318f.

The fix for copy slowness did not handle circular symlinks.
2017-07-20 08:01:29 -07:00
Brian Coca
4594bee65a keep unsafe .. unsafe (#23742)
* keep unsafe .. unsafe

fixes #23734, which was broken in previous fix that allowed non string types to be templated
use new 'is_template' function vs bastardizing others
refactored clean_data to allow for arbitrary data structures to clean
fixed/removed some tests

* deal with complex data for is_template

* typos
2017-04-21 16:07:38 -04:00
Toshio Kuratomi
78ced5318f Fix for recursive copy slowness
Copy module was walking over files in subdirectories repeatedly (a
directory tree a few levels deep could bring the time spent into the
tens of minutes)

This was traced to the fix for this bug report: https://github.com/ansible/ansible/issues/13013

Fixed #13013 a different way and added an integration test to check for
regressions of #13013 as we optimize this code.

Fixes #21513
2017-04-07 12:41:42 -04:00
Matt Clay
6bbd92e422 Initial ansible-test implementation. (#18556) 2016-11-29 21:21:53 -08:00
Adrien Vergé
0e834fc9e4 Fix cosmetic problems in YAML source
This change corrects problems reported by the `yamllint` linter.

Since key duplication problems were removed in 4d48711, this commit
mainly fixes trailing spaces and extra empty lines at beginning/end of
files.
2016-11-11 14:50:57 -08:00
Matt Clay
75e4645ee7 Migrate Linux CI roles to test targets. (#17997) 2016-10-13 09:09:25 -07:00