ansible/test/runner/lib/config.py
Matt Clay ca13e678ae Backport test infra fixes and updates to stable-2.5. (#46992)
* Fix unit test parametrize order on Python 3.5.

(cherry picked from commit 53b230ca74)

* Fix ansible-test unit test execution. (#45772)

* Fix ansible-test units requirements install.
* Run unit tests as unprivileged user under Docker.

(cherry picked from commit 379a7f4f5a)

* Run unit tests in parallel. (#45812)

(cherry picked from commit abe8e4c9e8)

* Minor fixes for unit test delegation.

(cherry picked from commit be199cfe90)

* add support for opening shell on remote Windows host (#43919)

* add support for opening shell on remote Windows host

* added arg completion and fix sanity check

* remove uneeded arg

(cherry picked from commit 6ca4ea0c1f)

* Block network access for unit tests in docker.

(cherry picked from commit 99cac99cbc)

* Make ansible-test available in the bin directory. (#45876)

(cherry picked from commit f3d1f9544b)

* Support comments in ansible-test flat files.

(cherry picked from commit 5a3000af19)

* Fix incorrect use of subprocess.CalledProcessError (#45890)

(cherry picked from commit 24dd87bd0a)

* Improve ansible-test match error handling.

(cherry picked from commit 2056c981ae)

* Improve error handling for docs-build test.

(cherry picked from commit 2148999048)

* Bug fixes and cleanup for ansible-test. (#45991)

* Remove unused imports.
* Clean up ConfigParser usage in ansible-test.
* Fix bare except statements in ansible-test.
* Miscellaneous cleanup from PyCharm inspections.
* Enable pylint no-self-use for ansible-test.
* Remove obsolete pylint ignores for Python 3.7.
* Fix shellcheck issuers under newer shellcheck.
* Use newer path for ansible-test.
* Fix issues in code-smell tests.

(cherry picked from commit ac492476e5)

* Fix integration test library search path.

This prevents tests from loading modules outside the source tree,
which could result in testing the wrong module if a system-wide
install is present, or custom modules exist.

(cherry picked from commit d603cd41fe)

* Update default container to version 1.2.0.

(cherry picked from commit d478a4c3f6)
(cherry picked from commit 21c4eb8db5)

* Fix ansible-test docker python version handling.

This removes the old name based version detection behavior and
uses versions defined in the docker completion file instead, as
the new containers do not follow the old naming scheme.

(cherry picked from commit 54937ba784)

* Reduce noise in docs-build test failures.

(cherry picked from commit 4085d01617)

* Fix ansible-test encoding issues for exceptions.

(cherry picked from commit 0d7a156319)

* Fix ansible-test multi-group smoke test handling. (#46363)

* Fix ansible-test smoke tests across groups.
* Fix ansible-test list arg defaults.
* Fix ansible-test require and exclude delegation.
* Fix detection of Windows specific changes.
* Add minimal Windows testing for Python 3.7.

(cherry picked from commit e53390b3b1)

* Use default-test-container version 1.3.0.

(cherry picked from commit 6d9be66418)

* Add file exists check in integration-aliases test.

(cherry picked from commit 33a8be9109)

* Improve ansible-test environment checking between tests. (#46459)

* Add unified diff output to environment validation.

This makes it easier to see where the environment changed.

* Compare Python interpreters by version to pip shebangs.

This helps expose cases where pip executables use a different
Python interpreter than is expected.

* Query `pip.__version__` instead of using `pip --version`.

This is a much faster way to query the pip version. It also more
closely matches how we invoke pip within ansible-test.

* Remove redundant environment scan between tests.

This reuses the environment scan from the end of the previous test
as the basis for comparison during the next test.

(cherry picked from commit 0dc7f38787)

* Add symlinks sanity test. (#46467)

* Add symlinks sanity test.
* Replace legacy test symlinks with actual content.
* Remove dir symlink from template_jinja2_latest.
* Update import test to use generated library dir.
* Fix copy test symlink setup.

(cherry picked from commit e2b6047514)

* Fix parametrize warning in unit tests.

(cherry picked from commit 1a28898a00)

* Update MANIFEST.in (#46502)

* Update MANIFEST.in:

- Remove unnecessary prune.
- Include files needed by tests.
- Exclude botmeta sanity test.

These changes permit sanity tests to pass on sdist output.
(cherry picked from commit cbb49f66ec)

* Fix unit tests which modify the source tree. (#45763)

* Fix CNOS unit test log usage.
* Use temp dir for Galaxy unit tests.
* Write to temp files in interfaces_file unit test.
* Fix log placement in netapp_e_ldap unit test.

(cherry picked from commit 0686450cae)

* Fix ansible-test custom docker image traceback.

(cherry picked from commit 712ad9ed64)

* ansible-test: Create public key creating Windows targets (#43760)

* ansible-test: Create public key creating Windows targets

* Changed to always set SSH Key for Windows hosts

(cherry picked from commit adc0efe10c)

* Fix and re-enable sts_assume_role integration tests (#46026)

* Fix the STS assume role error message assertion when the role to assume does not exist.

(cherry picked from commit 18dc928e28)

* Fix ACI unit test on Python 3.7.0.

The previous logic was only needed for pre-release versions of 3.7.

(cherry picked from commit c0bf9815c9)

* Remove placeboify from unit tests that are not calling AWS (i.e. creating a recording) (#45754)

(cherry picked from commit 2167ce6cb6)

* Update sanity test ignore entries.
2018-10-13 10:44:11 -07:00

273 lines
9.2 KiB
Python

"""Configuration classes."""
from __future__ import absolute_import, print_function
import os
import sys
from lib.util import (
CommonConfig,
is_shippable,
docker_qualify_image,
find_python,
generate_pip_command,
get_docker_completion,
)
from lib.metadata import (
Metadata,
)
class EnvironmentConfig(CommonConfig):
"""Configuration common to all commands which execute in an environment."""
def __init__(self, args, command):
"""
:type args: any
"""
super(EnvironmentConfig, self).__init__(args)
self.command = command
self.local = args.local is True
if args.tox is True or args.tox is False or args.tox is None:
self.tox = args.tox is True
self.tox_args = 0
self.python = args.python if 'python' in args else None # type: str
else:
self.tox = True
self.tox_args = 1
self.python = args.tox # type: str
self.docker = docker_qualify_image(args.docker) # type: str
self.docker_raw = args.docker # type: str
self.remote = args.remote # type: str
self.docker_privileged = args.docker_privileged if 'docker_privileged' in args else False # type: bool
self.docker_pull = args.docker_pull if 'docker_pull' in args else False # type: bool
self.docker_keep_git = args.docker_keep_git if 'docker_keep_git' in args else False # type: bool
self.docker_seccomp = args.docker_seccomp if 'docker_seccomp' in args else None # type: str
self.docker_memory = args.docker_memory if 'docker_memory' in args else None
if self.docker_seccomp is None:
self.docker_seccomp = get_docker_completion().get(self.docker_raw, {}).get('seccomp', 'default')
self.tox_sitepackages = args.tox_sitepackages # type: bool
self.remote_stage = args.remote_stage # type: str
self.remote_provider = args.remote_provider # type: str
self.remote_aws_region = args.remote_aws_region # type: str
self.remote_terminate = args.remote_terminate # type: str
if self.remote_provider == 'default':
self.remote_provider = None
self.requirements = args.requirements # type: bool
if self.python == 'default':
self.python = None
self.python_version = self.python or '.'.join(str(i) for i in sys.version_info[:2])
self.delegate = self.tox or self.docker or self.remote
self.delegate_args = [] # type: list[str]
if self.delegate:
self.requirements = True
self.inject_httptester = args.inject_httptester if 'inject_httptester' in args else False # type: bool
self.httptester = docker_qualify_image(args.httptester if 'httptester' in args else '') # type: str
@property
def python_executable(self):
"""
:rtype: str
"""
return find_python(self.python_version)
@property
def pip_command(self):
"""
:rtype: list[str]
"""
return generate_pip_command(self.python_executable)
class TestConfig(EnvironmentConfig):
"""Configuration common to all test commands."""
def __init__(self, args, command):
"""
:type args: any
:type command: str
"""
super(TestConfig, self).__init__(args, command)
self.coverage = args.coverage # type: bool
self.coverage_label = args.coverage_label # type: str
self.include = args.include or [] # type: list [str]
self.exclude = args.exclude or [] # type: list [str]
self.require = args.require or [] # type: list [str]
self.changed = args.changed # type: bool
self.tracked = args.tracked # type: bool
self.untracked = args.untracked # type: bool
self.committed = args.committed # type: bool
self.staged = args.staged # type: bool
self.unstaged = args.unstaged # type: bool
self.changed_from = args.changed_from # type: str
self.changed_path = args.changed_path # type: list [str]
self.lint = args.lint if 'lint' in args else False # type: bool
self.junit = args.junit if 'junit' in args else False # type: bool
self.failure_ok = args.failure_ok if 'failure_ok' in args else False # type: bool
self.metadata = Metadata.from_file(args.metadata) if args.metadata else Metadata()
self.metadata_path = None
class ShellConfig(EnvironmentConfig):
"""Configuration for the shell command."""
def __init__(self, args):
"""
:type args: any
"""
super(ShellConfig, self).__init__(args, 'shell')
class SanityConfig(TestConfig):
"""Configuration for the sanity command."""
def __init__(self, args):
"""
:type args: any
"""
super(SanityConfig, self).__init__(args, 'sanity')
self.test = args.test # type: list [str]
self.skip_test = args.skip_test # type: list [str]
self.list_tests = args.list_tests # type: bool
self.allow_disabled = args.allow_disabled # type: bool
if args.base_branch:
self.base_branch = args.base_branch # str
elif is_shippable():
self.base_branch = os.environ.get('BASE_BRANCH', '') # str
if self.base_branch:
self.base_branch = 'origin/%s' % self.base_branch
else:
self.base_branch = ''
class IntegrationConfig(TestConfig):
"""Configuration for the integration command."""
def __init__(self, args, command):
"""
:type args: any
:type command: str
"""
super(IntegrationConfig, self).__init__(args, command)
self.start_at = args.start_at # type: str
self.start_at_task = args.start_at_task # type: str
self.allow_destructive = args.allow_destructive # type: bool
self.allow_root = args.allow_root # type: bool
self.allow_disabled = args.allow_disabled # type: bool
self.allow_unstable = args.allow_unstable # type: bool
self.allow_unstable_changed = args.allow_unstable_changed # type: bool
self.allow_unsupported = args.allow_unsupported # type: bool
self.retry_on_error = args.retry_on_error # type: bool
self.continue_on_error = args.continue_on_error # type: bool
self.debug_strategy = args.debug_strategy # type: bool
self.changed_all_target = args.changed_all_target # type: str
self.changed_all_mode = args.changed_all_mode # type: str
self.list_targets = args.list_targets # type: bool
self.tags = args.tags
self.skip_tags = args.skip_tags
self.diff = args.diff
if self.list_targets:
self.explain = True
class PosixIntegrationConfig(IntegrationConfig):
"""Configuration for the posix integration command."""
def __init__(self, args):
"""
:type args: any
"""
super(PosixIntegrationConfig, self).__init__(args, 'integration')
class WindowsIntegrationConfig(IntegrationConfig):
"""Configuration for the windows integration command."""
def __init__(self, args):
"""
:type args: any
"""
super(WindowsIntegrationConfig, self).__init__(args, 'windows-integration')
self.windows = args.windows # type: list [str]
if self.windows:
self.allow_destructive = True
class NetworkIntegrationConfig(IntegrationConfig):
"""Configuration for the network integration command."""
def __init__(self, args):
"""
:type args: any
"""
super(NetworkIntegrationConfig, self).__init__(args, 'network-integration')
self.platform = args.platform # type: list [str]
self.inventory = args.inventory # type: str
self.testcase = args.testcase # type: str
class UnitsConfig(TestConfig):
"""Configuration for the units command."""
def __init__(self, args):
"""
:type args: any
"""
super(UnitsConfig, self).__init__(args, 'units')
self.collect_only = args.collect_only # type: bool
self.requirements_mode = args.requirements_mode if 'requirements_mode' in args else ''
if self.requirements_mode == 'only':
self.requirements = True
elif self.requirements_mode == 'skip':
self.requirements = False
class CoverageConfig(EnvironmentConfig):
"""Configuration for the coverage command."""
def __init__(self, args):
"""
:type args: any
"""
super(CoverageConfig, self).__init__(args, 'coverage')
self.group_by = frozenset(args.group_by) if 'group_by' in args and args.group_by else set() # type: frozenset [str]
self.all = args.all if 'all' in args else False # type: bool
self.stub = args.stub if 'stub' in args else False # type: bool
class CoverageReportConfig(CoverageConfig):
"""Configuration for the coverage report command."""
def __init__(self, args):
"""
:type args: any
"""
super(CoverageReportConfig, self).__init__(args)
self.show_missing = args.show_missing # type: bool
self.include = args.include # type: str
self.omit = args.omit # type: str