diff --git a/docs/docsite/rst/dev_guide/testing_integration.rst b/docs/docsite/rst/dev_guide/testing_integration.rst index 2ac896248b..101e3c5673 100644 --- a/docs/docsite/rst/dev_guide/testing_integration.rst +++ b/docs/docsite/rst/dev_guide/testing_integration.rst @@ -184,93 +184,50 @@ privileges. Network Tests ============= -This page details the specifics around testing Ansible Networking modules. - - -.. important:: Network testing requirements for Ansible 2.4 - - Starting with Ansible 2.4, all network modules MUST include corresponding unit tests to defend functionality. - The unit tests must be added in the same PR that includes the new network module, or extends functionality. - Integration tests, although not required, are a welcome addition. - How to do this is explained in the rest of this document. - - -Network integration tests can be ran by doing:: - - cd test/integration - ANSIBLE_ROLES_PATH=targets ansible-playbook network-all.yaml - - -.. note:: - - * To run the network tests you will need a number of test machines and suitably configured inventory file. A sample is included in ``test/integration/inventory.network`` - * As with the rest of the integration tests, they can be found grouped by module in ``test/integration/targets/MODULENAME/`` - -To filter a set of test cases set ``limit_to`` to the name of the group, generally this is the name of the module:: - - ANSIBLE_ROLES_PATH=targets ansible-playbook -i inventory.network network-all.yaml -e "limit_to=eos_command" - - -To filter a singular test case set the tags options to eapi or cli, set limit_to to the test group, -and test_cases to the name of the test:: - - ANSIBLE_ROLES_PATH=targets ansible-playbook -i inventory.network network-all.yaml --tags="cli" -e "limit_to=eos_command test_case=notequal" - - +Starting with Ansible 2.4, all network modules MUST include unit tests that cover all functionality. You must add unit tests for each new network module and for each added feature. Please submit the unit tests and the code in a single PR. Integration tests are also strongly encouraged. Writing network integration tests --------------------------------- -Test cases are added to roles based on the module being testing. Test cases -should include both cli and API test cases. Cli test cases should be -added to ``test/integration/targets/modulename/tests/cli`` and API tests should be added to -``test/integration/targets/modulename/tests/eapi``, or ``nxapi``. - -In addition to positive testing, negative tests are required to ensure user friendly warnings & errors are generated, rather than backtraces, for example: - -.. code-block: yaml - - - name: test invalid subset (foobar) - eos_facts: - provider: "{{ cli }}" - gather_subset: - - "foobar" - register: result - ignore_errors: true - - - assert: - that: - # Failures shouldn't return changes - - "result.changed == false" - # It's a failure - - "result.failed == true" - # Sensible Failure message - - "'Subset must be one of' in result.msg" +For guidance on writing network test see the `adding tests for Network modules guide `_. -Conventions -``````````` +Running network integration tests locally +----------------------------------------- -- Each test case should generally follow the pattern: +Ansible uses Shippable to run an integration test suite on every PR, including new tests introduced by that PR. To find and fix problems in network modules, run the network integration test locally before you submit a PR. - setup —> test —> assert —> test again (idempotent) —> assert —> teardown (if needed) -> done +To run the network integration tests, use a command in the form:: - This keeps test playbooks from becoming monolithic and difficult to - troubleshoot. + ansible-test network-integration --inventory /path/to/inventory tests_to_run -- Include a name for each task that is not an assertion. (It's OK to add names - to assertions too. But to make it easy to identify the broken task within a failed - test, at least provide a helpful name for each task.) +First, define a network inventory file:: -- Files containing test cases must end in `.yaml` + cd test/integration + cp inventory.network.template inventory.networking + ${EDITOR:-vi} inventory.networking + # Add in machines for the platform(s) you wish to test + +To run all Network tests for a particular platform:: + + ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking vyos_.* + +This example will run against all vyos modules. Note that ``vyos_.*`` is a regex match, not a bash wildcard - include the `.` if you modify this example. -Adding a new Network Platform -````````````````````````````` +To run integration tests for a specific module:: + + ansible-test network-integration --inventory /path/to/ansible/test/integration/inventory.networking vyos_vlan + + +For more options:: + + ansible-test network-integration --help + +If you need additional help or feedback, reach out in ``#ansible-network`` on Freenode. -A top level playbook is required such as ``ansible/test/integration/eos.yaml`` which needs to be references by ``ansible/test/integration/network-all.yaml`` Where to find out more ====================== -If you'd like to know more about the plans for improving testing Ansible then why not join the `Testing Working Group `_. +If you'd like to know more about the plans for improving testing Ansible, join the `Testing Working Group `_. diff --git a/test/integration/asa.yaml b/test/integration/asa.yaml deleted file mode 100644 index 1538025cd4..0000000000 --- a/test/integration/asa.yaml +++ /dev/null @@ -1,42 +0,0 @@ ---- -- hosts: asa - gather_facts: no - connection: local - - vars: - limit_to: "*" - debug: false - -# Run the tests within blocks allows the next module to be tested if the previous one fails. -# This is done to allow https://github.com/ansible/dci-partner-ansible/ to run the full set of tests. - - - tasks: - - set_fact: - test_failed: false - - block: - - include_role: - name: asa_command - when: "limit_to in ['*', 'asa_command']" - rescue: - - set_fact: test_failed=true - - - block: - - include_role: - name: asa_config - when: "limit_to in ['*', 'asa_config']" - rescue: - - set_fact: test_failed=true - - - block: - - include_role: - name: asa_acl - when: "limit_to in ['*', 'asa_acl']" - rescue: - - set_fact: test_failed=true - -########### - - name: Has any previous test failed? - fail: - msg: "One or more tests failed, check log for details" - when: test_failed \ No newline at end of file diff --git a/test/integration/eos.yaml b/test/integration/eos.yaml deleted file mode 100644 index 3b9385e237..0000000000 --- a/test/integration/eos.yaml +++ /dev/null @@ -1,158 +0,0 @@ ---- -- hosts: eos - gather_facts: no - connection: local - - vars: - limit_to: "*" - debug: false - -# Run the tests within blocks allows the next module to be tested if the previous one fails. -# This is done to allow https://github.com/ansible/dci-partner-ansible/ to run the full set of tests. - - - tasks: - - set_fact: - test_failed: false - failed_modules: [] - - - block: - - include_role: - name: eos_banner - when: "limit_to in ['*', 'eos_banner']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_banner' ]" - test_failed: true - - block: - - include_role: - name: eos_command - when: "limit_to in ['*', 'eos_command']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_command' ]" - test_failed: true - - block: - - include_role: - name: eos_config - when: "limit_to in ['*', 'eos_config']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_config' ]" - test_failed: true - - - block: - - include_role: - name: eos_facts - when: "limit_to in ['*', 'eos_facts']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_facts' ]" - test_failed: true - - - block: - - include_role: - name: eos_eapi - when: "limit_to in ['*', 'eos_eapi']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_eapi' ]" - test_failed: true - - - block: - - include_role: - name: eos_system - when: "limit_to in ['*', 'eos_system']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_system' ]" - test_failed: true - - - block: - - include_role: - name: eos_user - when: "limit_to in ['*', 'eos_user']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_user' ]" - test_failed: true - - - block: - - include_role: - name: eos_vlan - when: "limit_to in ['*', 'eos_vlan']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_vlan' ]" - test_failed: true - - - block: - - include_role: - name: eos_vrf - when: "limit_to in ['*', 'eos_vrf']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_vrf' ]" - test_failed: true - - - block: - - include_role: - name: eos_logging - when: "limit_to in ['*', 'eos_logging']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_logging' ]" - test_failed: true - - - block: - - include_role: - name: eos_static_route - when: "limit_to in ['*', 'eos_static_route']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_static_route' ]" - test_failed: true - - - block: - - include_role: - name: eos_linkagg - when: "limit_to in ['*', 'eos_linkagg']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_linkagg' ]" - test_failed: true - - - block: - - include_role: - name: eos_l3_interface - when: "limit_to in ['*', 'eos_l3_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_l3_interface' ]" - test_failed: true - - - block: - - include_role: - name: eos_interface - when: "limit_to in ['*', 'eos_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_interface' ]" - test_failed: true - - - block: - - include_role: - name: eos_lldp - when: "limit_to in ['*', 'eos_lldp']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'eos_lldp' ]" - test_failed: true -########### - - debug: var=failed_modules - when: test_failed - - - name: Has any previous test failed? - fail: - msg: "One or more tests failed, check log for details" - when: test_failed diff --git a/test/integration/inventory.network b/test/integration/inventory.network deleted file mode 100644 index fd0250115a..0000000000 --- a/test/integration/inventory.network +++ /dev/null @@ -1,44 +0,0 @@ -# Inventory file used by networking tests, see README.md for more information - -[all:vars] -ansible_python_interpreter=python - -[eos] -#veos-dut-01 -veos01 ansible_network_os=eos - - -[nxos] -nxos01 ansible_network_os=nxos - -[iosxr] -iosxr01 ansible_network_os=iosxr - -[ios] -ios01 ansible_network_os=ios -#csr01 - -[junos] -vsrx01 ansible_network_os=junos - -[cumulus] -clvx01 - -[vyos] -vyos02 ansible_network_os=vyos - -[ops] -ops01 - -[asa] -asa01 - -[platform_agnostic:children] -ios -iosxr -eos -junos -vyos -nxos - -# vim: nospell filetype=dosini diff --git a/test/integration/inventory.networking.template b/test/integration/inventory.networking.template index 12e48b830c..004db95474 100644 --- a/test/integration/inventory.networking.template +++ b/test/integration/inventory.networking.template @@ -12,3 +12,10 @@ [@OS] @NAME ansible_connection="local" ansible_host=@HOST ansible_network_os="@OS" ansible_user="@USER" ansible_ssh_private_key_file="@KEY_FILE" + + +### +# Example +# +# [vyos] +# vyos01.example.net ansible_connection=local ansible_network_os="vyos" ansible_user=admin ansible_ssh_pass=mypassword diff --git a/test/integration/ios.yaml b/test/integration/ios.yaml deleted file mode 100644 index 8a3521a688..0000000000 --- a/test/integration/ios.yaml +++ /dev/null @@ -1,150 +0,0 @@ ---- -- hosts: ios - gather_facts: no - connection: local - - vars: - limit_to: "*" - debug: false -# Run the tests within blocks allows the next module to be tested if the previous one fails. -# This is done to allow https://github.com/ansible/dci-partner-ansible/ to run the full set of tests. - - - tasks: - - set_fact: - test_failed: false - failed_modules: [] - - block: - - include_role: - name: ios_banner - when: "limit_to in ['*', 'ios_banner']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_banner' ]" - test_failed: true - - block: - - include_role: - name: ios_command - when: "limit_to in ['*', 'ios_command']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_command' ]" - test_failed: true - - - block: - - include_role: - name: ios_config - when: "limit_to in ['*', 'ios_config']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_config' ]" - test_failed: true - - - block: - - include_role: - name: ios_facts - when: "limit_to in ['*', 'ios_facts']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_facts' ]" - test_failed: true - - - block: - - include_role: - name: ios_system - when: "limit_to in ['*', 'ios_system']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_system' ]" - test_failed: true - - - block: - - include_role: - name: ios_user - when: "limit_to in ['*', 'ios_user']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_user' ]" - test_failed: true - - - block: - - include_role: - name: ios_static_route - when: "limit_to in ['*', 'ios_static_route']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_static_route' ]" - test_failed: true - - - block: - - include_role: - name: ios_logging - when: "limit_to in ['*', 'ios_logging']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_logging' ]" - test_failed: true - - - block: - - include_role: - name: ios_interface - when: "limit_to in ['*', 'ios_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_logging' ]" - test_failed: true - - - block: - - include_role: - name: ios_ping - when: "limit_to in ['*', 'ios_ping']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_ping' ]" - test_failed: true - - - block: - - include_role: - name: ios_l3_interface - when: "limit_to in ['*', 'ios_l3_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_l3_interface' ]" - test_failed: true - - - block: - - include_role: - name: ios_lldp - when: "limit_to in ['*', 'ios_lldp']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_lldp' ]" - test_failed: true - - - block: - - include_role: - name: ios_vlan - when: "limit_to in ['*', 'ios_vlan']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_vlan' ]" - test_failed: true - - - block: - - include_role: - name: ios_linkagg - when: "limit_to in ['*', 'ios_linkagg']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'ios_linkagg' ]" - test_failed: true - - -########### - - debug: var=failed_modules - when: test_failed - - - name: Has any previous test failed? - fail: - msg: "One or more tests failed, check log for details" - when: test_failed diff --git a/test/integration/iosxr.yaml b/test/integration/iosxr.yaml deleted file mode 100644 index d75005a29a..0000000000 --- a/test/integration/iosxr.yaml +++ /dev/null @@ -1,97 +0,0 @@ ---- -- hosts: iosxr - gather_facts: no - connection: local - - vars: - limit_to: "*" - debug: false - -# Run the tests within blocks allows the next module to be tested if the previous one fails. -# This is done to allow https://github.com/ansible/dci-partner-ansible/ to run the full set of tests. - - - tasks: - - set_fact: - test_failed: false - failed_modules: [] - - block: - - include_role: - name: iosxr_command - when: "limit_to in ['*', 'iosxr_command']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'iosxr_command' ]" - test_failed: true - - - block: - - include_role: - name: iosxr_config - when: "limit_to in ['*', 'iosxr_config']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'iosxr_config' ]" - test_failed: true - - - block: - - include_role: - name: iosxr_facts - when: "limit_to in ['*', 'iosxr_facts']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'iosxr_facts' ]" - test_failed: true - - - block: - - include_role: - name: iosxr_system - when: "limit_to in ['*', 'iosxr_system']" - rescue: - - set_fact: - test_failed: true - failed_modules: "{{ failed_modules }} + [ 'iosxr_system' ]" - - - block: - - include_role: - name: iosxr_user - when: "limit_to in ['*', 'iosxr_user']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'iosxr_user' ]" - test_failed: true - - - block: - - include_role: - name: iosxr_banner - when: "limit_to in ['*', 'iosxr_banner']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'iosxr_banner' ]" - test_failed: true - - - block: - - include_role: - name: iosxr_logging - when: "limit_to in ['*', 'iosxr_logging']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'iosxr_logging' ]" - test_failed: true - - - block: - - include_role: - name: iosxr_interface - when: "limit_to in ['*', 'iosxr_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'iosxr_interface' ]" - test_failed: true - -########### - - debug: var=failed_modules - when: test_failed - - - name: Has any previous test failed? - fail: - msg: "One or more tests failed, check log for details" - when: test_failed diff --git a/test/integration/junos.yaml b/test/integration/junos.yaml deleted file mode 100644 index c6e32bcf8b..0000000000 --- a/test/integration/junos.yaml +++ /dev/null @@ -1,186 +0,0 @@ ---- -- hosts: junos - gather_facts: no - connection: local - - vars: - limit_to: "*" - debug: false - - -# Run the tests within blocks allows the next module to be tested if the previous one fails. -# This is done to allow https://github.com/ansible/dci-partner-ansible/ to run the full set of tests. - - - tasks: - - set_fact: - test_failed: false - failed_modules: [] - - block: - - include_role: - name: junos_command - when: "limit_to in ['*', 'junos_command']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_command' ]" - test_failed: true - - - block: - - include_role: - name: junos_config - when: "limit_to in ['*', 'junos_config']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_config' ]" - test_failed: true - - - block: - - include_role: - name: junos_facts - when: "limit_to in ['*', 'junos_facts']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_facts' ]" - test_failed: true - - - block: - - include_role: - name: junos_netconf - when: "limit_to in ['*', 'junos_netconf']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_netconf' ]" - test_failed: true - - - block: - - include_role: - name: junos_rpc - when: "limit_to in ['*', 'junos_rpc']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_rpc' ]" - test_failed: true - - - block: - - include_role: - name: junos_vlan - when: "limit_to in ['*', 'junos_vlan']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_vlan' ]" - test_failed: true - - - block: - - include_role: - name: junos_interface - when: "limit_to in ['*', 'junos_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_interface' ]" - test_failed: true - - block: - - include_role: - name: junos_banner - when: "limit_to in ['*', 'junos_banner']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_banner' ]" - test_failed: true - - - block: - - include_role: - name: junos_system - when: "limit_to in ['*', 'junos_system']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_system' ]" - test_failed: true - - block: - - include_role: - name: junos_logging - when: "limit_to in ['*', 'junos_logging']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_logging' ]" - test_failed: true - - - block: - - include_role: - name: junos_user - when: "limit_to in ['*', 'junos_user']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_user' ]" - test_failed: true - - - block: - - include_role: - name: junos_static_route - when: "limit_to in ['*', 'junos_static_route']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_static_route' ]" - test_failed: true - - - block: - - include_role: - name: junos_linkagg - when: "limit_to in ['*', 'junos_linkagg']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_linkagg' ]" - test_failed: true - - - block: - - include_role: - name: junos_l3_interface - when: "limit_to in ['*', 'junos_l3_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_l3_interface' ]" - test_failed: true - - - block: - - include_role: - name: junos_lldp - when: "limit_to in ['*', 'junos_lldp']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_lldp' ]" - test_failed: true - - - block: - - include_role: - name: junos_lldp_interface - when: "limit_to in ['*', 'junos_lldp_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_lldp_interface' ]" - test_failed: true - - - block: - - include_role: - name: junos_vrf - when: "limit_to in ['*', 'junos_vrf']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_vrf' ]" - test_failed: true - - - block: - - include_role: - name: junos_l2_interface - when: "limit_to in ['*', 'junos_l2_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'junos_l2_interface' ]" - test_failed: true - -########### - - debug: var=failed_modules - when: test_failed - - - name: Has any previous test failed? - fail: - msg: "One or more tests failed, check log for details" - when: test_failed diff --git a/test/integration/network-all.yaml b/test/integration/network-all.yaml deleted file mode 100644 index 354cdf7d0b..0000000000 --- a/test/integration/network-all.yaml +++ /dev/null @@ -1,14 +0,0 @@ ---- -- { include: asa.yaml } -- { include: eos.yaml } -- { include: ios.yaml } -- { include: iosxr.yaml } -- { include: nxos.yaml } -- { include: junos.yaml } -- { include: vyos.yaml } -- { include: ops.yaml } -- { include: ovs.yaml } -- { include: dellos10.yaml } -- { include: dellos9.yaml } -- { include: dellos6.yaml } -- { include: nuage.yaml } diff --git a/test/integration/ops.yaml b/test/integration/ops.yaml deleted file mode 100644 index 3c656e2af0..0000000000 --- a/test/integration/ops.yaml +++ /dev/null @@ -1,29 +0,0 @@ ---- -- hosts: ops - gather_facts: no - connection: local - - vars: - limit_to: "*" - debug: false - -# Run the tests within blocks allows the next module to be tested if the previous one fails. -# This is done to allow https://github.com/ansible/dci-partner-ansible/ to run the full set of tests. - - - tasks: - - set_fact: - test_failed: false - - block: - - include_role: - name: ops_command - when: "limit_to in ['*', 'ops_command']" - rescue: - - set_fact: test_failed=true - - -########### - - name: Has any previous test failed? - fail: - msg: "One or more tests failed, check log for details" - when: test_failed \ No newline at end of file diff --git a/test/integration/vyos.yaml b/test/integration/vyos.yaml deleted file mode 100644 index e3ca121bc7..0000000000 --- a/test/integration/vyos.yaml +++ /dev/null @@ -1,142 +0,0 @@ ---- -- hosts: vyos - gather_facts: no - connection: local - - vars: - limit_to: "*" - debug: false - -# Run the tests within blocks allows the next module to be tested if the previous one fails. -# This is done to allow https://github.com/ansible/dci-partner-ansible/ to run the full set of tests. - - - tasks: - - set_fact: - test_failed: false - failed_modules: [] - - block: - - include_role: - name: vyos_command - when: "limit_to in ['*', 'vyos_command']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_command' ]" - test_failed: true - - - block: - - include_role: - name: vyos_config - when: "limit_to in ['*', 'vyos_config']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_config' ]" - test_failed: true - - - block: - - include_role: - name: vyos_user - when: "limit_to in ['*', 'vyos_user']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_user' ]" - test_failed: true - - - block: - - include_role: - name: vyos_banner - when: "limit_to in ['*', 'vyos_banner']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_banner' ]" - test_failed: true - - - block: - - include_role: - name: vyos_linkagg - when: "limit_to in ['*', 'vyos_linkagg']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_linkagg' ]" - test_failed: true - - - block: - - include_role: - name: vyos_static_route - when: "limit_to in ['*', 'vyos_static_route']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_static_route' ]" - test_failed: true - - - block: - - include_role: - name: vyos_l3_interface - when: "limit_to in ['*', 'vyos_l3_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_l3_interface' ]" - test_failed: true - - - block: - - include_role: - name: vyos_logging - when: "limit_to in ['*', 'vyos_logging']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_logging' ]" - test_failed: true - - - block: - - include_role: - name: vyos_lldp - when: "limit_to in ['*', 'vyos_lldp']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_lldp' ]" - test_failed: true - - - block: - - include_role: - name: vyos_lldp_interface - when: "limit_to in ['*', 'vyos_lldp_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_lldp_interface' ]" - test_failed: true - - - block: - - include_role: - name: vyos_interface - when: "limit_to in ['*', 'vyos_interface']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_interface' ]" - test_failed: true - - - block: - - include_role: - name: vyos_facts - when: "limit_to in ['*', 'vyos_facts']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_facts' ]" - test_failed: true - - - block: - - include_role: - name: vyos_vlan - when: "limit_to in ['*', 'vyos_vlan']" - rescue: - - set_fact: - failed_modules: "{{ failed_modules }} + [ 'vyos_vlan' ]" - test_failed: true - -########### - - debug: var=failed_modules - when: test_failed - - - name: Has any previous test failed? - fail: - msg: "One or more tests failed, check log for details" - when: test_failed