Since commit b27795a03e, each section of the test suite starts
and ends with a clean Podman state. This includes removing all images
from the local containers storage. Therefore, the images get downloaded
multiple times during the course of the test suite.
This commit restores the earlier behaviour where the images would get
downloaded only once, by copying them to separate directories outside
the local containers storage and then restoring them when the tests
are run.
https://github.com/containers/toolbox/pull/517https://github.com/containers/toolbox/pull/704
The bats-support[0] and bats-assert[1] libraries extend the
capabilities of bats[2]. Mainly, bats-assert is very useful for clean
checking of values/outputs/return codes.
Apart from updating the cases to use the libraries, the test cases have
been restructured in a way that they don't depend on each other anymore.
This required major changes in the helpers.bats file.
Overall, the tests are cleaner to read and easier to extend due to the
test cases being independent.
Some slight changes were made to the test cases themselves. Should not
alter their final behaviour.
There will be a follow up commit that will take care of downloading of
the tested images locally and caching them using Skopeo to speedup the
tests and try to resolve network problems when pulling the images that
we experienced in the past.
[0] https://github.com/bats-core/bats-support
[1] https://github.com/bats-core/bats-assert
[2] https://github.com/bats-core/bats-core
The Ansible playbooks are small enough as they are. Splitting things
across too many files makes it harder to remember which file does what.
https://github.com/containers/toolbox/pull/653
The system tests for Fedora 33 were failing:
not ok 21 Remove all images (2 should be present; --force should not
be necessary)
# (from function `is' in file test/system/helpers.bash, line 287,
# in test file test/system/302-rmi.bats, line 7)
# `is "$output" "" "The output should be empty"' failed
# $ /usr/local/bin/toolbox rmi --all
# Error: image
3ac100502d2123aff1cf6314760c7a89c55108b8de6ea3c10ddc79a1479f0fca
has dependent children
# Error: image
4a6adf1f2a96adf5ea0c02b61f9fa574306f77fc522f39c2ce6bb164daead882
has dependent children
# #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
# #| FAIL: The output should be empty
# #| expected: '[no output]'
# #| actual: 'Error: image
3ac100502d2123aff1cf6314760c7a89c55108b8de6ea3c10ddc79a1479f0fca
has dependent children'
# #| > 'Error: image
4a6adf1f2a96adf5ea0c02b61f9fa574306f77fc522f39c2ce6bb164daead882
has dependent children'
# #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Fallout from ff4e4905dahttps://github.com/containers/toolbox/pull/642
Not all tests are the same and the ones we're currently running are
system tests. Also the mention of 'podman-stable' is not that important
because we're using the version in the 'stable' stream of Fedora
releases.
https://github.com/containers/toolbox/pull/508
In the Go implementation, when the 'rm' and 'rmi' commands fail to
remove a container or image, they don't use a non-zero exit code.
There's currently no nice fix for this. So, the tests have been
adjusted as a temporary measure.
https://github.com/containers/toolbox/pull/507
Toolbox requires Go 1.13, while Fedora 30 only has Go 1.12.17.
Therefore the test environment needs to be upgraded to something more
recent.
Otherwise, the test fails with:
note: module requires Go 1.13
The name of the go-md2man package changed in Fedora 31, and hence had
to be updated.
https://github.com/containers/toolbox/pull/442
Current Rawhide is actually version 33. So the appropriate image should
be pre-pulled.
Because of the old version of image being pulled, the tests were
failing.
This change adds a pre-run task to pull the fedora-toolbox images from
the registry to reduce the number of false positives caused by
'podman pull' failing to download them during the actual test.
Each section needs a separate playbook because they use different
versions of Fedora, and hence different default images.
https://github.com/containers/toolbox/pull/375
This adds several .yaml files that specify jobs (those in folder
playbooks) and one that serves as the main config (.zuul.yaml).
Tests and builds are currently executed on every change in PRs (ie.,
check and gating) and periodically (according to the documentation
this pipeline should be run at least once a day).
There are 4 tests in total:
1. 'ninja test' - does the same thing that Travis did
2. Fedora 30 - runs the system tests with current Podman and Toolbox
in Fedora 30
3. Fedora 31 - the same but for Fedora 31
4. Fedora Rawhide - the same but for Fedora Rawhide
https://github.com/containers/toolbox/issues/68
