History

Debarshi Ray 5824f0adcb cmd/create, pkg/utils: Simplify hint after creating a container A subsequent commit will add support for configuration files, which can override the default toolbox image. Since this override affects all commands, it effectively ends up adding a fourth option to the 'enter' command, other than the existing options to change the distribution, release and container. This makes it a lot more difficult to reason when only 'toolbox enter --release N' is enough to enter the created container. https://github.com/containers/toolbox/pull/828 https://github.com/containers/toolbox/pull/851		2021-07-23 03:49:08 +02:00
..
libs	test/system: Track bats libs as submodules & install them better	2021-07-22 10:23:53 +02:00
000-setup.bats	cmd/list, test/system: Ignore the problem of UBI not being listed	2021-06-19 01:12:08 +02:00
001-version.bats	test/system: Use env var for invoking Toolbox	2021-05-26 22:52:40 +02:00
002-help.bats	cmd/root: Make 'toolbox' create or fall back to a container if possible	2021-06-29 15:27:56 +02:00
101-create.bats	cmd/create, pkg/utils: Simplify hint after creating a container	2021-07-23 03:49:08 +02:00
102-list.bats	test/system: Adjust asserts for broken test	2021-07-21 18:21:14 +02:00
103-container.bats	tests: Added a verbose test for container starting	2021-07-21 19:56:07 +02:00
104-run.bats	test/system: Fix flaky test	2021-07-22 11:19:02 +02:00
105-rm.bats	tests: Added a verbose test for container starting	2021-07-21 19:56:07 +02:00
106-rmi.bats	tests: Added a verbose test for container starting	2021-07-21 19:56:07 +02:00
999-teardown.bats	test/system: Decouple image caching from Zuul	2021-06-01 00:41:20 +02:00
README.md	test/system: Track bats libs as submodules & install them better	2021-07-22 10:23:53 +02:00

README.md

System tests

These tests are built with BATS (Bash Automated Testing System).

The tests are meant to ensure that Toolbox's functionality remains stable throughout updates of both Toolbox and Podman/libpod.

Warning: The tests are not executed in an isolated environment. Running the tests will clear all podman state (delete all containers, images, etc).

Dependencies

awk
bats
GNU coreutils
podman
skopeo
toolbox

These tests use a few standard libraries for bats which help with clarity and consistency. The libraries are bats-support and bats-assert. These libraries are provided as git submodules in the libs directory. Make sure both are present.

Convention

All tests should follow the nomenclature: [command]: <test description>...
When the test is expected to fail, start the test description with "Try to..."
When the test is to give a non obvious output, it should be put in parenthesis at the end of the title

Examples:

@test "create: Create the default container"
@test "rm: Try to remove a non-existent container"

All the tests start with a clean system (no images or containers) to make sure that there are no dependencies between tests and they are really isolated. Use the setup() and teardown() functions for that purpose.

How to run the tests

First, make sure you have all the dependencies installed.

Enter the toolbox root folder
Invoke command bats ./test/system/ and the test suite should fire up

Mocking of images is done automatically to prevent potential networking issues and to speed up the cases.

By default the test suite uses the system versions of podman, skopeo and toolbox.

If you have a podman, skopeo or toolbox installed in a nonstandard location then you can use the PODMAN, SKOPEO and TOOLBOX environmental variables to set the path to the binaries. So the command to invoke the test suite could look something like this: PODMAN=/usr/libexec/podman TOOLBOX=./toolbox bats ./test/system/.

When running the tests, make sure the test suite: [job] jobs are successful. These jobs set up the whole environment and are a strict requirement for other jobs to run correctly.