Currently, some of the names of the tests were too long, and had
inconsistent and verbose wording. This made it difficult to look at
them and get a gist of all the scenarios being tested. The names are
like headings. They shouldn't be too long, should capture the primary
objective of the test and be consistent in their wording.
https://github.com/containers/toolbox/pull/1276
Currently, some of the names of the tests were too long, and had
inconsistent and verbose wording. This made it difficult to look at
them and get a gist of all the scenarios being tested. The names are
like headings. They shouldn't be too long, should capture the primary
objective of the test and be consistent in their wording.
https://github.com/containers/toolbox/pull/1271
Currently, some of the names of the tests were too long, and had
inconsistent and verbose wording. This made it difficult to look at
them and get a gist of all the scenarios being tested. The names are
like headings. They shouldn't be too long, should capture the primary
objective of the test and be consistent in their wording.
https://github.com/containers/toolbox/pull/1265
Currently, the titles of the manuals are rendered with a pair of empty
parentheses and no section title:
toolbox(1)() toolbox(1)()
NAME
toolbox - Tool for containerized command line environments...
However, they should be:
toolbox(1) General Commands Manual toolbox(1)
NAME
toolbox - Tool for containerized command line environments...
This is because the troff generated by go-md2man from Markdown has a
faulty invocation of the .TH macro [1]:
.nh
.TH toolbox(1)
.SH NAME
.PP
toolbox - Tool for containerized command line environments on Linux
It should be:
.nh
.TH toolbox 1
.SH NAME
.PP
toolbox - Tool for containerized command line environments on Linux
Original patch from Andrew Denton for Podman [2].
[1] https://www.gnu.org/software/groff/manual/groff.html
[2] Podman commit 63c779a857b55b00
https://github.com/containers/podman/pull/15621https://github.com/containers/toolbox/pull/1210
Otherwise codespell would complain:
: @test "create: Try to create a container with invalid custom name...
> run $TOOLBOX -y create "ßpeci@l.Nam€"
:
./test/system/101-create.bats:57: Nam ==> Name
CentOS Stream 9 has codespell-2.2.1, while so far the 'unit tests' were
being run on Fedora 36, which only has codespell-2.1.0.
This is a step towards testing on CentOS Stream 9.
https://github.com/containers/toolbox/pull/1200
CentOS Stream 9 has codespell-2.2.1, while so far the 'unit tests' were
being run on Fedora 36, which only has codespell-2.1.0.
This is a step towards testing on CentOS Stream 9.
Fallout from ecd1ced719https://github.com/containers/toolbox/pull/1200
Note that 'run --keep-empty-lines' counts the trailing newline on the
last line as a separate line.
Until Bats 1.7.0, 'run --keep-empty-lines' had a bug where even when a
command produced no output, it would report a line count of one [1] due
to a stray line feed character. This needs to be conditionalized, since
Fedora 35 has Bats 1.5.0.
[1] https://github.com/bats-core/bats-core/issues/573https://github.com/containers/toolbox/issues/1043
Currently, if an image was copied with:
$ skopeo copy \
containers-storage:registry.fedoraproject.org/fedora-toolbox:36 \
containers-storage:localhost/fedora-toolbox:36
... or:
$ podman tag \
registry.fedoraproject.org/fedora-toolbox:36 \
localhost/fedora-toolbox:36
... then it would show up twice in 'list' with the same name, and in the
wrong order.
Either as:
$ toolbox list --images
IMAGE ID IMAGE NAME CREATED
2110dbbc33d2 localhost/fedora-toolbox:36 1 day...
e085805ade4a registry.access.redhat.com/ubi8/toolbox:latest 1 day...
2110dbbc33d2 localhost/fedora-toolbox:36 1 day...
70cbe2ce60ca registry.fedoraproject.org/fedora-toolbox:34 1 day...
... or as:
$ toolbox list --images
IMAGE ID IMAGE NAME CREATED
2110dbbc33d2 registry.fedoraproject.org/fedora-toolbox:36 1 day...
e085805ade4a registry.access.redhat.com/ubi8/toolbox:latest 1 day...
2110dbbc33d2 registry.fedoraproject.org/fedora-toolbox:36 1 day...
70cbe2ce60ca registry.fedoraproject.org/fedora-toolbox:34 1 day...
The correct output should be similar to 'podman images', and be sorted
in ascending order of the names:
$ toolbox list --images
IMAGE ID IMAGE NAME CREATED
2110dbbc33d2 localhost/fedora-toolbox:36 1 day...
e085805ade4a registry.access.redhat.com/ubi8/toolbox:latest 1 day...
70cbe2ce60ca registry.fedoraproject.org/fedora-toolbox:34 1 day...
2110dbbc33d2 registry.fedoraproject.org/fedora-toolbox:36 1 day...
The problem is that, in these situations, 'podman images --format json'
returns separate identical JSON collections for each copy of the image,
and all of those copies have multiple names:
[
{
"Id": "2110dbbc33d2",
...
"Names": [
"localhost/fedora-toolbox:36",
"registry.fedoraproject.org/fedora-toolbox:36"
],
...
},
{
"Id": "e085805ade4a",
...
"Names": [
"registry.access.redhat.com/ubi8/toolbox:latest"
],
...
},
{
"Id": "2110dbbc33d2",
...
"Names": [
"localhost/fedora-toolbox:36",
"registry.fedoraproject.org/fedora-toolbox:36"
],
...
}
{
"Id": "70cbe2ce60ca",
...
"Names": [
"registry.fedoraproject.org/fedora-toolbox:34"
],
...
},
]
The image objects need to be flattened to have only one unique name per
copy, but with the same ID, and then sorted to ensure the right order.
Note that the ordering was already broken since commit 2369da5d31,
which started using 'podman images --sort repository'. Podman can sort
by either the image's repository or tag, but not by the unified name,
which is what Toolbx needs. Therefore, even without copied images,
Toolbx really does need to sort the images itself.
Prior to commit 2369da5d31, the ordering was correct, but copied
images would only show up once.
Fallout from 2369da5d31
This reverts parts of commit 67e210378e.
https://github.com/containers/toolbox/issues/1043
If an image was copied with:
$ skopeo copy \
containers-storage:registry.fedoraproject.org/fedora-toolbox:36 \
containers-storage:localhost/fedora-toolbox:36
... or:
$ podman tag \
registry.fedoraproject.org/fedora-toolbox:36 \
localhost/fedora-toolbox:36
... then the image ID is only showed once in 'podman images --quiet',
not twice.
A subsequent commit will use this to write tests to ensure that copied
images are correctly handled.
https://github.com/containers/toolbox/issues/1043
Note that 'run --keep-empty-lines' counts the trailing newline on the
last line as a separate line.
Until Bats 1.7.0, 'run --keep-empty-lines' had a bug where even when a
command produced no output, it would report a line count of one [1] due
to a stray line feed character. This needs to be conditionalized, since
Fedora 35 has Bats 1.5.0.
[1] https://github.com/bats-core/bats-core/issues/573https://github.com/containers/toolbox/pull/1192
Note that 'run --keep-empty-lines' counts the trailing newline on the
last line as a separate line.
Until Bats 1.7.0, 'run --keep-empty-lines' had a bug where even when a
command produced no output, it would report a line count of one [1] due
to a stray line feed character. This needs to be conditionalized, since
Fedora 35 has Bats 1.5.0.
[1] https://github.com/bats-core/bats-core/issues/573https://github.com/containers/toolbox/pull/1192
A subsequent commit will test the order in which images with and without
names are listed. It's logical for that test to come after the one
about the basic support for images without names.
https://github.com/containers/toolbox/pull/1192
This was making it difficult to read the Bats assertions on test
failures, by polluting it with unexpected and irrelevant output from
'podman images'. For example [1]:
not ok 39 list: Images with and without names in 12332ms
# (from function `assert' in file test/system/libs/bats-assert/src/assert.bash, line 46,
# in test file test/system/102-list.bats, line 126)
# `assert [ ${#stderr_lines[@]} -eq 0 ]' failed
# REPOSITORY TAG IMAGE ID CREATED SIZE
# registry.fedoraproject.org/fedora-toolbox 35 862705390e8b 4 weeks ago 332 MB
# REPOSITORY TAG IMAGE ID CREATED SIZE
# registry.fedoraproject.org/fedora-toolbox 35 862705390e8b 4 weeks ago 332 MB
# registry.fedoraproject.org/fedora-toolbox 34 70cbe2ce60ca 7 months ago 354 MB
#
# -- assertion failed --
# expression : [ 1 -eq 0 ]
# --
#
Fallout from 7973181136
[1] https://github.com/containers/toolbox/pull/1192https://github.com/containers/toolbox/pull/1193
Commit ae43560d45 had added a test with a similar intention. When
the test suite is run on a Fedora Rawhide host, it tests whether the
containers for the two previous stable Fedora releases start or not.
Fedora N-2 reaches End of Life 4 weeks after Fedora N is released [1].
So, testing the containers for Fedora Rawhide and the two previous
stable releases on a Fedora Rawhide host is a decent test of general
backwards compatibility.
However, as seen recently [2], this isn't enough to catch some known
ABI compatibility issues [3,4]. These involve toolbox binaries built
on hosts with newer toolchains that aren't meant to be run against
containers with older runtimes. A targeted test is needed to defend
against these scenarios.
The fedora-toolbox:34 image has glibc-2.33, which is old enough to be
unable to run binaries compiled on Fedora 35 with glibc-2.34 and newer.
[1] https://docs.fedoraproject.org/en-US/releases/
[2] https://github.com/containers/toolbox/pull/1180
[3] Commit 6063eb27b9https://github.com/containers/toolbox/issues/821
[4] Commit 6ad9c63180https://github.com/containers/toolbox/issues/529https://github.com/containers/toolbox/pull/1187
Fedora 32 reached End of Life on 25th May 2021:
https://docs.fedoraproject.org/en-US/releases/eol/
That's quite old because right now Fedora 35 is nearing its End of Life.
Since the tests are intended for Toolbx, not the Fedora infrastructure,
it will be better to use a newer image, because images that are too old
can get lost from registry.fedoraproject.org. The fedora-toolbox:34
image can be a drop-in replacement for the fedora-toolbox:32 image for
the purposes of this test suite, and has the advantage of being newer.
Note that fedora-toolbox:34 is also old enough to test that the toolbox
binary runs against it's build-time ABI from the host, and not the
Toolbx container's ABI, when it's invoked as the entry point of the
container [1,2]. This is important because the subsequent commit will
add a test to ensure that.
[1] Commit 6063eb27b9https://github.com/containers/toolbox/issues/821
[2] Commit 6ad9c63180https://github.com/containers/toolbox/issues/529https://github.com/containers/toolbox/pull/1187
This is a precursor to checking that higher valued exit codes from the
command running inside the container are retained, and commands like
test(1) can be used with 'toolbox run ...' in subsequent test cases.
https://github.com/containers/toolbox/pull/1163
Currently, some of the names of the tests were too long, and had
inconsistent and verbose wording. This made it difficult to look at
them and get a gist of all the scenarios being tested. The names are
like headings. They shouldn't be too long, should capture the primary
objective of the test and be consistent in their wording.
https://github.com/containers/toolbox/pull/1161
