Commit graph

165 commits

Author SHA1 Message Date
Debarshi Ray
c37c5238dd test/system: Re-align
All the other Bats files for the system tests are indented by two
spaces, not four.

This will make the subsequent commit easier to read.

https://github.com/containers/toolbox/pull/1317
2023-06-21 09:07:25 +02:00
Nieves Montero
0a1417799a test/system: Fix warnings by specifying the minimum needed Bats version
Bats 1.7.0 emits a warning if a feature that is only available starting
from a certain version of Bats onwards is used without specifying that
version [1]:
  BW02: Using flags on `run` requires at least BATS_VERSION=1.5.0. Use
    `bats_require_minimum_version 1.5.0` to fix this message.
        (from function `bats_warn_minimum_guaranteed_version' in file
           /usr/lib/bats-core/warnings.bash, line 32,
         from function `run' in file
           /usr/lib/bats-core/test_functions.bash, line 227,
         in test file test/system/001-version.bats, line 27)

Note that bats_require_minimum_version itself is only available from
Bats 1.7.0 [2].  Hence, even though the specific feature here (using
flags on 'run') only requires Bats >= 1.5.0, in practice Bats >= 1.7.0
is needed.  Fortunately, commit e22a82fec8 already added a
dependency on Bats >= 1.7.0.  So, there's nothing to worry about.

[1] Bats commit 82002bb6c1a5c418
    https://github.com/bats-core/bats-core/issues/556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

[2] Bats commit 71d6b71cebc3d32b
    https://github.com/bats-core/bats-core/issues/556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

https://github.com/containers/toolbox/pull/1315
2023-06-13 12:48:11 +02:00
Debarshi Ray
bc067f12d6 test/system: Simplify the line count checks by relying on Bats >= 1.7.0
Commit e22a82fec8 already added a dependency on Bats >= 1.7.0,
which is present on Fedora >= 36.  Therefore, it should be exploited
wherever possible to simplify things.

Earlier, when the line counts were checked only with Bats >= 1.7.0,
there was a need to separately check the whole standard error and
output streams with 'assert_output' for the tests to be useful on
Fedora 35, which only had Bats 1.5.0.  Now that the line counts are
being checked unconditionally, there's no need for that anymore.

Note that bats_require_minimum_version itself is only available from
Bats 1.7.0 [1].

[1] Bats commit 71d6b71cebc3d32b
    https://github.com/bats-core/bats-core/issues/556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

https://github.com/containers/toolbox/pull/1314
2023-06-13 12:43:14 +02:00
Debarshi Ray
2ee82affeb pkg/utils: Offer built-in support for Arch Linux
This allows using the 'distro' option to create and enter Arch Linux
containers.  Due to Arch's rolling-release model, the 'release' option
isn't required.  If 'release' is used, the accepted values are 'latest'
and 'rolling'.

https://github.com/containers/toolbox/pull/1311
2023-06-12 22:26:46 +02:00
Debarshi Ray
b6ca18ead1 pkg/utils, test/system: Use the Ubuntu images from quay.io/toolbx/...
https://github.com/containers/toolbox/pull/1306
2023-06-08 15:35:44 +02:00
Ievgen Popovych
a84a358b3b pkg/utils, test/system: Offer built-in support for Ubuntu
This allows using the --distro and --release options to create and enter
Ubuntu containers.

Note that Ubuntu 4.10 was the first ever Ubuntu release [1].  Hence,
values older than that are not permitted for the --release option.

Some changes by Debarshi Ray.

[1] https://wiki.ubuntu.com/Releases

https://github.com/containers/toolbox/pull/483
https://github.com/containers/toolbox/pull/1284

Signed-off-by: Ievgen Popovych <jmennius@gmail.com>
2023-03-28 00:14:19 +02:00
Debarshi Ray
ddd1221d44 test/system: Test 'run' with a RHEL Toolbx container
https://github.com/containers/toolbox/pull/1283
2023-03-27 13:44:38 +02:00
Debarshi Ray
7e9d3918f5 test/system: Test 'run' with a Fedora Toolbx container
It's good to ensure that older well-known Toolbx images continue to work
with newer versions of 'run'.

https://github.com/containers/toolbox/pull/1283
2023-03-27 13:41:51 +02:00
Debarshi Ray
c56b74921a test/system: Test 'create' with a RHEL Toolbx container
https://github.com/containers/toolbox/pull/1282
2023-03-26 10:49:11 +02:00
Debarshi Ray
23e636bb2a test/system: Remove redundant quotes
https://github.com/containers/toolbox/pull/1282
2023-03-26 10:37:10 +02:00
Debarshi Ray
9f4d906e39 test/system: Remove redundant comment
It's quite obvious what the corresponding code is doing, and it isn't
any harder to understand than the rest of the code that's not commented.

https://github.com/containers/toolbox/pull/1282
2023-03-26 10:36:59 +02:00
Debarshi Ray
06245dbf52 test/system: Synchronize with the parseRelease() unit tests
https://github.com/containers/toolbox/pull/1280
2023-03-24 19:57:33 +01:00
Debarshi Ray
26b031f686 test/system: Split the tests for parsing releases
This will make it easier to find out exactly which test failed.

https://github.com/containers/toolbox/pull/1280
2023-03-23 18:09:45 +01:00
Debarshi Ray
44cbdb1f18 test/system: Ensure that both non-Toolbx containers & images are skipped
https://github.com/containers/toolbox/pull/1273
2023-03-22 12:05:06 +01:00
Debarshi Ray
11f5cde45e test/system: Check the number of images after pulling them
This is a quick sanity check with 'podman images' to ensure that all the
images are in place before running 'list'.  Other tests already do this,
so this change makes these two tests consistent with the rest.

https://github.com/containers/toolbox/pull/1273
2023-03-22 10:40:57 +01:00
Debarshi Ray
63aeabe86b test/system: Factor out some repeated code
https://github.com/containers/toolbox/pull/1273
2023-03-22 10:08:47 +01:00
Debarshi Ray
55bf3d06a8 test/system: Use fully qualified image names to check the 'list' output
https://github.com/containers/toolbox/pull/1273
2023-03-22 09:45:23 +01:00
Debarshi Ray
164c9eff36 test/system: Test 'list' with a RHEL Toolbx image
This restores parts of commit e09de9f3e5.

https://github.com/containers/toolbox/pull/1273
2023-03-22 09:18:04 +01:00
Debarshi Ray
aa9da193c6 test/system: Test 'list' with an old Fedora Toolbx image
It's good to ensure that older well-known Toolbx images continue to work
with newer versions of 'list'.

https://github.com/containers/toolbox/pull/1278
2023-03-22 09:16:58 +01:00
Debarshi Ray
571cfc1296 test/system: Use the same convenience function as elsewhere
Fallout from b27795a03e

https://github.com/containers/toolbox/pull/1278
2023-03-22 09:16:52 +01:00
Debarshi Ray
96d629c21f test/system: Test 'list' with the default Toolbx image
This is the 'simple' case of having a well-known Toolbx image (ie.,
not a copy, not an image without a name, not a non-Toolbx image).  It's
good to ensure that the default image works as expected with 'list'
before moving on to more complex scenarios.

https://github.com/containers/toolbox/pull/1278
2023-03-22 09:16:44 +01:00
Debarshi Ray
20fa122820 test/system: Test 'list' both with and without --images
https://github.com/containers/toolbox/pull/1278
2023-03-22 09:16:37 +01:00
Debarshi Ray
6890f5dc8d test/system: Rename a 'list' test for clarity
A subsequent commit will add a variant of this test that uses 'list'
without the '--images' option.

https://github.com/containers/toolbox/pull/1278
2023-03-22 09:16:29 +01:00
Debarshi Ray
9cae66ddd8 test/system: Check the standard error & output streams separately
https://github.com/containers/toolbox/pull/1278
2023-03-22 09:16:20 +01:00
Debarshi Ray
e22a82fec8 test/system: Simplify the line count checks by relying on Bats >= 1.7.0
Fedora 35 was stuck with Bats 1.5.0.  However, it reached End of Life on
13th December 2022 and was dropped from the CI [1].  Fedora 36 is the
oldest supported Fedora and it has Bats 1.8.2.

Therefore, there's no need to retain compatibility with Bats < 1.7.0.

Note that bats_require_minimum_version itself is only available from
Bats 1.7.0 [2].

[1] Commit 419e4e8cd9
    https://github.com/containers/toolbox/pull/1237

[2] Bats commit 71d6b71cebc3d32b
    https://github.com/bats-core/bats-core/issues/556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

https://github.com/containers/toolbox/pull/1273
2023-03-21 19:20:21 +01:00
Debarshi Ray
08e40e666e test/system: Shorten the names of the tests and use consistent wording
Currently, some of the names of the tests were too long, and had
inconsistent and verbose wording.  This made it difficult to look at
them and get a gist of all the scenarios being tested.  The names are
like headings.  They shouldn't be too long, should capture the primary
objective of the test and be consistent in their wording.

https://github.com/containers/toolbox/pull/1276
2023-03-21 19:11:20 +01:00
Debarshi Ray
5b8a7a00c0 test/system: Use long options, instead of their shorter aliases
The long options are easier to grep(1) for in the sources than their
shorter aliases.

https://github.com/containers/toolbox/pull/1276
2023-03-21 19:11:13 +01:00
Debarshi Ray
f364812781 test/system: Shorten the names of the tests and use consistent wording
Currently, some of the names of the tests were too long, and had
inconsistent and verbose wording.  This made it difficult to look at
them and get a gist of all the scenarios being tested.  The names are
like headings.  They shouldn't be too long, should capture the primary
objective of the test and be consistent in their wording.

https://github.com/containers/toolbox/pull/1271
2023-03-20 18:28:10 +01:00
Debarshi Ray
edd82d7e6e test/system: Style fixes
Fallout from e5bab51bca

https://github.com/containers/toolbox/pull/1271
2023-03-20 18:28:07 +01:00
Debarshi Ray
c6760299d2 pkg/utils, test/system: Add some more tests for parsing RHEL releases
https://github.com/containers/toolbox/pull/1262
2023-03-16 22:01:45 +01:00
Debarshi Ray
e01927bfdc pkg/utils, test/system: Add some more tests for parsing Fedora releases
https://github.com/containers/toolbox/pull/1262
2023-03-16 22:01:45 +01:00
Debarshi Ray
a4cc791f39 test/system: Synchronize with the parseRelease() unit tests
https://github.com/containers/toolbox/pull/1266
2023-03-16 18:41:24 +01:00
Debarshi Ray
1bd6cf17ce test/system: Split the tests for parsing releases
This will make it easier to find out exactly which test failed.

https://github.com/containers/toolbox/pull/1266
2023-03-16 18:31:12 +01:00
Debarshi Ray
e5bab51bca test/system: Shorten the names of the tests and use consistent wording
Currently, some of the names of the tests were too long, and had
inconsistent and verbose wording.  This made it difficult to look at
them and get a gist of all the scenarios being tested.  The names are
like headings.  They shouldn't be too long, should capture the primary
objective of the test and be consistent in their wording.

https://github.com/containers/toolbox/pull/1265
2023-03-16 17:57:46 +01:00
Debarshi Ray
068d22a199 test/system: Use long options, instead of their shorter aliases
The long options are easier to grep(1) for in the sources than their
shorter aliases.

https://github.com/containers/toolbox/pull/1265
2023-03-16 17:37:16 +01:00
Debarshi Ray
052666840a Fix spelling mistakes
The noun is 'setup' and the verb is 'set up'.  Similarly 'lookup' and
'look up'.

Original patch from Erik Sjölund for Podman [1,2].

[1] Podman commit aa4279ae151fa9df
    https://github.com/containers/podman/commit/aa4279ae151fa9df
    https://github.com/containers/podman/pull/14658

[2] Podman commit 2827140907255ed0
    https://github.com/containers/podman/commit/2827140907255ed0
    https://github.com/containers/podman/pull/14659

https://github.com/containers/toolbox/pull/1223
2023-01-31 15:53:15 +01:00
Debarshi Ray
936a157117 doc, test/system: Fix the titles of the manuals
Currently, the titles of the manuals are rendered with a pair of empty
parentheses and no section title:
  toolbox(1)()                                              toolbox(1)()

  NAME
         toolbox - Tool for containerized command line environments...

However, they should be:
  toolbox(1)             General Commands Manual              toolbox(1)

  NAME
         toolbox - Tool for containerized command line environments...

This is because the troff generated by go-md2man from Markdown has a
faulty invocation of the .TH macro [1]:
  .nh
  .TH toolbox(1)
  .SH NAME
  .PP
  toolbox - Tool for containerized command line environments on Linux

It should be:
  .nh
  .TH toolbox 1
  .SH NAME
  .PP
  toolbox - Tool for containerized command line environments on Linux

Original patch from Andrew Denton for Podman [2].

[1] https://www.gnu.org/software/groff/manual/groff.html

[2] Podman commit 63c779a857b55b00
    https://github.com/containers/podman/pull/15621

https://github.com/containers/toolbox/pull/1210
2023-01-12 18:45:00 +01:00
Debarshi Ray
cac6b25f19 test/system: Tighten the regular expression used to detect the image ID
An OCI image's identifier (or ID) is a SHA-256 hash [1] whose length is
64 hexadecimal digits.  The ID can be abbreviated to a smaller number of
digits but the minimum is 6.  The regular expression being used to parse
the image ID wasn't checking the number of digits, and would even accept
an empty string as a valid ID, which was clearly wrong.

The format of a image reference is defined in:
https://github.com/moby/moby/blob/master/image/spec/v1.1.md
https://github.com/distribution/distribution/blob/main/reference/reference.go
https://github.com/distribution/distribution/blob/main/reference/regexp.go

Fallout from c6c2e426e0

[1] https://en.wikipedia.org/wiki/SHA-2

https://github.com/containers/toolbox/pull/1209
2023-01-12 14:44:05 +01:00
Debarshi Ray
65194bd45e test/system: Avoid false positives from codespell
Otherwise codespell would complain:
  : @test "create: Try to create a container with invalid custom name...
  >   run $TOOLBOX -y create "ßpeci@l.Nam€"
  :
  ./test/system/101-create.bats:57: Nam ==> Name

CentOS Stream 9 has codespell-2.2.1, while so far the 'unit tests' were
being run on Fedora 36, which only has codespell-2.1.0.

This is a step towards testing on CentOS Stream 9.

https://github.com/containers/toolbox/pull/1200
2022-12-14 23:09:21 +01:00
Debarshi Ray
5f5f7b6aab test/system: Fix spelling mistake using codespell
CentOS Stream 9 has codespell-2.2.1, while so far the 'unit tests' were
being run on Fedora 36, which only has codespell-2.1.0.

This is a step towards testing on CentOS Stream 9.

Fallout from ecd1ced719

https://github.com/containers/toolbox/pull/1200
2022-12-14 23:09:21 +01:00
Martin Krajnak
05a062f8c9 test/system: Ensure that copied images are clearly identified
Note that 'run --keep-empty-lines' counts the trailing newline on the
last line as a separate line.

Until Bats 1.7.0, 'run --keep-empty-lines' had a bug where even when a
command produced no output, it would report a line count of one [1] due
to a stray line feed character.  This needs to be conditionalized, since
Fedora 35 has Bats 1.5.0.

[1] https://github.com/bats-core/bats-core/issues/573

https://github.com/containers/toolbox/issues/1043
2022-12-12 21:49:21 +01:00
Debarshi Ray
6aab0a6175 Unbreak sorting and clearly identify copied images in 'list'
Currently, if an image was copied with:
  $ skopeo copy \
      containers-storage:registry.fedoraproject.org/fedora-toolbox:36 \
      containers-storage:localhost/fedora-toolbox:36

... or:
  $ podman tag \
      registry.fedoraproject.org/fedora-toolbox:36 \
      localhost/fedora-toolbox:36

... then it would show up twice in 'list' with the same name, and in the
wrong order.

Either as:
  $ toolbox list --images
  IMAGE ID      IMAGE NAME                                      CREATED
  2110dbbc33d2  localhost/fedora-toolbox:36                     1 day...
  e085805ade4a  registry.access.redhat.com/ubi8/toolbox:latest  1 day...
  2110dbbc33d2  localhost/fedora-toolbox:36                     1 day...
  70cbe2ce60ca  registry.fedoraproject.org/fedora-toolbox:34    1 day...

... or as:
  $ toolbox list --images
  IMAGE ID      IMAGE NAME                                      CREATED
  2110dbbc33d2  registry.fedoraproject.org/fedora-toolbox:36    1 day...
  e085805ade4a  registry.access.redhat.com/ubi8/toolbox:latest  1 day...
  2110dbbc33d2  registry.fedoraproject.org/fedora-toolbox:36    1 day...
  70cbe2ce60ca  registry.fedoraproject.org/fedora-toolbox:34    1 day...

The correct output should be similar to 'podman images', and be sorted
in ascending order of the names:
  $ toolbox list --images
  IMAGE ID      IMAGE NAME                                      CREATED
  2110dbbc33d2  localhost/fedora-toolbox:36                     1 day...
  e085805ade4a  registry.access.redhat.com/ubi8/toolbox:latest  1 day...
  70cbe2ce60ca  registry.fedoraproject.org/fedora-toolbox:34    1 day...
  2110dbbc33d2  registry.fedoraproject.org/fedora-toolbox:36    1 day...

The problem is that, in these situations, 'podman images --format json'
returns separate identical JSON collections for each copy of the image,
and all of those copies have multiple names:
  [
    {
      "Id": "2110dbbc33d2",
      ...
      "Names": [
        "localhost/fedora-toolbox:36",
        "registry.fedoraproject.org/fedora-toolbox:36"
      ],
      ...
    },
    {
      "Id": "e085805ade4a",
      ...
      "Names": [
        "registry.access.redhat.com/ubi8/toolbox:latest"
      ],
      ...
    },
    {
      "Id": "2110dbbc33d2",
      ...
      "Names": [
        "localhost/fedora-toolbox:36",
        "registry.fedoraproject.org/fedora-toolbox:36"
      ],
      ...
    }
    {
      "Id": "70cbe2ce60ca",
      ...
      "Names": [
        "registry.fedoraproject.org/fedora-toolbox:34"
      ],
      ...
    },
  ]

The image objects need to be flattened to have only one unique name per
copy, but with the same ID, and then sorted to ensure the right order.

Note that the ordering was already broken since commit 2369da5d31,
which started using 'podman images --sort repository'.  Podman can sort
by either the image's repository or tag, but not by the unified name,
which is what Toolbx needs.  Therefore, even without copied images,
Toolbx really does need to sort the images itself.

Prior to commit 2369da5d31, the ordering was correct, but copied
images would only show up once.

Fallout from 2369da5d31

This reverts parts of commit 67e210378e.

https://github.com/containers/toolbox/issues/1043
2022-12-12 21:49:21 +01:00
Debarshi Ray
51eccd3da5 test/system: Test 'rmi' with an image
https://github.com/containers/toolbox/pull/1195
2022-12-12 20:59:52 +01:00
Debarshi Ray
bbdf4ddb63 test/system: Test 'rmi' with an image and its copy
https://github.com/containers/toolbox/issues/1043
2022-12-12 20:59:52 +01:00
Martin Krajnak
d5daa7167e test/system: Add a helper to pull the default image and copy it
This will be used in subsequent commits to test the handling of such
copied images in 'toolbox list' and 'toolbox rmi'.

https://github.com/containers/toolbox/issues/1043
2022-12-12 20:59:47 +01:00
Debarshi Ray
a0d4c957b3 test/system: Test 'rmi' with an image without a name
https://github.com/containers/toolbox/pull/1195
2022-12-12 15:13:26 +01:00
Debarshi Ray
e25ab310fa test/system: Test 'rmi --all' with an image without a name
https://github.com/containers/toolbox/pull/1195
2022-12-12 15:13:26 +01:00
Debarshi Ray
8a37c08780 test/system: Test 'rmi --all' without any images
https://github.com/containers/toolbox/pull/1195
2022-12-12 15:13:26 +01:00
Debarshi Ray
210985ecd1 test/system: Ensure that error messages go to the standard error stream
Currently, there's no way to get assert_line to use the stderr_lines
array [1].  This is worked around by assigning stderr_lines to the
'lines' array.

[1] https://github.com/bats-core/bats-assert/issues/42

https://github.com/containers/toolbox/pull/1195
2022-12-12 15:13:26 +01:00
Debarshi Ray
26ed682cd1 test/system: Keep empty lines to prevent missing and spurious newlines
https://github.com/containers/toolbox/pull/1195
2022-12-12 15:13:26 +01:00