wbrawner/toolbox
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

test/system: Make it easier to spot failures to download & cache images

Browse source 
Currently, if 'skopeo copy ...' fails to download and cache an OCI image
during setup_suite(), the test suite doesn't immediately fail, but
continues.  It only fails later when trying to set up the Docker
registry and contains a lot of noise:
  not ok 1 setup_suite
  # (from function `assert_success' in file
       test/system/libs/bats-assert/src/assert.bash, line 114,
  #  from function `_setup_docker_registry' in file
       test/system/libs/helpers.bash, line 211,
  #  from function `setup_suite' in test file
       test/system/setup_suite.bash, line 59)
  #   `_setup_docker_registry' failed
  # Failed to cache image registry.fedoraproject.org/fedora-toolbox:38
      to /tmp/bats-run-GyTP7A/image-cache/fedora-toolbox-38
  #
  # -- command failed --
  # status : 1
  # output : time="2023-09-25T12:19:52+02:00" level=fatal
      msg="initializing source
        docker://registry.fedoraproject.org/fedora-toolbox:38-foo:
        reading manifest 38-foo in
        registry.fedoraproject.org/fedora-toolbox: manifest unknown"
  # --
  #
  # Failed to cache image quay.io/toolbx/arch-toolbox:latest to
      /tmp/bats-run-GyTP7A/image-cache/arch-toolbox-latest
  #
  # -- command failed --
  # status : 1
  # output : time="2023-09-25T12:20:48+02:00" level=fatal
      msg="initializing source
        docker://quay.io/toolbx/arch-toolbox:latest-foo: reading
        manifest latest-foo in quay.io/toolbx/arch-toolbox: manifest
        unknown"
  # --
  #
  # Failed to cache image registry.fedoraproject.org/fedora-toolbox:34
      to /tmp/bats-run-GyTP7A/image-cache/fedora-toolbox-34
  #
  # -- command failed --
  # status : 1
  # output : time="2023-09-25T12:21:42+02:00" level=fatal
      msg="initializing source
        docker://registry.fedoraproject.org/fedora-toolbox:34-foo:
        reading manifest 34-foo in
        registry.fedoraproject.org/fedora-toolbox: manifest unknown"
  # --
  #
  # ...
  #
  # -- command failed --
  # status : 1
  # output : time="2023-09-25T12:26:33+02:00" level=fatal
      msg="determining manifest MIME type for
        dir:/tmp/bats-run-GyTP7A/image-cache/fedora-toolbox-34: open
        /tmp/bats-run-GyTP7A/image-cache/fedora-toolbox-34/manifest.json:
        no such file or directory"
  # --
  #
  # docker-registry
  # 27fa141e291e64e4c7a148c88ddab219ff2bfb5802a2982dc4188dc11f41692d
  # Untagged: quay.io/toolbox_tests/registry:latest
  # Deleted: fea5a12cde107bb407bc44ede6dd9edea1d2b4171cd8e52b0cb330bf45e517e1

It makes it look as if the root cause of the failure is related to
setting up the Docker registry, which it isn't, and all that noise makes
it difficult to spot the actual problem.

Instead, from now on, it will be more obvious:
  not ok 1 setup_suite
  # (from function `setup_suite' in test file
       test/system/setup_suite.bash, line 44)
  #   `_pull_and_cache_distro_image "$system_id" "$system_version" ||
         false' failed
  # Failed to cache image registry.fedoraproject.org/fedora-toolbox:38
      to /tmp/bats-run-62b8CU/image-cache/fedora-toolbox-38
  # time="2023-09-25T13:55:42+02:00" level=fatal msg="initializing
      source docker://registry.fedoraproject.org/fedora-toolbox:38-foo:
      reading manifest 38-foo in
      registry.fedoraproject.org/fedora-toolbox: manifest unknown"

Note that Bats' 'run' helper [1] isn't designed to work inside
setup_suite().  eg., 'run --separate-stderr' doesn't work because
BATS_TEST_TMPDIR isn't defined.

[1] https://bats-core.readthedocs.io/en/stable/writing-tests.html

https://github.com/containers/toolbox/pull/1377
This commit is contained in:
Debarshi Ray 2023-09-23 00:51:10 +02:00
parent 74bf1af983
commit 61e2c970f8
1 changed files with 9 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
test/system/libs/helpers.bash
View file

@ -108,12 +108,17 @@ function _pull_and_cache_distro_image() {
assert_success assert_success
fi fi
local error_message
local -i j local -i j
local -i ret_val
for ((j = 0; j < num_of_retries; j++)); do for ((j = 0; j < num_of_retries; j++)); do
run "$SKOPEO" copy --dest-compress "docker://${image}" "dir:${IMAGE_CACHE_DIR}/${image_archive}" error_message="$( ("$SKOPEO" copy --dest-compress \
"docker://${image}" \
"dir:${IMAGE_CACHE_DIR}/${image_archive}" >/dev/null) 2>&1)"
ret_val="$?"
if [ "$status" -eq 0 ]; then if [ "$ret_val" -eq 0 ]; then
cached=true cached=true
break break
fi fi
@ -123,7 +128,8 @@ function _pull_and_cache_distro_image() {
if ! $cached; then if ! $cached; then
echo "Failed to cache image ${image} to ${IMAGE_CACHE_DIR}/${image_archive}" echo "Failed to cache image ${image} to ${IMAGE_CACHE_DIR}/${image_archive}"
assert_success [ "$error_message" != "" ] && echo "$error_message" >&2
return "$ret_val"
fi fi
cleanup_all cleanup_all