From 0146d223d5b8fde87c8a86d86c58f7d3b16bb0ee Mon Sep 17 00:00:00 2001
From: Debarshi Ray <rishi@fedoraproject.org>
Date: Wed, 20 Sep 2023 22:05:52 +0200
Subject: [PATCH] test/system: Make it easier to debug 'podman logs' failures

Bats' 'run' helper returns with an exit code of 0 even when the command
that it was given to run failed with a non-zero exit code [1].  This is
to enable making further assertions about the command after 'run' has
finished.  If there's nothing that checks for failures, then it will
continue as if everything is alright.

Therefore, currently, if 'podman logs' fails, there's no indication of
it and the test only fails later because it thinks that the container
failed to initialize:
  not ok 106 container: Check container starts without issues
  # (from function `assert_success' in file
       test/system/libs/bats-assert/src/assert.bash, line 114,
  #  in test file test/system/103-container.bats, line 39)
  #   `assert_success' failed
  #
  # -- command failed --
  # status : 1
  # output :
  # --
  #

Instead, from now on, it will be more obvious:
  not ok 106 container: Check container starts without issues
  # (from function `assert_success' in file
       test/system/libs/bats-assert/src/assert.bash, line 114,
  #  in test file test/system/103-container.bats, line 39)
  #   `assert_success' failed
  #
  # -- command failed --
  # status : 125
  # output (2 lines):
  #   Failed to invoke '/usr/bin/podman logs'
  #   Error: no container with name or ID "foo" found: no such container
  # --
  #

One alternative was to use 'assert_success' [2] to assert that the
command given to 'run' succeeded.  That would show the 'podman logs'
failure as:
  not ok 106 container: Check container starts without issues
  # (from function `assert_success' in file
       test/system/libs/bats-assert/src/assert.bash, line 114,
  #  in test file test/system/103-container.bats, line 39)
  #   `assert_success' failed
  #
  # -- command failed --
  # status : 1
  # output (29 lines):
  #
  #   -- command failed --
  #   status : 125
  #   output : Error: no container with name or ID "foo" found: no such
        container
  #   --
  #
  # ...
  #
  #   -- command failed --
  #   status : 125
  #   output : Error: no container with name or ID "foo" found: no such
        container
  #   --
  # --
  #

However, it's a bit too noisy because of the 'assert_success' not
terminating container_started() and continuing to loop for the remaining
attempts.

[1] https://bats-core.readthedocs.io/en/stable/writing-tests.html

[2] https://github.com/bats-core/bats-assert

https://github.com/containers/toolbox/pull/1372
---
 test/system/libs/helpers.bash | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/test/system/libs/helpers.bash b/test/system/libs/helpers.bash
index 6187f78..3e3cbc6 100644
--- a/test/system/libs/helpers.bash
+++ b/test/system/libs/helpers.bash
@@ -444,11 +444,20 @@ function container_started() {
   local num_of_retries=5
 
   for ((j = 0; j < num_of_retries; j++)); do
-    run "$PODMAN" logs "$container_name"
+    run --separate-stderr "$PODMAN" logs "$container_name"
+
+    # shellcheck disable=SC2154
+    if [ "$status" -ne 0 ]; then
+      fail "Failed to invoke '$PODMAN logs'"
+      [ "$output" != "" ] && echo "$output"
+      [ "$stderr" != "" ] && echo "$stderr" >&2
+      ret_val="$status"
+      break
+    fi
 
     # Look for last line of the container startup log
     # shellcheck disable=SC2154
-    if echo "$output" | grep "Listening to file system and ticker events"; then
+    if echo "$output $stderr" | grep "Listening to file system and ticker events"; then
       ret_val=0
       break
     fi