Commit graph

77 commits

Author SHA1 Message Date
Debarshi Ray
8454b31a82 cmd/utils, pkg/utils: Improve the error messages for the distro option
https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1103
2022-09-02 14:52:44 +02:00
Debarshi Ray
8ca5611942 Increase the validation coverage for the container & release options
Currently, the container name and release are only validated if they
were specified as command line options.  Neither the value of release
in the configuration file nor the container name generated from an
image are validated.

There's also a lot of repeated code in the command front-ends to
validate the container name and release.  This opens the door for
mistakes.  Any adjustment to the code must be repeated elsewhere, and
there are subtle interactions and overlaps between the validation code
and the code to resolve container and image names.

It's worth noting that the container and image name resolution happens
for both the command line and configuration file options, and generates
the container name from the image when necessary.

Therefore, validating everything while resolving cleans up the command
front-ends and increases the coverage of the validation.

This introduces the use of sentinel error values and custom error
implementations to identify the different errors that can occur while
resolving the container and images, so that they can be appropriately
shown to the user.

https://github.com/containers/toolbox/pull/1101
2022-09-02 13:11:32 +02:00
Debarshi Ray
b5474bff84 cmd, pkg/utils: Clarify the error message if the release is invalid
Currently, if --release has an invalid argument, the error message
doesn't give any hints as to what's an acceptable value.  This can be
confusing.  eg., is 36 a valid argument for Fedora?  Or is it f36?  Or
is it F36?  Is 'rawhide' accepted?

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1100
2022-09-01 17:57:39 +02:00
Debarshi Ray
df7e01df10 pkg/utils: Ensure that the distro CLI and config file options are valid
Currently, if an invalid or unsupported string is specified as the
distro on the command line or in the configuration file, then it would
silently fallback to Fedora.  This shouldn't happen.

It should only fallback to Fedora when no distro was specified and
there's no supported Toolbox image matching the host operating system.
If a distro was explicitly specified then it should either be supported
or it should error out.

The test cases were resurrected from commit 8b6418d8aa.

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1080
2022-09-01 17:43:20 +02:00
Debarshi Ray
0e66af91fe Revert "cmd, pkg/utils: Split distro and release parsing and ..."
The idea of splitting ResolveContainerAndImageNames into two public
functions [1] didn't turn out to be so useful [2].  Splitting things
even further might make it worse.  A better approach might be to
(re-)unify the code further.

This is the first step towards that.

This reverts the following commits:
  * 5c8ad7a7ec
  * 02f45fd3f2
  * 8b6418d8aa

... but retains the test cases that were not tied to the changes in
those commits.

[1] Commit fd756089ef
    https://github.com/containers/toolbox/pull/828
    https://github.com/containers/toolbox/pull/838

[2] https://github.com/containers/toolbox/pull/977

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1080
2022-09-01 17:43:04 +02:00
Debarshi Ray
4f78c5ef86 cmd/utils, test/system: Tweak an error message for consistency
Barring the first line, all other lines are terminated with a full stop
elsewhere.

https://github.com/containers/toolbox/pull/1099
2022-09-01 13:02:40 +02:00
Debarshi Ray
a9a5b96ec6 test/system: Fix typo
This isn't causing any problems at the moment.  However, the test can
break if the order in which the command line arguments are validated
changes.  eg., if the presence of a command is checked before the
release, then the error message will be different.

Fallout from 8b6418d8aa

https://github.com/containers/toolbox/pull/1091
2022-08-26 17:00:56 +02:00
Debarshi Ray
978bb524e4 test/system: Silence warning with Bats >= 1.7.0
Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1]:
  BW01: `run`'s command `/opt/bin/toolbox run non-existent-command`
    exited with code 127, indicating 'Command not found'. Use run's
    return code checks, e.g. `run -127`, to fix this message.
        (from function `run' in file
          /usr/lib/bats-core/test_functions.bash, line 299,
         in test file test/system/104-run.bats, line 148)

This requires Bats >= 1.5.0, which is present in Fedora >=35, and
supports specifying the exit code as an argument to Bats' 'run'
command [2].

However, bats_require_minimum_version can't be used, because it's
only available from Bats 1.7.0, which is new enough that it's absent
from Fedora 35.

[1] Bats commit c6dc2f88361a4f5b
    https://github.com/bats-core/bats-core/issues/547
    https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

[2] https://github.com/bats-core/bats-core/pull/367
    https://github.com/bats-core/bats-core/pull/507
    https://bats-core.readthedocs.io/en/stable/writing-tests.html

[3] Bats commit 71d6b71cebc3d32b
    https://github.com/bats-core/bats-core/issues/556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

https://github.com/containers/toolbox/pull/1081
2022-08-01 10:47:13 +02:00
Oliver Gutierrez
f8e21a31b3 cmd/run, root: Exit with exit code of invoked command
When a command is executed with toolbox run and it returns a non-zero
exit code, it is just ignored if that exit code is not handled. This
prevents users to identify errors when executing commands in toolbox.

With this fix, the exit codes of the invoked command are propagated
and returned by 'toolbox run'. This includes even exit codes returned
by Podman on error.

https://github.com/containers/toolbox/pull/1013

Co-authored-by: Ondřej Míchal <harrymichal@seznam.cz>
2022-03-21 00:05:45 +02:00
Ondřej Míchal
ecd1ced719 cmd/create: Add option --authfile
The option accepts a path to a file that is passed to an internal call
to 'podman pull' via the '--authfile' option. This will make it easier
to pull images from registries with authentication in-place.

Fixes https://github.com/containers/toolbox/issues/689

https://github.com/containers/toolbox/pull/935
2022-03-20 18:08:42 +02:00
Ondřej Míchal
192b9c8265 test/system: Mention where to find available environmental variables
https://github.com/containers/toolbox/pull/1024
2022-03-20 17:11:33 +02:00
Ondřej Míchal
0cdc6b8305 test/system: Add a section about test writing guidelines
https://github.com/containers/toolbox/pull/1024
2022-03-20 17:11:33 +02:00
Ondřej Míchal
dfc38b156c test/system: Adjust the return value of container start test
The output of 'podman logs' does not need to be returned as it can already
be seen in the logs when bats is properly configured.

https://github.com/containers/toolbox/pull/1024
2022-03-20 17:11:33 +02:00
Ondřej Míchal
fc71b9de34 test/system: Skip caching of an image if it is already cached
https://github.com/containers/toolbox/pull/1024
2022-03-20 17:11:33 +02:00
Ondřej Míchal
754c6fb4d1 test/system: Set custom runtime root for rootless Podman
To more completely separate the test environment Podman runtime from the
default system one set also a custom runtime directory.

https://github.com/containers/toolbox/pull/1024
2022-03-20 17:11:33 +02:00
Ondřej Míchal
813f971181 test/system: Don't cleanup tests by resetting Podman
Calling 'podman system cleanup' causes problems with containers/images
in a separate Podman root. Despite being stored elsewhere, they are
still under Podman's influence and the cleanup removes them. Also,
running containers (outside the scope of the tests) still got affected
by this call and e.g., lost the ability to follow terminal size changes.

Despite the raised concerns, to ensure proper cleanup of any Podman
state, the reset still needs to be done. Thus, do it only once during
the test suite teardown, moving the potential source of problems to a
single position..

https://github.com/containers/toolbox/pull/1024
2022-03-20 17:11:33 +02:00
Ondřej Míchal
5012cda506 test/system: Minor fixes & renaming
https://github.com/containers/toolbox/pull/1024
2022-03-20 17:11:33 +02:00
Ondřej Míchal
8b6418d8aa cmd, pkg/utils: Split distro and release parsing and report better errors
Using a non-supported distribution via `--distro` resulted in a silent
fallback to the Fedora image which confuses users. When a faulty release
format was used with `--release` a message without any hint about the
correct format was shown to the user.

This separates distro and release parsing into two chunks that have
greater control over error reporting and provides more accurate error
reports for the user.

Fixes https://github.com/containers/toolbox/issues/937

https://github.com/containers/toolbox/pull/977
2022-02-21 13:43:24 +02:00
pablomh
2af0b30ed3 test/system: Delete .swp file
Delete .swp file that was inadvertenly introduced in [1].

[1] 7a5f3ba2e2

https://github.com/containers/toolbox/pull/996
2022-02-07 01:10:10 +02:00
Ondřej Míchal
f456c173b6 pkg/utils: Use new UBI toolbox image
Red Hat has published a new UBI image made specificaly for Toolbx.
Make use of it from now on.

Fixes: https://github.com/containers/toolbox/issues/961

https://github.com/containers/toolbox/issues/976
2022-01-09 16:46:37 +01:00
Debarshi Ray
411147988b cmd, test/system: Make the behaviour of 'toolbox' conditional
Commit 6c86cabbe5 changed the command line interface to behave
a lot similar to that of github.com/coreos/toolbox, which makes things
easier for those switching over from it. Make it conditional so that
only those OS distributors who truly need it may enable it, and
restore the previous behaviour as the default.

The tests were updated to test the default behaviour that the vast
majority of users would be seeing. Ideally, the test suite would be run
twice with the migration path turned off and on. However, that would
require a more intrusive surgery of the test suite and likely make it
slower. It might not be worth the hassle because of the small number
of users who should be using the migration path.

Note that the copyright and license notices really must use C++-style
// line comments, because build constraints can only be preceded by
blank lines and other line comments. C-style /* */ block comments can't
precede the build constraints.

This reverts commit ca899c8a56 and parts
of commit 3aeb7cf288.

[1] go help buildconstraint
    https://pkg.go.dev/cmd/go#hdr-Build_constraints

https://github.com/containers/toolbox/pull/951
2021-12-10 01:33:24 +01:00
Ondřej Míchal
ae43560d45 test/system: Test startup on Rawhide with supported versions
We need to know if the latest changes in the libc (that is dynamically
linked to the binary) causes problems in containers based on older
releases of Fedora.

The estimate of the version numbers is very crude and does not follow
the upstream schedule. That should not be a problem, though.

A part of an existing test has been reused and made into a helper
function to implement this.

This increases the run time of the test suite on Rawhide which already
takes longer than the same test suite on released versions of Fedora.
Make up for it by increasing the timeout by 2 minutes.

https://github.com/containers/toolbox/pull/899
2021-12-04 17:37:40 +02:00
Ondřej Míchal
ae8bd1a9c9 test/system: Properly bail-out if failed to cache images
The 'die' function is a remnant from times before the system tests
rewrite. It served for writing an error message and then failing
the test. Since the rewrite it is no longer present. Instead, simply
use 'false' in case a caching step fails.

Fallout from da6b6a7c5a6e5e284e6642c29159a872ceec94e2

https://github.com/containers/toolbox/pull/899
2021-12-04 17:37:40 +02:00
Ondřej Míchal
fb565af0a0 test/system: Add function for setting up environment
Instead of typing out two function names to set up the test environment,
type out only one. We never know if a new set up function will show up.

https://github.com/containers/toolbox/pull/818
2021-12-01 01:09:34 +02:00
Oliver Gutierrez
7a5f3ba2e2 test/system: Make tests non-destructive
This allows to run the test suite without having to worry about blasting
the whole local state of Podman.

This is done by creating a configuration file with a custom path for the
storage of Podman and specifying the config file using an env var.

The used location for the temporary storage is located either under
XDG_CACHE_HOME and if the one is not defined, $HOME/.cache is used
instead. The data are namespaced. This follows the XDG Base Directory
Specification[0]. Other locations could be /tmp or /run but those
locations usually use tmpfs and that filesystem can not be used by
Podman[1] due to missing features in tmpfs.

https://github.com/containers/toolbox/pull/818

[0] https://specifications.freedesktop.org/basedir-spec/latest/index.html
[1] https://github.com/containers/podman/issues/10693#issuecomment-863007516
2021-12-01 01:09:34 +02:00
Debarshi Ray
01863d8fe0 test/system: Use 'command -v' to detect the presence of man(1)
'command -v' is more obvious when reading and is POSIX compatible [1].
While 'hash' also gets the job done, it's more of a caching mechanism.

[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/command.html

https://github.com/containers/toolbox/pull/922
2021-11-13 01:21:52 +01:00
Ondřej Míchal
07afff0c30 test/system: Adjust help test to work only with man present
Unlike the following test this one tests using the content of the
toolbox(1) manual page in man. man has to be present in PATH for this
test to be relevant.

Also, this changes the text used to test the output. The current text
can be found in the added short help message and that causes the test
to pass even though it should not. Instead, look for the text in the
"header" of the manual page.

https://github.com/containers/toolbox/pull/837
2021-11-08 01:44:15 +02:00
Oliver Gutierrez
40fc1689a3 utils, help: Show basic help when "man" command is not available.
Fedora CoreOS systems do not have the man command installed. Running
toolbox --help on such a system results in a "man(1) not found" error.

As a compromise for systems without man, we added a simple help text
showing the most commonly used toolbox commands and an URL that direct
users to the Toolbox website where they can find the manuals in Markdown
format.

Fixes #713

https://github.com/containers/toolbox/pull/837
2021-11-08 01:44:15 +02:00
Ondřej Míchal
3aeb7cf288 test/system: Add test placeholders for entering containers
These tests need to be implemented in the future but they require some
magic with socat or similar tools as entering a container is creating
a new subshell and that is hard to monitor from a bash script. Better
not to forget then.

https://github.com/containers/toolbox/pull/915
2021-11-07 17:06:53 +02:00
Ondřej Míchal
ca899c8a56 test/system: Add tests for failures for 'root'
https://github.com/containers/toolbox/pull/915
2021-11-07 17:06:53 +02:00
Ondřej Míchal
a7e7e0ac2b test/system: Add tests for failures for 'enter'
https://github.com/containers/toolbox/pull/915
2021-11-07 17:06:53 +02:00
Ondřej Míchal
fc336bd8b0 test/system: Add tests for failures for 'run'
https://github.com/containers/toolbox/pull/915
2021-11-07 17:06:53 +02:00
Alex Jia
26ae0bb896 test/system: fix typo in 102-list.bats
Signed-off-by: Alex Jia <chuanchang.jia@gmail.com>
2021-09-10 10:42:16 +01:00
Oliver Gutierrez
075b9a8d27 tests: Fix tests to setup the XDG_RUNTIME_DIR variable when empty
https://github.com/containers/toolbox/pull/857

When the XDG_RUNTIME_DIR variable is empty toolbox is not able to
initialize the container correctly and fails to run.
2021-08-09 19:57:41 +02:00
Oliver Gutierrez
09fb237727 tests: Changed container image source for busybox
Due to docker rate limiting we can not rely in docker.io for
retrieving the images.

This was detected when executing our tests for podman fedora
gating pipeline. Our busybox image was not downloaded and
one of the list tests was failing.
2021-08-09 17:09:29 +02:00
Ondřej Míchal
04c673dd06 test/system: Test pull failure
Follow-up to https://github.com/containers/toolbox/pull/852

https://github.com/containers/toolbox/pull/854
2021-07-29 10:10:02 +02:00
Ondřej Míchal
9820550c82 test/system: Use BATS_RUN_TMPDIR for image cache
Using the current working directory for cache is not a good solution
since the test files may reside in a location that is unwritable (e.g.,
/usr/share). The `BATS_RUN_TMPDIR` variable should point to a location
that is sure to be writeable from the test suite.

https://github.com/containers/toolbox/pull/850
2021-07-28 09:31:35 +02:00
Debarshi Ray
5824f0adcb cmd/create, pkg/utils: Simplify hint after creating a container
A subsequent commit will add support for configuration files, which can
override the default toolbox image. Since this override affects all
commands, it effectively ends up adding a fourth option to the 'enter'
command, other than the existing options to change the distribution,
release and container. This makes it a lot more difficult to reason
when only 'toolbox enter --release N' is enough to enter the created
container.

https://github.com/containers/toolbox/pull/828
https://github.com/containers/toolbox/pull/851
2021-07-23 03:49:08 +02:00
Ondřej Míchal
b0beb68255 test/system: Fix flaky test
This is a naive attempt to solve the flaking. It could be caused by what
is going inside of the run function.
2021-07-22 11:19:02 +02:00
Ondřej Míchal
2594199fef test/system: Track bats libs as submodules & install them better
This will make it easier to work with system tests.

https://github.com/containers/toolbox/pull/842
2021-07-22 10:23:53 +02:00
Ondřej Míchal
6c2cd0e929 test/system: Drop hack around unknown dirs in containers
This hack should not be needed since v0.0.99.2[0].

[0] https://github.com/containers/toolbox/releases/tag/0.0.99.2

https://github.com/containers/toolbox/pull/843
2021-07-21 23:43:41 +02:00
Ondřej Míchal
16b0c5d88f test/system: Check whole lines instead of partials
The 'toolbox run' command has one downside: all newlines contain
a carriage return (CR). This is caused by the unconditional use of the
--tty option in `podman exec`[0]. In these particular tests this can be
worked around by not printing a newline at all.

Another quirk around partial is to check the last line of the output.

[0] https://github.com/containers/podman/issues/9718

https://github.com/containers/toolbox/pull/843
2021-07-21 23:43:41 +02:00
Oliver Gutierrez
1f0c6d5100 tests: Added a verbose test for container starting
https://github.com/containers/toolbox/pull/831
2021-07-21 19:56:07 +02:00
Ondřej Míchal
259afdf815 test/system: Adjust asserts for broken test
The output of `podman build` has changed a bit. Each line of log
describing the build is now in the format of:

- STEP i/n: msg

instead of:

- STEP i: msg

where i is the current step and n the maximum number of steps.

The exact format is not important for the purpose of testing Toolbox, so
we may fallback to partial string testing.

Also the latest step ("COMMIT") seems to no longer be considered a step,
so just check for the word.

https://github.com/containers/toolbox/pull/846
2021-07-21 18:21:14 +02:00
Ondřej Míchal
210eded9a3 test/system: Update README
https://github.com/containers/toolbox/pull/824
2021-07-07 12:27:41 +02:00
Ondřej Míchal
6c86cabbe5 cmd/root: Make 'toolbox' create or fall back to a container if possible
This makes 'toolbox', without any commands specified, behave a lot like
'toolbox enter'. When there aren't any toolbox containers, it will
offer to create a new container matching the same parameters passed to
the command. If there's just one toolbox container available, then it
will fall back to it.

This makes the command line interface a lot similar to that of
github.com/coreos/toolbox, which makes things easier for those
switching over from it.

Some changes by Debarshi Ray.

https://github.com/containers/toolbox/pull/811
2021-06-29 15:27:56 +02:00
Debarshi Ray
d98f89aaa2 Update the short description to match the text on the GitHub project
https://github.com/containers/toolbox/pull/814
2021-06-26 02:42:31 +02:00
Debarshi Ray
5ac77773af test/system: Test the handling of unknown flags with each command
This is a continuation of commit 9fdf10f2e1, which added a test
for the handling of unknown flags but without specifying any command.

https://github.com/containers/toolbox/pull/812
2021-06-25 19:24:01 +02:00
Debarshi Ray
9fdf10f2e1 test/system: Test the handling of unknown flags
https://github.com/containers/toolbox/pull/802
2021-06-23 13:13:59 +02:00
Ondřej Míchal
c6c2e426e0 cmd/list: Support images without names
Some people create images manually. If such created images are recognize
as toolbox images (they have the proper labels) but do not have
a name/tag then 'toolbox list' will panic due to index being out of
range.

https://github.com/containers/toolbox/pull/800
2021-06-22 21:58:29 +02:00