Commit graph

1114 commits

Author SHA1 Message Date
Nieves Montero
c1e238f689 Add findutils and sudo to missing docs for image f38
Signed-off-by: Nieves Montero <nmontero@redhat.com>
2022-10-13 07:18:06 -04:00
Jonathan Lebon
1f7b9d0d75 images: add findutils and sudo to missing docs for f36+
Noticed today that `man xargs` was returning the POSIX manpage instead
of the one shipped by `findutils`.

Signed-off-by: Jonathan Lebon <jonathan@jlebon.com>
2022-10-13 07:18:06 -04:00
Nieves Montero
d85c71795d Add new packages to Fedora 38
The following packages have also been added to Fedora 38 image:
mesa-dri-drivers
mesa-vulkan-drivers
vulkan-loader
Fixing up fedora 38 image to match the changes made earlier on fedora 37.

Signed-off-by: Nieves Montero <nmontero@redhat.com>
2022-09-29 08:09:20 -04:00
Nieves Montero
e6a27d7926 Add package glibc-all-langpacks to f37 and f38
This new packet allows the user to set a locale inside the
toolbox and make locale dependent commands work

https://github.com/containers/toolbox/issues/60

Signed-off-by: Nieves Montero <nmontero@redhat.com>
2022-09-29 08:08:59 -04:00
Ondřej Míchal
8f6deadaef test/system: Drop extra pull when caching images
In 54a2ca1 image caching has been done by first pulling using Podman and
then moving the image from the local container store to a directory. The
pull to the local container store can be skipped and instead we can use
Skopeo to directly save the pulled image into a directory.

On my machine this reduced the time of the system test setup "test" by
about 50 seconds. This speed-up largely depends on the available network
connection, though.
2022-09-21 14:49:21 +02:00
Nieves Montero
f10fe7fbb2 Added directory for f38 image
https://github.com/containers/toolbox/issues/1111

Signed-off-by: Nieves Montero <nmontero@redhat.com>
2022-09-21 11:41:23 +02:00
Nieves Montero
2f7c8586ef Added new packages to images f36 AND f35
The following packages have also been added to images f36 and f35:
mesa-dri-drivers
mesa-vulkan-drivers
vulkan-loader

https://github.com/containers/toolbox/pull/1124
Signed-off-by: Nieves Montero <nmontero@redhat.com>
2022-09-14 07:00:57 -04:00
Nieves Montero
a87fd19980 Added new packages
The following packages have been added to the
image to make OpenGL and Vulkan work:
mesa-dri-drivers
mesa-vulkan-drivers
vulkan-loader

https://github.com/containers/toolbox/issues/1110
Signed-off-by: Nieves Montero <nmontero@redhat.com>
2022-09-14 07:00:57 -04:00
Debarshi Ray
01d3510141 build: Remove a redundant message
If systemd-tmpfiles(8) couldn't be spawned, then the attempted command
is already included in the traceback:
  Traceback (most recent call last):
    File "toolbox/meson_post_install.py", line 26, in <module>
      subprocess.run(['systemd-tmpfiles', '--create'], check=True)
    File "/usr/lib64/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['systemd-tmpfiles',
      '--create']' returned non-zero exit status 73.

https://github.com/containers/toolbox/pull/1122
2022-09-09 18:36:02 +02:00
Debarshi Ray
a0569fdc3e build: Don't try to handle exceptions when spawning subprocesses
In short, it's a lot of effort to cover all possible exceptions that can
be thrown, and things work reasonably well even without handling them.
Since this is just part of the build, there's no point in complicating
things for aesthetic reasons.

More details below.

First, not every runtime error leads to a subprocess.CalledProcessError.
It's only thrown if the spawned process returns with a non-zero exit
code.  There can be other problems.  eg., if the gofmt file isn't
executable then a PermissionError is thrown that's currently not
handled, and the wrapper Python script returns with a non-zero exit
code:
  Traceback (most recent call last):
    File "toolbox/src/meson_go_fmt.py", line 28, in <module>
      gofmt = subprocess.run(['gofmt', '-d', source_dir],
          capture_output=True, check=True)
    File "/usr/lib64/python3.10/subprocess.py", line 501, in run
      with Popen(*popenargs, **kwargs) as process:
    File "/usr/lib64/python3.10/subprocess.py", line 969, in __init__
      self._execute_child(args, executable, preexec_fn, close_fds,
    File "/usr/lib64/python3.10/subprocess.py", line 1845, in
        _execute_child
      raise child_exception_type(errno_num, err_msg, err_filename)
  PermissionError: [Errno 13] Permission denied: 'gofmt'

Second, when a subprocess.CalledProcessError is thrown, the wrapper
Python script will still return with a non-zero exit code with an
understandable error message, even if the exception isn't handled.  eg.,
if 'meson install' is called without the adequate permissions, then
systemd-tmpfiles(8) will return with a non-zero exit code, which shows
up as:
  --- stdout ---
  Calling systemd-tmpfiles --create ...

  --- stderr ---
  Failed to open directory 'cryptsetup': Permission denied
  Failed to open directory 'certs': Permission denied
  Failed to create directory or subvolume "/var/spool/cups/tmp":
    Permission denied
  ...
  ...
  ...
  Traceback (most recent call last):
    File "toolbox/meson_post_install.py", line 26, in <module>
      subprocess.run(['systemd-tmpfiles', '--create'], check=True)
    File "/usr/lib64/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['systemd-tmpfiles',
      '--create']' returned non-zero exit status 73.

Similarly, if there problems generating the shell completions:
  --- stderr ---
  Error: unknown command "__completion" for "toolbox"
  Run 'toolbox --help' for usage.
  exit status 1
  Traceback (most recent call last):
    File "toolbox/completion/generate_completions.py", line 35, in
        <module>
    output = subprocess.run(['go', 'run', '.', '__completion',
        completion_type], check=True)
    File "/usr/lib64/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['go', 'run', '.',
      '__completion', 'bash']' returned non-zero exit status 1.

https://github.com/containers/toolbox/pull/1122
2022-09-09 18:35:55 +02:00
Ondřej Míchal
9d1b5887ae Revert "cmd/completion: Add prefix to command to hide it better"
Cobra provides a default command 'completion' that is always visible.
The reverted change caused an additional command 'completion' to show up
in the list because the then called command '__completion' didn't
override the default one. This became apparent due to d69ce6794b
dynamically generating completion arguments for the 'help' command.

This reverts commit 4469774fb1.

https://github.com/containers/toolbox/pull/1055
https://github.com/containers/toolbox/pull/1121
2022-09-09 17:58:31 +02:00
Debarshi Ray
0212ce0db9 build: Tweak the names of the ShellCheck tests
https://github.com/containers/toolbox/pull/1120
2022-09-09 17:16:25 +02:00
Debarshi Ray
e6c0c00d79 build: Disambiguate the 'toolbox' file and target
Fallout from 6c0b045e1a

https://github.com/containers/toolbox/pull/1120
2022-09-09 17:13:31 +02:00
Ondřej Míchal
66d0418595 cmd/completion: Use Cobra's Bash completion v2
Bash completion v2 was introduced in Cobra v1.2.0 and v1 will be
deprecated in the future:
https://github.com/spf13/cobra/releases/tag/v1.2.0

Since Toolbox already requires Cobra >= v1.3.0, it's better to use the
new Bash completion.

Fallout from d69ce6794b

https://github.com/containers/toolbox/pull/1055
https://github.com/containers/toolbox/pull/1119
2022-09-09 12:10:57 +02:00
Ondřej Míchal
9bdbb55741 build: Fix typo in install dir for Z shell completions
Fallout from bafbbe81c9

https://github.com/containers/toolbox/pull/1055
https://github.com/containers/toolbox/pull/1118
2022-09-08 23:00:08 +02:00
Debarshi Ray
af2c5325ef build: Wire go-build-wrapper's output to Meson's
Note that Meson's '@OUTPUT@' is not just the basename of the output, but
includes the relative path under the project's root directory.

https://github.com/containers/toolbox/pull/1117
2022-09-08 22:01:16 +02:00
Debarshi Ray
14f02b244d build: Remove redundant build_by_default
By default, the value of the 'build_by_default' argument is determined
by the value of the 'install' argument, which was set to 'true' once the
Go implementation was considered stable enough for end users.

Fallout from 0b3c66434e

https://github.com/containers/toolbox/pull/1116
2022-09-08 20:34:37 +02:00
Ondřej Míchal
6c0b045e1a build: Make the completion depend on the Toolbx binary
Fallout from bafbbe81c9

https://github.com/containers/toolbox/pull/1055
https://github.com/containers/toolbox/pull/1115
2022-09-08 19:59:17 +02:00
Debarshi Ray
284a2bdf39 cmd/completion: Simplify code
Fallout from d69ce6794b

https://github.com/containers/toolbox/pull/1114
2022-09-08 17:53:27 +02:00
Debarshi Ray
7cad4dc60c cmd/completion: Add copyright and license notices
https://github.com/containers/toolbox/pull/1114
2022-09-08 17:53:24 +02:00
Ondřej Míchal
d9085dd70c cmd/completion: Remove unneeded documentation
We don't provide completion for PowerShell, we support only Linux and
instructions for loading completion files are redundant in Toolbx.

Fallout from d69ce6794b

https://github.com/containers/toolbox/pull/1055
https://github.com/containers/toolbox/pull/1113
2022-09-08 16:40:04 +02:00
Ondřej Míchal
bcc3dc93f5 cmd: Don't use Logrus to call panic
While the use of Logrus is convenient, it causes unneeded fluff to be
printed during a panic which distracts from finding the cause of the
panic in the first place.

Fallout from d69ce6794b

https://github.com/containers/toolbox/pull/1055
https://github.com/containers/toolbox/pull/1112
2022-09-08 16:18:13 +02:00
Debarshi Ray
32b147b9ff cmd/create: Improve the error messages for mutually exclusive options
https://github.com/containers/toolbox/pull/1109
2022-09-07 19:20:17 +02:00
Debarshi Ray
947e582c0f cmd/create: Improve the error message for the --authfile option
https://github.com/containers/toolbox/pull/1109
2022-09-07 19:20:05 +02:00
Debarshi Ray
ac00c06c97 doc/toolbox-create: Mention the file format accepted by --authfile
https://github.com/containers/toolbox/pull/1108
2022-09-07 17:02:56 +02:00
Debarshi Ray
44e9b1473f doc/toolbox-create: Tweak an example for consistency
When describing the --authfile option, the word 'private' is used to
refer to images needing authentication.  Using the same word shortens
the text so that the word 'custom' can be used in the same way as in the
other examples.

https://github.com/containers/toolbox/pull/1107
2022-09-07 16:35:12 +02:00
Debarshi Ray
00def007f5 pkg/utils: Address the confusion around handling errors from Viper
It turns out that Viper's custom error implementations use non-pointer
receivers, whereas often people assume pointer receivers.  This can
cause confusion when trying to use errors.As(...) with those errors [1].

Secondly, Viper may or may not throw ConfigFileNotFoundError depending
on its build tags.

[1] https://github.com/spf13/viper/issues/1139

https://github.com/containers/toolbox/pull/1105
2022-09-02 18:51:32 +02:00
Debarshi Ray
53c5694040 cmd/utils, pkg/utils: Improve an error message for the image option
https://github.com/containers/toolbox/pull/1104
2022-09-02 15:19:00 +02:00
Debarshi Ray
8454b31a82 cmd/utils, pkg/utils: Improve the error messages for the distro option
https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1103
2022-09-02 14:52:44 +02:00
Debarshi Ray
8ca5611942 Increase the validation coverage for the container & release options
Currently, the container name and release are only validated if they
were specified as command line options.  Neither the value of release
in the configuration file nor the container name generated from an
image are validated.

There's also a lot of repeated code in the command front-ends to
validate the container name and release.  This opens the door for
mistakes.  Any adjustment to the code must be repeated elsewhere, and
there are subtle interactions and overlaps between the validation code
and the code to resolve container and image names.

It's worth noting that the container and image name resolution happens
for both the command line and configuration file options, and generates
the container name from the image when necessary.

Therefore, validating everything while resolving cleans up the command
front-ends and increases the coverage of the validation.

This introduces the use of sentinel error values and custom error
implementations to identify the different errors that can occur while
resolving the container and images, so that they can be appropriately
shown to the user.

https://github.com/containers/toolbox/pull/1101
2022-09-02 13:11:32 +02:00
Debarshi Ray
b5474bff84 cmd, pkg/utils: Clarify the error message if the release is invalid
Currently, if --release has an invalid argument, the error message
doesn't give any hints as to what's an acceptable value.  This can be
confusing.  eg., is 36 a valid argument for Fedora?  Or is it f36?  Or
is it F36?  Is 'rawhide' accepted?

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1100
2022-09-01 17:57:39 +02:00
Debarshi Ray
aead0023e3 pkg/utils: Rename a variable
This will make the subsequent commit easier to read.

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1100
2022-09-01 17:57:35 +02:00
Debarshi Ray
df7e01df10 pkg/utils: Ensure that the distro CLI and config file options are valid
Currently, if an invalid or unsupported string is specified as the
distro on the command line or in the configuration file, then it would
silently fallback to Fedora.  This shouldn't happen.

It should only fallback to Fedora when no distro was specified and
there's no supported Toolbox image matching the host operating system.
If a distro was explicitly specified then it should either be supported
or it should error out.

The test cases were resurrected from commit 8b6418d8aa.

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1080
2022-09-01 17:43:20 +02:00
Debarshi Ray
f5bcd22ade pkg/utils: Clarify the default and fallback values
The terms 'default' and 'fallback' are used to mean very specific
things in this context.

The 'default' values are those that are used when the 'create', 'enter'
and 'run' commands were used without any option.  These values are
picked to match the host operating system.

However, if there's no supported Toolbox image matching the host
operating system, and no options were provided to the 'create', 'enter'
and 'run' commands, then the 'fallback' values are used as a last
resort.

Consistently using this terminology leads to a clear mental model and
makes the code easier to read.

This rough arrangement of the code was already being used for
'release', and has now been been extended to 'container name prefix'
and 'distro'.  The suffix for the 'fallback' values was simplified to
'Fallback', instead of 'DefaultFallback'.

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1080
2022-09-01 17:43:15 +02:00
Debarshi Ray
344dda6d8d pkg/utils: Re-unify container & image name resolution
The idea of splitting ResolveContainerAndImageNames into two public
functions [1] didn't turn out to be so useful [2].  It pushed the
burden on the callers, who needed to carefully call the split
functions in the right order, because the container, distro, image and
release values are very tightly related.  This opens the door for
mistakes.

A better approach would be to restore ResolveContainerAndImageNames as
the single public API.  If necessary it could be internally split into
smaller private functions.  It would keep things simple for the
callers.

Note that this commit doesn't include the private split.  If necessary,
it can be done in future.

This reverts commit fd756089ef.

[1] Commit fd756089ef
    https://github.com/containers/toolbox/pull/828
    https://github.com/containers/toolbox/pull/838

[2] https://github.com/containers/toolbox/pull/977

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1080
2022-09-01 17:43:10 +02:00
Debarshi Ray
0e66af91fe Revert "cmd, pkg/utils: Split distro and release parsing and ..."
The idea of splitting ResolveContainerAndImageNames into two public
functions [1] didn't turn out to be so useful [2].  Splitting things
even further might make it worse.  A better approach might be to
(re-)unify the code further.

This is the first step towards that.

This reverts the following commits:
  * 5c8ad7a7ec
  * 02f45fd3f2
  * 8b6418d8aa

... but retains the test cases that were not tied to the changes in
those commits.

[1] Commit fd756089ef
    https://github.com/containers/toolbox/pull/828
    https://github.com/containers/toolbox/pull/838

[2] https://github.com/containers/toolbox/pull/977

https://github.com/containers/toolbox/issues/937
https://github.com/containers/toolbox/pull/1080
2022-09-01 17:43:04 +02:00
Debarshi Ray
4f78c5ef86 cmd/utils, test/system: Tweak an error message for consistency
Barring the first line, all other lines are terminated with a full stop
elsewhere.

https://github.com/containers/toolbox/pull/1099
2022-09-01 13:02:40 +02:00
Debarshi Ray
59f71219b7 pkg/utils: Mark a private function as such
Figuring out the container name prefix for a given image only needs to
happen as part of resolving the final Toolbox container name from the
given command line and configuration options.

Fallout from c990fb43ca

https://github.com/containers/toolbox/pull/1098
2022-08-31 21:06:51 +02:00
Debarshi Ray
e6c9c0c925 pkg/utils: Be more strict about what is acceptable
https://github.com/containers/toolbox/pull/1097
2022-08-31 20:26:21 +02:00
Debarshi Ray
b864280e42 playbooks: Make all Meson warnings fatal
This should help detect the kind of problem that was fixed in the
previous commit.

https://github.com/containers/toolbox/pull/1096
2022-08-31 19:35:05 +02:00
Debarshi Ray
1567d207c6 build: Silence a WARNING
Otherwise, Meson complains:
  completion/meson.build:4: WARNING: Project targeting '>= 0.58.0' but
    tried to use feature deprecated since '0.56.0':
    dependency.get_pkgconfig_variable. use
    dependency.get_variable(pkgconfig : ...) instead

Fallout from bafbbe81c9

https://github.com/containers/toolbox/pull/1096
2022-08-31 19:35:01 +02:00
Debarshi Ray
4dd73ad160 .zuul, playbooks: Run unit tests on -Dmigration_path_for_coreos_toolbox
The -Dmigration_path_for_coreos_toolbox option enables a different code
path that's currently not tested by the CI at all.  In fact, since it's
a build-time option, the corresponding code path is not even built by
the CI.

To properly support the -Dmigration_path_for_coreos_toolbox option, it
needs to be covered by the CI.  This is a step in that direction by
running the unit tests on it.

https://github.com/containers/toolbox/pull/1095
2022-08-31 13:42:40 +02:00
Debarshi Ray
f3a15c60fe playbooks: Split out the post-configuration steps into a separate file
A subsequent commit will introduce builds performed with the
-Dmigration_path_for_coreos_toolbox option to the CI.  It will be good
to avoid duplicating the build and installation steps for builds with
and without the -Dmigration_path_for_coreos_toolbox option.

https://github.com/containers/toolbox/pull/1095
2022-08-31 13:19:33 +02:00
Debarshi Ray
e965dac9f6 playbooks: Split out the dependencies into a separate file
A subsequent commit will introduce builds performed with the
-Dmigration_path_for_coreos_toolbox option to the CI.  It will be good
to avoid duplicating the installation of RPM packages, Git submodule
handling, and the listing of various debug and version information for
builds with and without -Dmigration_path_for_coreos_toolbox option.

https://github.com/containers/toolbox/pull/1095
2022-08-31 12:46:44 +02:00
Debarshi Ray
6f4e8b97dd Link to the installation guide from the shell Toolbox
This will provide a path forward to those who stumble across the POSIX
shell implementation and don't know how to use the Go implementation.

https://github.com/containers/toolbox/pull/1094
2022-08-29 22:09:15 +02:00
Debarshi Ray
56a18d2a15 Revert "build: Drop ShellCheck on Shell Toolbox"
The subsequent commit will touch the POSIX shell implementation, and
hence ShellCheck needs to be run on it.

As long as the POSIX shell implementation is part of the Git repository,
ShellCheck needs to keep running on it, unless it causes some serious
problems.  The ShellCheck test is very fast, and the reassurance and
mental peace that it provides is invaluable.

This reverts commit 8c1d441916.

https://github.com/containers/toolbox/pull/1094
2022-08-29 22:09:07 +02:00
Debarshi Ray
486762925d cmd: Refactor common code into a function
https://github.com/containers/toolbox/pull/1093
2022-08-29 17:49:00 +02:00
Debarshi Ray
a9a5b96ec6 test/system: Fix typo
This isn't causing any problems at the moment.  However, the test can
break if the order in which the command line arguments are validated
changes.  eg., if the presence of a command is checked before the
release, then the error message will be different.

Fallout from 8b6418d8aa

https://github.com/containers/toolbox/pull/1091
2022-08-26 17:00:56 +02:00
Debarshi Ray
afb002c90c .mailmap: Canonicalize my email
From now on, Debarshi Ray <debarshir@gnome.org> will show up as
Debarshi Ray <rishi@fedoraproject.org>.

Toolbox isn't quite a GNOME project (it doesn't use elements from the
GNOME platform, like GLib), even though it's part of the same
ecosystem and many Toolbox contributors are also GNOME contributors.

Toolbox was conceived to improve the developer experience on Fedora
Silverblue, and expanded over time to cover other use-cases (eg.,
troubleshooting the operating system) and Fedora editions (eg., CoreOS
and Workstation).  Even though there's a growing number of users on
other distributions, they are not the primary reason for Toolbox to
exist.

Toolbox heavily depends on Podman, and as a result is more aligned with
the Containers organization on GitHub than anything else, which is
driven, to a large degree, by Fedora contributors.

Hence, my desire to use my Fedora identity.

https://github.com/containers/toolbox/pull/1083
2022-08-01 18:37:43 +02:00
Debarshi Ray
978bb524e4 test/system: Silence warning with Bats >= 1.7.0
Bats 1.7.0 emits a warning if a command passed to 'run' returns with an
exit code of 127 [1]:
  BW01: `run`'s command `/opt/bin/toolbox run non-existent-command`
    exited with code 127, indicating 'Command not found'. Use run's
    return code checks, e.g. `run -127`, to fix this message.
        (from function `run' in file
          /usr/lib/bats-core/test_functions.bash, line 299,
         in test file test/system/104-run.bats, line 148)

This requires Bats >= 1.5.0, which is present in Fedora >=35, and
supports specifying the exit code as an argument to Bats' 'run'
command [2].

However, bats_require_minimum_version can't be used, because it's
only available from Bats 1.7.0, which is new enough that it's absent
from Fedora 35.

[1] Bats commit c6dc2f88361a4f5b
    https://github.com/bats-core/bats-core/issues/547
    https://bats-core.readthedocs.io/en/stable/warnings/BW01.html

[2] https://github.com/bats-core/bats-core/pull/367
    https://github.com/bats-core/bats-core/pull/507
    https://bats-core.readthedocs.io/en/stable/writing-tests.html

[3] Bats commit 71d6b71cebc3d32b
    https://github.com/bats-core/bats-core/issues/556
    https://bats-core.readthedocs.io/en/stable/warnings/BW02.html

https://github.com/containers/toolbox/pull/1081
2022-08-01 10:47:13 +02:00