Currently, the toolbox script depends on both the buildah and podman
commands. However, both are Go programs, and like all Go programs the
absense of shared libraries leads to bigger binaries. eg., the buildah
and podman binaries are approximately 22 MB and 37 MB respectively,
whereas the flatpak binary is a mere 1.4 MB. This isn't surprising
because both buildah and podman vendor in each other, so they are
almost, but not quite, clones of each other.
Due to this, there's some nascent desire from the Endless OS folks to
reduce the dependency footprint of the toolbox script by replacing
Buildah with the corresponding Podman commands. This is a step in that
direction.
For a locally built squashed fedora-toolbox:29 image [1], the size
reported by 'podman images' goes from 630 MB to 425 MB.
[1] Using: buildah bud --squash ...
Detected by https://www.shellcheck.net/.
Note that for coloured output the escape sequences need to be
interpreted as control characters and hence the variables containing
them need to be expanded inside the format string. This is an exception
to the general rule of thumb regarding printf [1] and hence SC2059
needs to be silenced. Otherwise https://www.shellcheck.net/ would
complain:
Line 740:
printf "${LBC}Images created by toolbox${NC}\n"
^-- SC2059: Don't use variables in the printf format string.
Use printf "..%s.." "$foo".
Some changes by Debarshi Ray.
[1] https://github.com/koalaman/shellcheck/wiki/SC2059https://github.com/debarshiray/toolbox/issues/53
There's no easy way to introspect the Kerberos configuration from the
command line. eg., the credential cache type being used, or the value
of the socket_path setting that denotes which socket the KCM service
will listen on. Therefore, it's assumed that the former is KCM if the
socket's path can be parsed from the sssd-kcm.socket unit.
Given the immutable nature of Podman containers, the toolbox container
and its corresponding image will have to be re-created if the host OS
is sufficiently re-configured.
The krb5-libs package was added to the base toolbox images to ensure
the presence of the /etc/krb5.conf.d directory with the correct
permissions. Currently, the package is already pulled in by various
dependencies. Therefore, it doesn't increase the size of the base
image, but serves as a safeguard against any inadvertent changes.
https://github.com/debarshiray/toolbox/pull/74
Currently, when udisks is configured to use /run/media instead of
/media, on most operating systems, the /run/media directory is created
by udisks itself when the first mount is handled [1]. This causes
problems when creating the toolbox container, if nothing has been
mounted after the current boot, because a missing directory cannot be
bind mounted.
Fedora Silverblue is a significant exception to the above, where
rpm-ostree takes care of creating /run/media with systemd-tmpfiles [2]
during boot.
The correct long-term solution is to get udisks to create /run/media
during boot with systemd-tmpfiles by installing a snippet in
tmpfiles.d [3, 4]. Until that happens, and is widely deployed, the
toolbox needs to provide the snippet itself to make things work on
the majority of operating systems.
Note that, in case udisks is configured to use /media instead of
/run/media, then this will create an unused /run/media directory. This
is probably fine because /run/media is the default setting for udisks.
Moreover, an unused directory is way better than not being able to
access mount points from a toolbox container or having 'podman create'
fail due to a missing directory.
Based on 4a2a15f2eb and as suggested by
Daniel J Walsh.
[1] UDisks commit aa02e5fc53efdeaf
https://github.com/storaged-project/udisks/commit/aa02e5fc53efdeaf
[2] rpm-ostree commit 958dfa435e4e4a3e
https://github.com/projectatomic/rpm-ostree/commit/958dfa435e4e4a3e
[3] https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
[4] https://github.com/storaged-project/udisks/pull/641https://github.com/debarshiray/toolbox/issues/3
The current implementation of the --image flag, including the commit
message in commit 31de3ff96f that added it, was a total fiasco
and didn't match the intended behaviour or the documentation in the
manual at all. At the moment it overrides the name of the user-specific
customized image. This doesn't make sense because it's mostly an
implementation detail of the toolbox script - a way to get from the
base image to a toolbox container that's seamlessly integrated with the
host.
In other words, there's no need for a separate flag to allow having
multiple user-specific customized images from the same base image. It
already happens as a side effect of creating multiple toolbox
containers from the same base image using the --container flag.
What it really should do is override the base image so that toolbox
containers with different content can be created.
The format of a image reference is defined in:
https://github.com/moby/moby/blob/master/image/spec/v1.1.mdhttps://github.com/docker/distribution/blob/master/reference/reference.go
The code in create_toolbox_image_name is based on the ResolveName
function implemented in:
https://github.com/containers/buildah/blob/master/util/util.go
Fallout from 31de3ff96fhttps://github.com/debarshiray/toolbox/pull/73
The manuals shouldn't be installed in the top-level directory, but in
one of the sub-directories corresponding to the relevant section.
Fallout from 0a972dfccc
It's useful to know things like a running toolbox container has to be
stopped before removal or 'rm --force' needs to be used, etc.. This is
implicitly tied to the fact that entering a toolbox container is
equivalent to a 'podman start' followed by a 'podman exec'.
Otherwise, it breaks 'podman start ...' in Podman commit
52df1fa7e054d577 [1]. Even though the podman regression was fixed in
commit 21bc766ee3829776 [2], it's prudent to also fix the toolbox
script to be more correct.
This problem isn't localized to a single command, but spans across
'create' and 'enter'. If a customized toolbox image created by the
'create' command has volumes for host bind mounts, then it will break
'podman start ...' in the 'enter' command. Therefore, users need to be
encouraged to recreate both their customized toolbox images. This is
currently done through the --verbose debug logs to avoid needless
noise because the regression was restricted to development snapshots
of podman.
Toolbox containers created from images built by broken Podman versions
between commits 52df1fa7e054d577 and 21bc766ee3829776, and broken
toolbox scripts prior to this commit, will continue to not start.
Those toolboxes and their corresponding customized images need to be
re-created with at least one of the components, either podman or this
script, fixed.
Note that 'podman inspect --type container ...' doesn't have a Go
template field for "Image". However, "ImageName" gracefully falls back
to the ID for images without a human-readable name, which is arguably
better than always using an ID.
As suggested by Daniel J Walsh and Giuseppe Scrivano.
[1] https://github.com/containers/libpod/commit/52df1fa7e054d577
[2] https://github.com/containers/libpod/commit/21bc766ee3829776https://github.com/containers/libpod/issues/2441https://github.com/debarshiray/toolbox/issues/62
This is necessary for the toolbox script to work inside the toolbox
container [1]. Having the flatpak-spawn command, provided by the
flatpak-xdg-utils package, readily available in the base image makes
for a better out of the box experience.
The flatpak-xdg-utils package is not available on Fedora 28. Given
that Fedora 28 is getting close to the end of its life cycle, it's
probably not worth doing all the paperwork to offer the package there.
[1] Commit 5b3d234c9ehttps://github.com/debarshiray/toolbox/pull/54
Currently, 'toolbox list' doesn't handle errors thrown by any of the
functions used by it. If something fails, it will send the error
message to the standard error output and continue trying to list other
things.
Instead, it's better to ensure that all the internal functions have
succeeded before listing anything. If there's an error, print the error
message but don't mix it with attempts to list other things so that
the failure mode is cleaner and it's more obvious that something went
wrong.
Listing toolbox containers involves parsing the output of 'podman ps'
and rearranging the columns into a table. Parsing and rearranging
output is inherently brittle, and there's no guarantee that podman
itself won't fail. Therefore, it's prudent to sprinkle some error
handling.
Listing toolbox images involves parsing the output of 'podman images'
and rearranging the columns into a table. Parsing and rearranging
output is inherently brittle, and there's no guarantee that podman
itself won't fail. Therefore, it's prudent to sprinkle some error
handling.
Unsetting the IFS variable doesn't disable field splitting. It makes
the shell behave as if the default field separators (ie. <space>,
<tab> and <newline>) are in play. See:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
The original reason for unsetting IFS was to disable field splitting,
which was clearly wrong. Moreover, there was no need to worry about it
in the first place, because none of the command substitutions being
performed involve multiline output where field splitting would have
become relevant.
Fallout from 5e4e63a11b
... detected by https://www.shellcheck.net/.
The opening solid bracket (ie. [) is actually a command like any other.
It expects it's final argument to be the closing bracket (ie. ]). Thus,
the redirection needs to be after the trailing bracket.
Fallout from 80f25c6924
A truly seamless developer experience requires erasing the divide
between the host and the toolbox container as much as possible.
Currently, various tools don't work at all when used from inside the
toolbox because they expect to be run on the host. eg., flatpak,
podman, and the toolbox script itself. This puts a significant enough
cognitive burden on the developer. At the very least, the human
operator needs to keep track of the context in which those commands
are being issued.
To make things better, the toolbox script has been made aware of the
context in which it is running. If it detects that it's running inside
the toolbox container, denoted by a 'podman exec ...' parent process
and the presence of /run/.containerenv, then it tries to forward its
own invocation to the host over D-Bus using 'flatpak-spawn --host' [1].
This uses the HostCommand method on the org.freedesktop.Flatpak D-Bus
service underneath to do the forwarding.
The process offering the org.freedesktop.Flatpak service doesn't have
some variables, like COLORTERM and TERM, set in its environment, and
their absence hinders the use of interactive shells. This is addressed
by tunneling the same set of environment variables that are also passed
to podman.
[1] http://docs.flatpak.org/en/latest/flatpak-command-reference.html#flatpak-spawnhttps://github.com/debarshiray/toolbox/pull/54
