Even if one of the filters, either c.gh.debarshiray.toolbox or
c.rh.component, returns an empty list then the merged list of image
names would have an element that's an empty string. Some versions of
Podman (eg., 1.1.2) throw an error if an empty string is passed as the
image reference, while newer versions (eg., the snapshots leading to
1.2.0) will ignore the empty string and list all images. Therefore, in
the former case 'toolbox list' will error out; and in the latter case,
depending on whether only one of lists was empty or both, it will
either have duplicate entries or rightly return an empty output.
Fallout from 459763be82
Commit 0ab6eb7401 introduced the com.github.debarshiray.toolbox
label as a better way to identify toolbox containers and images. It
might be necessary to support creating toolbox containers from base
toolbox images that are not appropriately labeled. eg., this would
allow users to use newer versions of the toolbox script with older base
images to debug problems that are only seen in those versions of the
images.
While no change has been made, yet, to either enforce the presence or
override the absence of the com.github.debarshiray.toolbox label it is
prudent [1] to ensure that artifacts produced by 'toolbox create' are
adequately labeled regardless of what labels the input base image had.
[1] https://en.wikipedia.org/wiki/Robustness_principlehttps://github.com/debarshiray/toolbox/pull/91
Commit 0ab6eb7401 introduced the com.github.debarshiray.toolbox
label as a better way to identify toolbox containers and images. Images
having that label should be included in 'toolbox list'.
Since 'podman images' re-formats the creation time (ie., {{.Created}})
into a human-readable form (eg., 34 seconds ago), successive
'podman images' invocations might have slightly different strings for
the same image. This becomes a problem for images that have both the
com.github.debarshiray.toolbox and com.redhat.component tags. If they
have slightly different entries for the creation time, sort(1) and
uniq(1) won't consider them as the same entry.
Moreoever, specifying multiple filters will only match images that have
both tags, instead of images that have either one of them. This will
break compatibility with older images that didn't have the
com.github.debarshiray.toolbox label.
Therefore, the images are queried as two separate lists without the
creation time. The creation time is queried in a separate step once the
two separate lists of images have been merged into one. To ensure some
human-understandable ordering, the lists are merged based on the image
names, not the image IDs.
https://github.com/debarshiray/toolbox/pull/91
Currently the toolbox script identifies toolbox images and containers
by checking whether the com.redhat.component label matches
"fedora-toolbox". However, as per the Fedora Container Guidelines [1],
the com.redhat.com label should match the Red Hat Bugzilla component
name where bugs against the image should be reported. This means that
images derived from the base fedora-toolbox image would likely end up
overwriting it.
One option would've been to mandate that all toolbox images have the
"fedora-toolbox-" prefix in their names. However, it's better to avoid
putting limitations on how images can be named. The "fedora" name
wouldn't anyway work for images based on other distributions, and not
all images are going to use the Red Hat bugzilla for tracking bugs.
It's better to use a tag that's uniquely associated with the toolbox
project, and isn't tied to a particular distribution or bug tracker.
[1] https://fedoraproject.org/wiki/Container:Guidelines
Currently, the toolbox script depends on both the buildah and podman
commands. However, both are Go programs, and like all Go programs the
absense of shared libraries leads to bigger binaries. eg., the buildah
and podman binaries are approximately 22 MB and 37 MB respectively,
whereas the flatpak binary is a mere 1.4 MB. This isn't surprising
because both buildah and podman vendor in each other, so they are
almost, but not quite, clones of each other.
Due to this, there's some nascent desire from the Endless OS folks to
reduce the dependency footprint of the toolbox script by replacing
Buildah with the corresponding Podman commands. This is a step in that
direction.
For a locally built squashed fedora-toolbox:29 image [1], the size
reported by 'podman images' goes from 630 MB to 425 MB.
[1] Using: buildah bud --squash ...
Detected by https://www.shellcheck.net/.
Note that for coloured output the escape sequences need to be
interpreted as control characters and hence the variables containing
them need to be expanded inside the format string. This is an exception
to the general rule of thumb regarding printf [1] and hence SC2059
needs to be silenced. Otherwise https://www.shellcheck.net/ would
complain:
Line 740:
printf "${LBC}Images created by toolbox${NC}\n"
^-- SC2059: Don't use variables in the printf format string.
Use printf "..%s.." "$foo".
Some changes by Debarshi Ray.
[1] https://github.com/koalaman/shellcheck/wiki/SC2059https://github.com/debarshiray/toolbox/issues/53
There's no easy way to introspect the Kerberos configuration from the
command line. eg., the credential cache type being used, or the value
of the socket_path setting that denotes which socket the KCM service
will listen on. Therefore, it's assumed that the former is KCM if the
socket's path can be parsed from the sssd-kcm.socket unit.
Given the immutable nature of Podman containers, the toolbox container
and its corresponding image will have to be re-created if the host OS
is sufficiently re-configured.
The krb5-libs package was added to the base toolbox images to ensure
the presence of the /etc/krb5.conf.d directory with the correct
permissions. Currently, the package is already pulled in by various
dependencies. Therefore, it doesn't increase the size of the base
image, but serves as a safeguard against any inadvertent changes.
https://github.com/debarshiray/toolbox/pull/74
Currently, when udisks is configured to use /run/media instead of
/media, on most operating systems, the /run/media directory is created
by udisks itself when the first mount is handled [1]. This causes
problems when creating the toolbox container, if nothing has been
mounted after the current boot, because a missing directory cannot be
bind mounted.
Fedora Silverblue is a significant exception to the above, where
rpm-ostree takes care of creating /run/media with systemd-tmpfiles [2]
during boot.
The correct long-term solution is to get udisks to create /run/media
during boot with systemd-tmpfiles by installing a snippet in
tmpfiles.d [3, 4]. Until that happens, and is widely deployed, the
toolbox needs to provide the snippet itself to make things work on
the majority of operating systems.
Note that, in case udisks is configured to use /media instead of
/run/media, then this will create an unused /run/media directory. This
is probably fine because /run/media is the default setting for udisks.
Moreover, an unused directory is way better than not being able to
access mount points from a toolbox container or having 'podman create'
fail due to a missing directory.
Based on 4a2a15f2eb and as suggested by
Daniel J Walsh.
[1] UDisks commit aa02e5fc53efdeaf
https://github.com/storaged-project/udisks/commit/aa02e5fc53efdeaf
[2] rpm-ostree commit 958dfa435e4e4a3e
https://github.com/projectatomic/rpm-ostree/commit/958dfa435e4e4a3e
[3] https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
[4] https://github.com/storaged-project/udisks/pull/641https://github.com/debarshiray/toolbox/issues/3
The current implementation of the --image flag, including the commit
message in commit 31de3ff96f that added it, was a total fiasco
and didn't match the intended behaviour or the documentation in the
manual at all. At the moment it overrides the name of the user-specific
customized image. This doesn't make sense because it's mostly an
implementation detail of the toolbox script - a way to get from the
base image to a toolbox container that's seamlessly integrated with the
host.
In other words, there's no need for a separate flag to allow having
multiple user-specific customized images from the same base image. It
already happens as a side effect of creating multiple toolbox
containers from the same base image using the --container flag.
What it really should do is override the base image so that toolbox
containers with different content can be created.
The format of a image reference is defined in:
https://github.com/moby/moby/blob/master/image/spec/v1.1.mdhttps://github.com/docker/distribution/blob/master/reference/reference.go
The code in create_toolbox_image_name is based on the ResolveName
function implemented in:
https://github.com/containers/buildah/blob/master/util/util.go
Fallout from 31de3ff96fhttps://github.com/debarshiray/toolbox/pull/73
The manuals shouldn't be installed in the top-level directory, but in
one of the sub-directories corresponding to the relevant section.
Fallout from 0a972dfccc
It's useful to know things like a running toolbox container has to be
stopped before removal or 'rm --force' needs to be used, etc.. This is
implicitly tied to the fact that entering a toolbox container is
equivalent to a 'podman start' followed by a 'podman exec'.
Otherwise, it breaks 'podman start ...' in Podman commit
52df1fa7e054d577 [1]. Even though the podman regression was fixed in
commit 21bc766ee3829776 [2], it's prudent to also fix the toolbox
script to be more correct.
This problem isn't localized to a single command, but spans across
'create' and 'enter'. If a customized toolbox image created by the
'create' command has volumes for host bind mounts, then it will break
'podman start ...' in the 'enter' command. Therefore, users need to be
encouraged to recreate both their customized toolbox images. This is
currently done through the --verbose debug logs to avoid needless
noise because the regression was restricted to development snapshots
of podman.
Toolbox containers created from images built by broken Podman versions
between commits 52df1fa7e054d577 and 21bc766ee3829776, and broken
toolbox scripts prior to this commit, will continue to not start.
Those toolboxes and their corresponding customized images need to be
re-created with at least one of the components, either podman or this
script, fixed.
Note that 'podman inspect --type container ...' doesn't have a Go
template field for "Image". However, "ImageName" gracefully falls back
to the ID for images without a human-readable name, which is arguably
better than always using an ID.
As suggested by Daniel J Walsh and Giuseppe Scrivano.
[1] https://github.com/containers/libpod/commit/52df1fa7e054d577
[2] https://github.com/containers/libpod/commit/21bc766ee3829776https://github.com/containers/libpod/issues/2441https://github.com/debarshiray/toolbox/issues/62
This is necessary for the toolbox script to work inside the toolbox
container [1]. Having the flatpak-spawn command, provided by the
flatpak-xdg-utils package, readily available in the base image makes
for a better out of the box experience.
The flatpak-xdg-utils package is not available on Fedora 28. Given
that Fedora 28 is getting close to the end of its life cycle, it's
probably not worth doing all the paperwork to offer the package there.
[1] Commit 5b3d234c9ehttps://github.com/debarshiray/toolbox/pull/54
Currently, 'toolbox list' doesn't handle errors thrown by any of the
functions used by it. If something fails, it will send the error
message to the standard error output and continue trying to list other
things.
Instead, it's better to ensure that all the internal functions have
succeeded before listing anything. If there's an error, print the error
message but don't mix it with attempts to list other things so that
the failure mode is cleaner and it's more obvious that something went
wrong.
Listing toolbox containers involves parsing the output of 'podman ps'
and rearranging the columns into a table. Parsing and rearranging
output is inherently brittle, and there's no guarantee that podman
itself won't fail. Therefore, it's prudent to sprinkle some error
handling.
Listing toolbox images involves parsing the output of 'podman images'
and rearranging the columns into a table. Parsing and rearranging
output is inherently brittle, and there's no guarantee that podman
itself won't fail. Therefore, it's prudent to sprinkle some error
handling.
Unsetting the IFS variable doesn't disable field splitting. It makes
the shell behave as if the default field separators (ie. <space>,
<tab> and <newline>) are in play. See:
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
The original reason for unsetting IFS was to disable field splitting,
which was clearly wrong. Moreover, there was no need to worry about it
in the first place, because none of the command substitutions being
performed involve multiline output where field splitting would have
become relevant.
Fallout from 5e4e63a11b
