Otherwise https://www.shellcheck.net/ would complain:
Line 2479:
shift
^---^ SC2317 (info): Command appears to be unreachable. Check usage
(or ignore if invoked indirectly).
See: https://www.shellcheck.net/wiki/SC2317
Fedora Rawhide now has ShellCheck-0.9.0, which flags these new problems,
while so far it only had ShellCheck-0.8.0.
ShellCheck is correct that this is unreachable code. However, given the
lack of built-in command line parsing facilities in POSIX shell, this
code pattern has so far turned out to be quite handy. It's flexible
enough to be able to handle different combinations of commands and
options, and is easy to read. Trying to 'fix' the code will likely
cause more problems than it will solve.
Moreover, the POSIX shell implementation has been replaced by the Go
implementation quite a long time ago. It's no longer maintained and has
been kept only for historical reasons. Therefore, it's not worth
spending any significant amount of time on it.
https://github.com/containers/toolbox/pull/1211
Otherwise https://www.shellcheck.net/ would complain:
Line 86:
term_just_first_character="${TERM%$term_without_first_character}"
^-- SC2295 (info): Expansions inside
${..} need to be quoted
separately, otherwise they match
as patterns.
See: https://www.shellcheck.net/wiki/SC2295
CentOS Stream 9 has ShellCheck-0.8.0, while so far the 'unit tests' were
being run on Fedora 36, which only has ShellCheck-0.7.2.
This is a step towards testing on CentOS Stream 9.
https://github.com/containers/toolbox/pull/1200
This will provide a path forward to those who stumble across the POSIX
shell implementation and don't know how to use the Go implementation.
https://github.com/containers/toolbox/pull/1094
The POSIX shell Toolbox has been replaced by the Go implementation
quite a long time ago. People on several ocassions created PRs that
still update it, or end up using it by mistake when building from
source.
It was not clear that the POSIX shell implementation has been
deprecated and is no longer maintained.
https://github.com/containers/toolbox/pull/698
The USER environment variable is set in different ways on different
operating system distributions. It's usually set when logging into a
new user session, but thereafter it may or may not get propagated into
new instances of a shell depending on how it's getting set. eg.,
Fedora sets USER in /etc/profile, but Ubuntu doesn't. This means that
shells running on a Fedora host and a Fedora toolbox container will
automatically have the USER environment variable set. However, it's
only available on Ubuntu hosts, but not on Ubuntu toolbox containers.
Passing the USER environment variable to the container, if it's set on
the host, will avoid such inconsistencies in the environment.
https://github.com/containers/toolbox/pull/609
It's necessary to propagate the XAUTHORITY environment variable from
the host when an X11 client is run as 'sudo <some-client>'. If an X11
client is started inside a 'su -' session, then xauth(1) needs to be
present so that pam_xauth.so can add a new XAUTHORITY environment
variable to the 'su -' session.
https://github.com/containers/toolbox/pull/572
... and other hybrid set-ups where the host and container OSes aren't
the same.
The entry point of a toolbox container already runs as root:root.
Therefore, there's no need to run it with an additional group.
Interactive shells spawned by 'sudo su -' both inside the container
and on the host don't run with such an additional group either. They
run just as root:root.
This prevented toolbox containers from starting up on Fedora CoreOS
hosts, because CoreOS has both the 'sudo' and 'wheel' groups but the
fedora-toolbox images only have the 'wheel' group. Therefore, it
ended up calling 'podman create --group-add sudo ...', and since the
'sudo' group was missing from the image, the container failed to start.
The --group-add flag was added in commit 4bda42d414 when the
entry point ran as $USER as specified in the user-specific customized
image. The additional group was specified to retain consistency with
interactive shells run as $USER.
Since then, things have changed. There's no longer any user-specific
customized image and commit f74400f450 made the entry point run
as root:root. The --group-add flag should have been removed as part of
those changes.
https://github.com/containers/toolbox/issues/423
It might happen that the current working directory in the present
environment is missing inside the container that's going to be used.
So far, Toolbox would fail silently, which wasn't very obvious.
Some changes by Debarshi Ray and Harry Míchal.
https://github.com/containers/toolbox/issues/369
/usr/share/profile.d is the default location where toolbox.sh is
installed, even though, in practice, most (all?) distributions use
/etc/profile.d. It's reasonable to at least make the code work with the
default build values.
https://github.com/containers/toolbox/pull/362
/sys/fs/selinux is only present when SELinux is 'enforcing' or
'permissive'. When it's disabled, /sys/fs/selinux doesn't exist and
sysfs doesn't let you create it either. Therefore, the attempt to wipe
out the toolbox container's /sys/fs/selinux by bind mounting
/usr/share/empty over it fails, and in turn prevents the container from
starting up.
Fallout from f9cca5719dhttps://github.com/containers/toolbox/issues/344
On Silverblue, /mnt is a symbolic link to /var/mnt. Matching the
presence of /var/mnt on the host inside the toolbox container would
make things less confusing for users.
https://github.com/containers/toolbox/issues/92
A subsequent commit will give access to /var/mnt from the host, if its
present, by bind mounting /run/host/var/mnt at runtime. However, it
turns out that an attempt to non-recursively bind it will error out, if
the host's /var/mnt already contains a mount point.
On the host:
$ sudo mkdir --parents /var/mnt/tmp
$ sudo mount -t tmpfs none /var/mnt/tmp
Inside the container:
$ sudo mkdir --parents /var/mnt
$ sudo mount --bind -o rslave /run/host/var/mnt /var/mnt
mount: /var/mnt: wrong fs type, bad option, bad superblock on
/run/host/var/mnt, missing codepage or helper, or other error.
https://github.com/containers/toolbox/issues/92
This is the second time a Podman regression has caused a selinuxfs
instance to leak into the toolbox container's /sys/fs/selinux,
tricking various components into trying to use SELinux. It might be
better to work this around in Toolbox until the situation in Podman is
figured out.
Based on an idea from Colin Walters.
https://github.com/containers/libpod/issues/4452
Toolbox containers created prior to commit 8b84b5e460 didn't use
'toolbox init-container' as their entry points. This prevents them
from being configured at runtime through the entry points.
Being able to configure a toolbox container at runtime through the
entry point is very handy, as compared to doing it statically via
'podman create', because the configuration doesn't get permanently
baked into the container's definition. Instead, it's codified in
toolbox(1), which can be updated over time, and the container
reconfigured everytime it's started.
A deprecation notice is the precursor to actually dropping support for
these old containers in the future.
Preliminary testing suggests that toolbox containers created prior to
commit 8b84b5e460 already don't start on cgroups v2 systems. So,
this is mainly targetted at cgroups v1 users, who are still able to
work with those old containers.
https://github.com/containers/toolbox/pull/336
Otherwise, it would lead to:
$ toolbox run
/usr/bin/toolbox: line 1287: shift: 4: shift count out of range
toolbox: command '' not found in container fedora-toolbox-31
Fallout from 2da4cc4634https://github.com/containers/toolbox/pull/332
Currently, toolbox(1) offers a --verbose option that only shows debug
information from toolbox(1) itself and the error stream of internal
commands. There's no way to further increase the log level of the
internal commands. It's sometimes very useful to be able to get more
detailed logs from Podman.
This adds a new --very-verbose or -vv option that makes this possible.
This should have been implemented as '--verbose --verbose', which
could be conveniently shortened to '-vv'. This is what flatpak(1)
does. However, due to the lack of built-in command line parsing
facilities in POSIX shell, there's no support for multiple short
options expressed as one single argument. eg., '-vy' doesn't expand to
'-v -y'.
Therefore, a separate --very-verbose or -vv option was added to make
things convenient for the user. It's expected that most people will
refer to this as -vv.
If this option is used, every Podman command in the code is run with
'--log-level debug'. Use wisely, Podman can be 'very verbose'.
https://github.com/containers/toolbox/pull/289
This makes the following work from inside a toolbox container:
$ logger "syslog: hello world"
$ python3 <<< "from systemd import journal; \
journal.send('journal: hello world')"
https://github.com/containers/toolbox/pull/327
It's now possible to use journalctl(1) to query the user's systemd
journal entries from the host. However, messages from other users and
the system aren't shown.
https://github.com/containers/toolbox/pull/327
The machine ID is necessary to query the host operating system's
systemd journal, and currently toolbox containers have an empty
/etc/machine-id file.
Unlike /etc/resolv.conf, the machine ID is supposed to stay constant
once the host is booted. Therefore, it is safe to bind mount
/etc/machine-id from the host, as opposed to using a symbolic link;
because there's no chance of the file getting atomically updated on
the host and diverging from the bind mount due to being allocated a
new inode. Incidentally, this is also what Flatpak does.
A subsequent commit will use this to enable accessing the host's
systemd journal via journalctl(1) inside toolbox containers.
https://github.com/containers/toolbox/pull/327
For what it's worth, this does alter the mount propagation flags by
adding 'slave'.
Earlier with 'podman create --volume ...' it was:
$ findmnt -o OPTIONS,PROPAGATION /run/libvirt
OPTIONS PROPAGATION
rw,nosuid,nodev,seclabel,mode=755 private
Now with 'mount --bind ...' it is:
$ findmnt -o OPTIONS,PROPAGATION /run/libvirt
OPTIONS PROPAGATION
ro,relatime,seclabel private,slave
This difference was ignored because it doesn't appear to cause any
real problem.
https://github.com/containers/toolbox/pull/327
For what it's worth, this does alter the mount propagation flags by
adding 'slave'.
Earlier with 'podman create --volume ...' it was:
$ findmnt -o OPTIONS,PROPAGATION /var/lib/flatpak
OPTIONS PROPAGATION
ro,relatime,seclabel private
Now with 'mount --bind -o ro ...' it is:
$ findmnt -o OPTIONS,PROPAGATION /var/lib/flatpak
OPTIONS PROPAGATION
ro,relatime,seclabel private,slave
This difference was ignored because it doesn't appear to cause any
real problem.
https://github.com/containers/toolbox/pull/327
Subsequent commits will use this to perform some of the bind mounts in
the toolbox container's entry point, instead of doing them as part of
'podman create ...'.
Anything that's specified during 'podman create ...' gets statically
baked into the container's configuration, and is either difficult or
impossible to change afterwards. This means that toolbox containers
created with older versions of Toolbox keep diverging from those
created with newer versions. Hence making it complicated to keep older
containers working with a newer Toolbox.
In the case of bind mounts, a good chunk of the host's file hierarchy
is already bind mounted by 'podman create ...' under the toolbox
container's /run/host. Therefore, the more granular bind mounts like
$XDG_RUNTIME_DIR and /var/lib/flatpak can be performed by the
container's entry point at runtime using what's already inside
/run/host, and reduce the footprint of the static configuration.
Older containers created with Toolbox 0.0.10 onwards will see two bind
mounts for locations that get moved from 'podman create ...' to the
entry point. The presence of the second mount should be harmless.
Based on an idea from Colin Walters.
https://github.com/containers/toolbox/pull/327
Toolbox containers using runc as their runtime don't work on host
operating systems using cgroups v2. They need to be migrated to crun.
'podman start' throws a specific error for such containers:
ERRO[0000]: oci runtime "runc" does not support CGroups V2: use
system migrate to mitigate
Error: unable to start container "fedora-toolbox-30": this version
of runc doesn't work on cgroups v2: OCI runtime error
This error is identified by the phrase "use system migrate to mitigate"
to avoid encoding any assumptions about updating from cgroups v1 to v2
or downgrading in the other direction.
If the migration fails, 'toolbox reset' is suggested as the last hope.
https://github.com/containers/toolbox/pull/309
A subsequent commit will leverage this to detect 'podman start'
failures caused by attempting to run runc-based toolbox containers on
cgroups v2 sytems, and try to migrate them if possible.
https://github.com/containers/toolbox/pull/309
Asking for the Podman version is one of the most common support
questions. So it can't hurt to have it in the debug output, especially
when the version is already being read to decide if migration is
necessary or not.
https://github.com/containers/toolbox/pull/309
It was possible to have 'podman exec' invoked against a toolbox
container before the entry point had finished initializing it. This
could lead to situations where '$USER' didn't yet exist inside the
container when 'podman exec' attempted running a binary as that user,
which would end up failing 'toolbox enter'.
There are a number of corner cases that need to be kept in mind while
implementing any kind of synchronization.
First, older containers don't use 'toolbox init-container' as their
entry point. This might mean that their start-up can't be synchronized
but they should still be kept working in their current state.
Second, once a container has been started, subsequent 'podman start'
invocations are NOPs. They won't lead to newer instances of the entry
point process being launched.
Third, the entry point process can crash or get killed due to an
out-of-band 'podman stop'. In such cases, 'toolbox enter' should not
get confused or deadlocked. It should give a meaningful error message
to the user.
Fourth, it would be nice to not have to touch the 'create' command so
that toolbox containers created with Toolbox 0.0.10 onwards can have
their start-up synchronized. This means that the host can't add any
new environment variable or bind mount to the container to agree upon
a path that's keyed by the container's identity and shared with the
host.
Given all these considerations, a timed busy loop that looks for the
presence of a stamp file, keyed by the entry point's PID, is the most
robust solution that can be verified as correct. Anything involving
file locks becomes increasingly complicated and hard to verify.
Under normal circumstances, the loop isn't expected to last more than
a few iterations. In case the entry point dies, the loop will time out
after approximately 25 seconds, the same interval as the default for
D-Bus method calls.
Some changes by Debarshi Ray based on an idea from Jan Hlaváč.
https://github.com/containers/toolbox/pull/305
POSIX only supports single digit file descriptors. Therefore, there's
value in being frugal about how we allocate them throughout the code.
The 'reset' command is very standalone and isolated from the other
code paths, because it's meant to be a last-ditch attempt to unbreak a
broken Podman installation. This can be exploited to re-use one of the
file descriptors that's used elsewhere in the code. In this case, file
descriptor number 4 is also used to control the spinner.
https://github.com/containers/toolbox/pull/305
The 'reset' command is meant to factory reset the local Podman and
Toolbox installations. Every now and then early adopters and testers of
Toolbox have to do this when their local Podman state has gotten
irrecoverably broken due to some Podman bug.
It's useful to have a command that encapsulates all the steps to do a
factory reset, as opposed to having to spell them out separately. It's
easier to document, helps with user support, and can enable less opaque
error messages that suggest a way forward when nothing is working.
Since this command is meant to be used when the Podman installation is
completely broken, it must avoid using any Podman commands at all
costs. This is why it cannot use 'podman stop' to stop any running
containers, nor can it use 'podman unshare' to delete
~/.local/share/containers when running rootless. Instead, it relies on
the user rebooting the machine for the former, and uses newgidmap(1),
newuidmap(1) and unshare(1) to reimplement 'podman unshare' for the
latter.
Note that when running as root, some care has been taken to avoid
removing directories that might be owned by the operating system. eg.,
on Fedora /var/lib/containers/sigstore is owned by the
containers-common RPM.
https://github.com/containers/toolbox/pull/295
There's no reason to fail the toolbox container's entry point, if the
container doesn't have its own copies of configuration files that are
ultimately going to be replaced with symbolic links to their host
counterparts.
https://github.com/containers/toolbox/pull/294
The older com.github.debarshiray.toolbox label is still used in most
places as an alias for the new name for the sake of simplicity and
compatibility; except in 'create', where the new label is explicitly
specified in addition to the older one to help popularize it via newly
created toolbox containers.
The older com.github.debarshiray.toolbox label should eventually be
dropped, but before that, the even older use of com.redhat.component
for tagging needs to be phased out. The com.github.debarshiray.toolbox
label was introduced in commit 0ab6eb7401, as part of Toolbox
0.0.8, right before the release of Fedora 30 [1]. Therefore,
com.redhat.component needs to stay at least until Fedora 29 is
supported.
[1] https://fedoraproject.org/wiki/Releases/30/Schedulehttps://github.com/containers/toolbox/pull/293
A year ago, when rootless Podman was in its infancy, it was often
necessary to run rootful to test and shake out bugs in Podman. Things
are lot more mature now and this hasn't been necessary in the past few
months. Therefore, it's time to sunset this option.
Removing the --sudo option doesn't break backwards compatibility
because it was neither documented nor advertised to the user in any
way. It was a hidden option only meant to be used by those hacking on
Toolbox itself.
Note that this is different from running 'sudo toolbox ...', which is a
different use-case and uses separate code paths. This is about running
the rest of toolbox(1) as non-root and only invoking the container
tools like Podman as root.
This reverts commit 66ab4da724.
https://github.com/debarshiray/toolbox/pull/285
It seems cleaner to limit the use of colour to only marking running
containers. It's redundant to mention that the containers and images
were created by Toolbox because they are being shown by 'toolbox list'
anyway; and there's a second uncoloured heading in a different case,
that differentiates containers from images.
https://github.com/debarshiray/toolbox/pull/284
This should make it more obvious which part of the two-step process of
copying /etc/profile.d/toolbox.sh into a container the strings are
coming from.
https://github.com/debarshiray/toolbox/pull/279
It's common for people to run the toolbox script straight out of the
source tree without installing it system-wide. In such cases, it's
likely that /etc/profile.d/toolbox.sh would be absent on the host, and
as a result also absent from the toolbox container.
The welcome messages and the primary shell prompt (or PS1) are set
through /etc/profile.d/toolbox.sh, so not having it does degrade the
user experience, but it's probably not severe enough to fail the 'run'
command.
This should have been part of commit 0db54946b4 which split the
copying of /etc/profile.d/toolbox.sh into a container into two steps to
avoid using 'podman cp'. It already tried to handle the missing file
in the first step, but not in the second step.
It's also nice to at least make the user aware of the situation by
printing an error message.
https://github.com/debarshiray/toolbox/pull/278
Podman defaults to bind-mounting locations as read-write when neither
'rw' nor 'ro' is explicitly specified.
On Silverblue /usr is mounted read-only on the host. Therefore, it's
not possible to bind-mount it as read-write inside the toolbox
container.
It turns out that Podman doesn't downgrade the default mount flag to
read-only when the source location is such, and this breaks creating
new toolbox containers on Silverblue. See:
https://github.com/containers/libpod/issues/4061
Fallout from d63b0a9c0fhttps://github.com/debarshiray/toolbox/pull/276
