The /usr/bin/toolbox binary is not only used to interact with toolbox
containers and images from the host. It's also used as the entry point
of the containers by bind mounting the binary from the host into the
container. This means that the /usr/bin/toolbox binary on the host must
also work inside the container, even if they have different operating
systems.
In the past, this worked perfectly well with the POSIX shell
implementation because it got intepreted by whichever /bin/sh was
available. However, the Go implementation, can run into ABI
compatibility issues because binaries built on newer toolchains aren't
meant to be run against older runtimes.
The previous approach [1] of restricting the versions of the glibc
symbols that are linked against isn't actually supported by glibc, and
breaks if the early process start-up code changes. This is seen in
glibc-2.34, which is used by Fedora 35 onwards, where a new version of
the __libc_start_main symbol [2] was added as part of some security
hardening:
$ objdump -T ./usr/bin/toolbox | grep GLIBC_2.34
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
__libc_start_main
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
pthread_detach
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
pthread_create
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
pthread_attr_getstacksize
This means that /usr/bin/toolbox binaries built against glibc-2.34 on
newer Fedoras fail to run against older glibcs in older Fedoras.
Another option is to make the host's runtime available inside the
toolbox container and ensure that the binary always runs against it.
Luckily, almost all supported containers have the host's /usr available
at /run/host/usr. This is exploited by embedding RPATHs or RUNPATHs to
/run/host/usr/lib and /run/host/usr/lib64 in the binary, and changing
the path of the dynamic linker (ie., PT_INTERP) to the one inside
/run/host.
Unfortunately, there can only be one PT_INTERP entry inside the
binary, so there must be a /run/host on the host too. Therefore, a
/run/host symbolic link is created on the host that points to the
host's /.
Based on ideas from Alexander Larsson and Ray Strode.
[1] Commit 6ad9c63180https://github.com/containers/toolbox/pull/534
[2] glibc commit 035c012e32c11e84
https://sourceware.org/git/?p=glibc.git;a=commit;h=035c012e32c11e84https://sourceware.org/bugzilla/show_bug.cgi?id=23323https://github.com/containers/toolbox/issues/821
The subsequent commit will add an entry to create a /run/host symbolic
link on the host that points to /, and it will require explicitly
skipping some of the columns. Doing the same for the existing entry
will make the file more readable.
https://github.com/containers/toolbox/issues/821
Currently, when udisks is configured to use /run/media instead of
/media, on most operating systems, the /run/media directory is created
by udisks itself when the first mount is handled [1]. This causes
problems when creating the toolbox container, if nothing has been
mounted after the current boot, because a missing directory cannot be
bind mounted.
Fedora Silverblue is a significant exception to the above, where
rpm-ostree takes care of creating /run/media with systemd-tmpfiles [2]
during boot.
The correct long-term solution is to get udisks to create /run/media
during boot with systemd-tmpfiles by installing a snippet in
tmpfiles.d [3, 4]. Until that happens, and is widely deployed, the
toolbox needs to provide the snippet itself to make things work on
the majority of operating systems.
Note that, in case udisks is configured to use /media instead of
/run/media, then this will create an unused /run/media directory. This
is probably fine because /run/media is the default setting for udisks.
Moreover, an unused directory is way better than not being able to
access mount points from a toolbox container or having 'podman create'
fail due to a missing directory.
Based on 4a2a15f2eb and as suggested by
Daniel J Walsh.
[1] UDisks commit aa02e5fc53efdeaf
https://github.com/storaged-project/udisks/commit/aa02e5fc53efdeaf
[2] rpm-ostree commit 958dfa435e4e4a3e
https://github.com/projectatomic/rpm-ostree/commit/958dfa435e4e4a3e
[3] https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
[4] https://github.com/storaged-project/udisks/pull/641https://github.com/debarshiray/toolbox/issues/3
