The /usr/bin/toolbox binary is not only used to interact with toolbox
containers and images from the host. It's also used as the entry point
of the containers by bind mounting the binary from the host into the
container. This means that the /usr/bin/toolbox binary on the host must
also work inside the container, even if they have different operating
systems.
In the past, this worked perfectly well with the POSIX shell
implementation because it got intepreted by whichever /bin/sh was
available.
The Go implementation also mostly worked so far because it's largely
statically linked, with the notable exception of the standard C
library. However, recently glibc-2.32, which is used by Fedora 33
onwards, added a new version of the pthread_sigmask symbol [1] as part
of the libpthread removal project:
$ objdump -T /usr/bin/toolbox | grep GLIBC_2.32
0000000000000000 DO *UND* 0000000000000000 GLIBC_2.32
pthread_sigmask
This means that /usr/bin/toolbox binaries built against glibc-2.32 on
newer Fedoras pick up the latest version of the symbol and fail to run
against older glibcs in older Fedoras.
One way to fix this is to disable the use of any C code from Go by
using the CGO_ENABLED environment variable [2]. However, this can
negatively impact packages like "os/user" [3] and "net" [4], where the
more featureful glibc APIs will be replaced by more limited
equivalents written only in Go.
Instead, since glibc uses symbol versioning, it's better to tell the
Go toolchain to avoid linking against any symbols from glibc-2.32.
This was accomplished by a few linker tricks:
* The GNU ld linker's --wrap flag was used when building the Go code
to divert pthread_sigmask invocations from Go to another function
called __wrap_pthread_sigmask.
* A static library was added to provide this __wrap_pthread_sigmask
function, which forwards calls to the actual pthread_sigmask API in
glibc. This library itself was not linked with --wrap, and
specifies the latest permissible version of the pthread_sigmask
symbol from glibc for each architecture. Currently, the list of
architectures covers the ones that Fedora builds for.
* The Go cmd/link linker was switched to external mode [5]. This
ensures that the final object file containing all the Go code gets
linked to the standard C library and the wrapper static library by
the GNU ld linker for the --wrap flag to kick in.
Based on ideas from Ondřej Míchal.
[1] glibc commit c6663fee4340291c
https://sourceware.org/git/?p=glibc.git;a=commit;h=c6663fee4340291c
[2] https://golang.org/cmd/cgo/
[3] https://golang.org/pkg/os/user/
[4] https://golang.org/pkg/net/
[5] https://golang.org/src/cmd/cgo/doc.gohttps://github.com/containers/toolbox/issues/529
Meson doesn't support Go [1], so this is implemented by a custom target
that invokes 'go build' to generate the binary.
Unfortunately, when using Go modules, 'go build' insists on being
invoked in the same source directory where the go.mod file lives,
while Meson insists on using a build directory separate from the
corresponding source directory. This is addressed by using a build
script that goes into the source directory and then invokes 'go build'.
Currently, the Go code is only built when a Go implementation is found,
and even then, it's not installed. Non-technical end-users are supposed
to continue using the POSIX shell implementation until the Go version
is blessed as stable.
[1] https://github.com/mesonbuild/meson/issues/123https://github.com/containers/toolbox/pull/318
Make using toolbox a bit more convenient by properly completing its
options. The completions should be complete (that is, there are
completions for all the commands and options shown in --help),
but no attempt is made to filter out conflicting options (for
example "toolbox rm --all my-container").
https://github.com/debarshiray/toolbox/pull/133
Currently, when udisks is configured to use /run/media instead of
/media, on most operating systems, the /run/media directory is created
by udisks itself when the first mount is handled [1]. This causes
problems when creating the toolbox container, if nothing has been
mounted after the current boot, because a missing directory cannot be
bind mounted.
Fedora Silverblue is a significant exception to the above, where
rpm-ostree takes care of creating /run/media with systemd-tmpfiles [2]
during boot.
The correct long-term solution is to get udisks to create /run/media
during boot with systemd-tmpfiles by installing a snippet in
tmpfiles.d [3, 4]. Until that happens, and is widely deployed, the
toolbox needs to provide the snippet itself to make things work on
the majority of operating systems.
Note that, in case udisks is configured to use /media instead of
/run/media, then this will create an unused /run/media directory. This
is probably fine because /run/media is the default setting for udisks.
Moreover, an unused directory is way better than not being able to
access mount points from a toolbox container or having 'podman create'
fail due to a missing directory.
Based on 4a2a15f2eb and as suggested by
Daniel J Walsh.
[1] UDisks commit aa02e5fc53efdeaf
https://github.com/storaged-project/udisks/commit/aa02e5fc53efdeaf
[2] rpm-ostree commit 958dfa435e4e4a3e
https://github.com/projectatomic/rpm-ostree/commit/958dfa435e4e4a3e
[3] https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
[4] https://github.com/storaged-project/udisks/pull/641https://github.com/debarshiray/toolbox/issues/3
The "fedora" prefix was used because this project was specifically
incubated to make it easier to hack on Fedora Silverblue. That and the
mix of upstream technologies (ie., Buildah and Podman) made it uniquely
"Fedora".
However, over time it has gotten clear that other groups, currently
Fedora downstreams like RHEL, are interested in it too. It won't be
surprising if in future it transcends the Fedora universe altogether.
Moreover, this project was inspired by coreos/toolbox [1]. There are
good reasons and enough interest to have a unified toolbox project
that addresses the needs of both Fedora CoreOS and Silverblue.
Therefore, it is best to drop the "fedora" prefix and call the whole
thing just "toolbox".
No extra effort was made to retain compatibility with the older name
due to the project's young age. Its userbase is limited to the earliest
of early adopters, and the benefits of a clean break outweigh the
loss of compatibility.
The OCI images and the toolbox container still retain the "fedora"
prefix to disambiguate them from their counterparts from other
operating systems.
[1] https://github.com/coreos/toolboxhttps://github.com/debarshiray/toolbox/issues/8
