The path of the dynamic linker (ie., PT_INTERP), as specified in an
architecture's ABI, often starts with /lib or /lib64, not /usr/lib or
/usr/lib64. eg., it's /lib/ld-linux-aarch64.so.1 for aarch64 and
/lib64/ld-linux-x86-64.so.2 for x86_64.
Unfortunately, until very recently [1], only the host's /usr was
present inside a toolbox container's /run/host, not /lib or /lib64.
Therefore, simply prepending /run/host to the /usr/bin/toolbox
binary's existing PT_INTERP entry wouldn't locate the host's dynamic
linker inside the toolbox container. This broke backwards compatibility
with every container out there, except the ones created with the
current development version in Git.
To restore backwards compatibility, the /lib and /lib64 symbolic links
must be resolved to their respective locations inside /usr.
The following caveats must be noted:
* With glibc, even the basename of the path of the dynamic linker as
specified in an architecture's ABI, is a symbolic link to a file
named ld-<glibc-version>.so. However, this file can't be used as
the PT_INTERP entry, because its name will change when glibc is
updated and the PT_INTERP entry will become invalid until the
/usr/bin/toolbox binary is rebuilt.
* On Debian, a path like /lib64/ld-linux-x86-64.so.2 doesn't resolve
to something inside /usr/lib64. Instead it ends up inside
/usr/lib/x86_64-linux-gnu through a series of symbolic links:
- /lib64 -> usr/lib64
- /usr/lib64/ld-linux-x86-64.so.2
-> /lib/x86_64-linux-gnu/ld-2.28.so
- /lib -> usr/lib
* It's assumed that a symbolic link with the basename specified in
the ABI lives in the same directory as the actual dynamic linker
binary named ld-<glibc-version>.so.
Fallout from 6063eb27b9
[1] Commit d03a5fee80https://github.com/containers/toolbox/pull/827https://github.com/containers/toolbox/issues/821
PR #897 made adjustmnets to the Toolbx binary that it requires presence
of /run/host in both the host filesystem and the filesystem in
a container.
The presence of the directory is assured by systemd-tmpfiles by
running it before the binary is started for the first time. For the run
to be effective 'data/tmpfiles.d/toolbox.conf' has to be installed in
a location visible to systemd-tmpfiles. Therefore, the call to
'systemd-tmpfiles --create' had to be placed after the install step.
https://github.com/containers/toolbox/pull/898
There is no significant benefit in keeping this configuration separated.
Now the to-be installed packages are tracked in a single place and the
test playbooks only call the relevant tests.
This was pointed out by in 6063eb27b9https://github.com/containers/toolbox/pull/898
The /usr/bin/toolbox binary is not only used to interact with toolbox
containers and images from the host. It's also used as the entry point
of the containers by bind mounting the binary from the host into the
container. This means that the /usr/bin/toolbox binary on the host must
also work inside the container, even if they have different operating
systems.
In the past, this worked perfectly well with the POSIX shell
implementation because it got intepreted by whichever /bin/sh was
available. However, the Go implementation, can run into ABI
compatibility issues because binaries built on newer toolchains aren't
meant to be run against older runtimes.
The previous approach [1] of restricting the versions of the glibc
symbols that are linked against isn't actually supported by glibc, and
breaks if the early process start-up code changes. This is seen in
glibc-2.34, which is used by Fedora 35 onwards, where a new version of
the __libc_start_main symbol [2] was added as part of some security
hardening:
$ objdump -T ./usr/bin/toolbox | grep GLIBC_2.34
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
__libc_start_main
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
pthread_detach
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
pthread_create
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34
pthread_attr_getstacksize
This means that /usr/bin/toolbox binaries built against glibc-2.34 on
newer Fedoras fail to run against older glibcs in older Fedoras.
Another option is to make the host's runtime available inside the
toolbox container and ensure that the binary always runs against it.
Luckily, almost all supported containers have the host's /usr available
at /run/host/usr. This is exploited by embedding RPATHs or RUNPATHs to
/run/host/usr/lib and /run/host/usr/lib64 in the binary, and changing
the path of the dynamic linker (ie., PT_INTERP) to the one inside
/run/host.
Unfortunately, there can only be one PT_INTERP entry inside the
binary, so there must be a /run/host on the host too. Therefore, a
/run/host symbolic link is created on the host that points to the
host's /.
Based on ideas from Alexander Larsson and Ray Strode.
[1] Commit 6ad9c63180https://github.com/containers/toolbox/pull/534
[2] glibc commit 035c012e32c11e84
https://sourceware.org/git/?p=glibc.git;a=commit;h=035c012e32c11e84https://sourceware.org/bugzilla/show_bug.cgi?id=23323https://github.com/containers/toolbox/issues/821
The subsequent commit will add an entry to create a /run/host symbolic
link on the host that points to /, and it will require explicitly
skipping some of the columns. Doing the same for the existing entry
will make the file more readable.
https://github.com/containers/toolbox/issues/821
Currently, 'toolbox enter' can get into a loop if the user tried to
run something inside the shell that didn't exist, and quit immediately
afterwards:
$ toolbox enter
⬢$ foo
bash: foo: command not found
⬢$
logout
Error: command /bin/bash not found in container fedora-toolbox-34
Using /bin/bash instead.
⬢$
This is because:
* The shell forwards the exit code of the last command that was
invoked as its own exit code. If the last command that was
attempted was absent then this exit code is 127.
* 'podman exec' uses 127 as the exit code when it can't invoke the
command. If it's able to successfully invoke the command, it
forwards the exit code of the command itself.
Therefore, in the above example 'podman exec' itself returns with an
exit code of 127 even though both the working directory and the command
that were passed to it were present. Hence, it's necessary to
explicitly check if the requested command was really absent before
attempting the fallbacks.
Fallout from 4536e2c8c2https://github.com/containers/toolbox/pull/872
Due to docker rate limiting we can not rely in docker.io for
retrieving the images.
This was detected when executing our tests for podman fedora
gating pipeline. Our busybox image was not downloaded and
one of the list tests was failing.
Using the current working directory for cache is not a good solution
since the test files may reside in a location that is unwritable (e.g.,
/usr/share). The `BATS_RUN_TMPDIR` variable should point to a location
that is sure to be writeable from the test suite.
https://github.com/containers/toolbox/pull/850
It looks like there are some oddities with Viper [1]. The errors can't
be examined with errors.As [2] and Viper doesn't actually throw
ConfigFileNotFoundError if a configuration file is not found. Secondly,
there's no way to find out if a key was actually specified in a
configuration file. The InConfig API doesn't return 'true' even if a
key was mentioned in a configuration file, and the IsSet API returns
'true' even if the key was only set via SetDefault in the code.
Some changes by Debarshi Ray.
[1] https://pkg.go.dev/github.com/spf13/viper
[2] https://blog.golang.org/go1.13-errorshttps://github.com/containers/toolbox/pull/828https://github.com/containers/toolbox/pull/851
A subsequent commit will add support for configuration files, which can
override the default toolbox image. Since this override affects all
commands, it effectively ends up adding a fourth option to the 'enter'
command, other than the existing options to change the distribution,
release and container. This makes it a lot more difficult to reason
when only 'toolbox enter --release N' is enough to enter the created
container.
https://github.com/containers/toolbox/pull/828https://github.com/containers/toolbox/pull/851
The 'toolbox run' command has one downside: all newlines contain
a carriage return (CR). This is caused by the unconditional use of the
--tty option in `podman exec`[0]. In these particular tests this can be
worked around by not printing a newline at all.
Another quirk around partial is to check the last line of the output.
[0] https://github.com/containers/podman/issues/9718https://github.com/containers/toolbox/pull/843
The output of `podman build` has changed a bit. Each line of log
describing the build is now in the format of:
- STEP i/n: msg
instead of:
- STEP i: msg
where i is the current step and n the maximum number of steps.
The exact format is not important for the purpose of testing Toolbox, so
we may fallback to partial string testing.
Also the latest step ("COMMIT") seems to no longer be considered a step,
so just check for the word.
https://github.com/containers/toolbox/pull/846
Having the entire host file system hierarchy mounted inside a toolbox
container gives the containers a more complete environment that's
resilient against future changes in the layout of the file system
hierarchy and the need for giving access to new paths to support new
use-cases. Otherwise, one would have to create a new container to get
access to any path that lies outside the /boot, /etc, /run, /tmp, /usr
and /var directories.
As a nice side-effect, this also simplifies the bind mount handling
code.
https://github.com/containers/toolbox/pull/827
Turns out the braces do not need to be escaped.
The equivalent code in the POSIX shell implementation was:
echo "$image" | grep "^[a-f0-9]\{6,64\}$"
There the braces had to be escaped because it was using grep(1) with
basic regular expressions (ie., without the --extended-regexp flag),
where the meta-characters ?, +, {, |, ( and ) lose their special
meaning unless they are escaped.
However, that was grep(1), and this is Go's regexp package.
Fallout from dd947016b3https://github.com/containers/toolbox/pull/825
The regexp.MatchString [1] API returns an error only when the regular
expression is faulty, and the boolean return value tells if a match was
found. In this case, the regular expression is baked into the code as a
string literal. So, unless there's a programmer error, it should always
be valid.
Fallout dd947016b3
[1] https://golang.org/pkg/regexp/#MatchStringhttps://github.com/containers/toolbox/pull/825
When installing to a non-system-wide prefix as a non-root user, the
tmpfilesdir path defined by systemd might not be accessible. Overriding
the path helps to prevent the installation from failing.
https://github.com/containers/toolbox/pull/717
This makes 'toolbox', without any commands specified, behave a lot like
'toolbox enter'. When there aren't any toolbox containers, it will
offer to create a new container matching the same parameters passed to
the command. If there's just one toolbox container available, then it
will fall back to it.
This makes the command line interface a lot similar to that of
github.com/coreos/toolbox, which makes things easier for those
switching over from it.
Some changes by Debarshi Ray.
https://github.com/containers/toolbox/pull/811
SELinux is always meant to be disabled. The exact location of the code
is a historical accident and isn't meant to imply that SELinux might
be optionally enabled.
https://github.com/containers/toolbox/pull/814
