Commit graph

517 commits

Author SHA1 Message Date
Debarshi Ray
561fe8b12d Lay the foundations for logging Podman in Go
https://github.com/containers/toolbox/pull/318
2020-05-13 09:40:41 +02:00
Debarshi Ray
c9ae8ad3b6 pkg/utils: Add function to tunnel a Toolbox invocation to the host
https://github.com/containers/toolbox/pull/318
2020-05-13 09:40:41 +02:00
Harry Míchal
ebad3af632 pkg/utils: Add function to get the options to propagate the environment
https://github.com/containers/toolbox/pull/318
2020-05-13 09:40:40 +02:00
Debarshi Ray
03ca79a7b0 pkg/utils: Add function to check if running inside a toolbox container
https://github.com/containers/toolbox/pull/318
2020-05-13 09:40:40 +02:00
Harry Míchal
8f03a902f2 pkg/utils: Add function to check if running inside a container
https://github.com/containers/toolbox/pull/318
2020-05-13 09:40:40 +02:00
Harry Míchal
bd5b8065ae pkg/utils: Add function to conveniently check if a path exists
https://github.com/containers/toolbox/pull/318
2020-05-13 09:40:40 +02:00
Debarshi Ray
7ab4ca868c Add a utility function for showing manuals in Go
A subsequent commit will use this to implement the help command.

Just like the existing POSIX shell implementation, the standard error
needs to be redirected to standard output. For details see commit
5e63e9ec9b.

https://github.com/containers/toolbox/pull/318
2020-05-13 09:40:00 +02:00
Debarshi Ray
19081b5d4a Add convenience wrappers to shell out to any binary in Go
https://github.com/containers/toolbox/pull/318
2020-05-13 09:39:35 +02:00
Harry Míchal
53f4d0c2f0 Add versioning infrastructure in Go
https://github.com/containers/toolbox/pull/318
2020-05-13 09:38:58 +02:00
Debarshi Ray
1b6d7d6410 build: Hook up the Go code with Meson
Meson doesn't support Go [1], so this is implemented by a custom target
that invokes 'go build' to generate the binary.

Unfortunately, when using Go modules, 'go build' insists on being
invoked in the same source directory where the go.mod file lives,
while Meson insists on using a build directory separate from the
corresponding source directory. This is addressed by using a build
script that goes into the source directory and then invokes 'go build'.

Currently, the Go code is only built when a Go implementation is found,
and even then, it's not installed. Non-technical end-users are supposed
to continue using the POSIX shell implementation until the Go version
is blessed as stable.

[1] https://github.com/mesonbuild/meson/issues/123

https://github.com/containers/toolbox/pull/318
2020-05-13 09:38:52 +02:00
Harry Míchal
d857471aa2 Add a skeleton for the Go rewrite
To build the Go, enter the src sub-directory and use 'go build':
  $ cd src
  $ go build

https://github.com/containers/toolbox/pull/318
2020-05-12 16:58:03 +02:00
Debarshi Ray
14db6622dd Update copyright notices 2020-05-12 16:56:52 +02:00
Debarshi Ray
ecc3ce029b Update URL in /etc/krb5.conf.d/kcm_default_ccache 2020-05-12 16:24:42 +02:00
Jens Petersen
b4337d4f86 images/fedora/f33: Don't install docs for packages that aren't present
This fixes the following build failure:
  atomic_reactor.util - Package chkconfig available, but not installed.
  atomic_reactor.util - No match for argument: chkconfig
  atomic_reactor.util - Package dbus-daemon available, but not
    installed.
  atomic_reactor.util - No match for argument: dbus-daemon
  atomic_reactor.util - Package rpm-plugin-systemd-inhibit available,
    but not installed.
  atomic_reactor.util - No match for argument:
    rpm-plugin-systemd-inhibit
  ...
  ...
  ...
  atomic_reactor.util - ERROR - {'errorDetail': {'code': 143,
    'message': "The command '/bin/sh -c dnf -y reinstall
    $(<missing-docs)' returned a non-zero code: 143"}, 'error': "The
    command '/bin/sh -c dnf -y reinstall $(<missing-docs)' returned a
    non-zero code: 143"}
2020-04-03 19:41:00 +02:00
Debarshi Ray
17e384b7fb images: Add fedora-toolbox image definition for Fedora 33 2020-04-03 19:29:44 +02:00
Harry Míchal
2d18f295a7 test/system: Update pre-pulled image for Rawhide
Current Rawhide is actually version 33. So the appropriate image should
be pre-pulled.

Because of the old version of image being pulled, the tests were
failing.
2020-03-13 17:16:12 +01:00
Harry Míchal
1e2232762c test/system: Rework the tests
The tests introduced by commit b5cdc57ae3 have proven to be
rather unstable due to mistakes in their design. The tests were quite
chaotically structured, and because of that images were deleted and
pulled too often, causing several false positives [1, 2].

This changes the structure of the tests in a major way. The tests
(resp. commands) are now run in a manner that better simulates the way
Toolbox is actually used. From a clean state, through creating
containers, using them and in the end deleting them. This should
reduce the strain on the bandwidth and possibly even speed up the
tests themselves.

[1] https://github.com/containers/toolbox/pull/372
[2] https://github.com/containers/toolbox/pull/374

https://github.com/containers/toolbox/pull/375
2020-02-18 14:00:59 +01:00
Tristan Cacqueray
50683c9d9a playbooks: Reduce flakiness due to network errors when pulling images
This change adds a pre-run task to pull the fedora-toolbox images from
the registry to reduce the number of false positives caused by
'podman pull' failing to download them during the actual test.

Each section needs a separate playbook because they use different
versions of Fedora, and hence different default images.

https://github.com/containers/toolbox/pull/375
2020-02-18 14:00:01 +01:00
Harry Míchal
5cea6c60eb test/system/README.md: Tweak
https://github.com/containers/toolbox/pull/377
2020-02-18 13:18:51 +01:00
TomSweeneyRedHat
518b8f55d1 Add Code of Conduct
https://github.com/containers/toolbox/pull/374

Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
2020-02-12 17:12:23 +01:00
Harry Míchal
c86a715d3c Remove Travis
Travis was running 'ninja test' and that's now covered by Zuul.

https://github.com/containers/toolbox/issues/68
2020-01-22 16:11:53 +01:00
Tristan Cacqueray
a28177a7ab Enable Zuul
This adds several .yaml files that specify jobs (those in folder
playbooks) and one that serves as the main config (.zuul.yaml).

Tests and builds are currently executed on every change in PRs (ie.,
check and gating) and periodically (according to the documentation
this pipeline should be run at least once a day).

There are 4 tests in total:

1. 'ninja test' - does the same thing that Travis did
2. Fedora 30 - runs the system tests with current Podman and Toolbox
   in Fedora 30
3. Fedora 31 - the same but for Fedora 31
4. Fedora Rawhide - the same but for Fedora Rawhide

https://github.com/containers/toolbox/issues/68
2020-01-22 16:11:53 +01:00
Harry Míchal
da4fea271f Add a README file for system tests
https://github.com/containers/toolbox/issues/68
2020-01-22 15:54:31 +01:00
Harry Míchal
b5cdc57ae3 Add system test scripts
These tests are written using BATS (Bash Automated Testing System). I
used a very helpful helpers.bash script from the libpod project (Thank
you!) that I tweaked slightly.

https://github.com/containers/toolbox/issues/68
2020-01-22 15:54:31 +01:00
Debarshi Ray
7b460e390d Prepare 0.0.18 2020-01-14 15:47:45 +01:00
Harry Míchal
8f4070224f Check /usr/share/profile.d when bind mounting toolbox.sh
/usr/share/profile.d is the default location where toolbox.sh is
installed, even though, in practice, most (all?) distributions use
/etc/profile.d. It's reasonable to at least make the code work with the
default build values.

https://github.com/containers/toolbox/pull/362
2020-01-14 15:19:27 +01:00
Debarshi Ray
cb79382967 Unbreak 'enter' when SELinux is disabled
/sys/fs/selinux is only present when SELinux is 'enforcing' or
'permissive'. When it's disabled, /sys/fs/selinux doesn't exist and
sysfs doesn't let you create it either. Therefore, the attempt to wipe
out the toolbox container's /sys/fs/selinux by bind mounting
/usr/share/empty over it fails, and in turn prevents the container from
starting up.

Fallout from f9cca5719d

https://github.com/containers/toolbox/issues/344
2020-01-10 18:22:47 +01:00
Debarshi Ray
47c32712f4 Set up /mnt to match the host
On Silverblue /mnt is a symbolic link to /var/mnt. Matching what the
host does will reduce weird side-effects.

https://github.com/containers/toolbox/issues/92
2020-01-06 19:16:17 +01:00
Debarshi Ray
3de605aec6 Give access to /var/mnt from the host operating system
On Silverblue, /mnt is a symbolic link to /var/mnt. Matching the
presence of /var/mnt on the host inside the toolbox container would
make things less confusing for users.

https://github.com/containers/toolbox/issues/92
2019-11-22 19:05:35 +01:00
Debarshi Ray
5595cc065f Do recursive bind mounts when binding locations at runtime
A subsequent commit will give access to /var/mnt from the host, if its
present, by bind mounting /run/host/var/mnt at runtime. However, it
turns out that an attempt to non-recursively bind it will error out, if
the host's /var/mnt already contains a mount point.

On the host:
$ sudo mkdir --parents /var/mnt/tmp
$ sudo mount -t tmpfs none /var/mnt/tmp

Inside the container:
$ sudo mkdir --parents /var/mnt
$ sudo mount --bind -o rslave /run/host/var/mnt /var/mnt
mount: /var/mnt: wrong fs type, bad option, bad superblock on
  /run/host/var/mnt, missing codepage or helper, or other error.

https://github.com/containers/toolbox/issues/92
2019-11-22 19:05:31 +01:00
Debarshi Ray
65e3eec06e Set up /media to match the host
On Silverblue /media is a symbolic link to /run/media. Matching what
the host does will reduce weird side-effects.

https://github.com/containers/toolbox/issues/330
2019-11-22 16:31:50 +01:00
Debarshi Ray
8bf970776b Check if /media is available
No /media on openSUSE.

https://github.com/containers/toolbox/issues/230
2019-11-21 17:31:03 +01:00
Debarshi Ray
7f5ac939e8 Prepare 0.0.17 2019-11-20 18:08:03 +01:00
Debarshi Ray
f9cca5719d Wipe out the container's /sys/fs/selinux to not advertise SELinux
This is the second time a Podman regression has caused a selinuxfs
instance to leak into the toolbox container's /sys/fs/selinux,
tricking various components into trying to use SELinux. It might be
better to work this around in Toolbox until the situation in Podman is
figured out.

Based on an idea from Colin Walters.

https://github.com/containers/libpod/issues/4452
2019-11-20 16:35:54 +01:00
Debarshi Ray
9dc5281430 Deprecate all toolbox containers that don't use a reflexive entry point
Toolbox containers created prior to commit 8b84b5e460 didn't use
'toolbox init-container' as their entry points. This prevents them
from being configured at runtime through the entry points.

Being able to configure a toolbox container at runtime through the
entry point is very handy, as compared to doing it statically via
'podman create', because the configuration doesn't get permanently
baked into the container's definition. Instead, it's codified in
toolbox(1), which can be updated over time, and the container
reconfigured everytime it's started.

A deprecation notice is the precursor to actually dropping support for
these old containers in the future.

Preliminary testing suggests that toolbox containers created prior to
commit 8b84b5e460 already don't start on cgroups v2 systems. So,
this is mainly targetted at cgroups v1 users, who are still able to
work with those old containers.

https://github.com/containers/toolbox/pull/336
2019-11-20 16:07:13 +01:00
Akira TAGOH
2d6c59157c Ensure that 'run' has at least one argument for the command
Otherwise, it would lead to:
  $ toolbox run
  /usr/bin/toolbox: line 1287: shift: 4: shift count out of range
  toolbox: command '' not found in container fedora-toolbox-31

Fallout from 2da4cc4634

https://github.com/containers/toolbox/pull/332
2019-11-19 14:59:49 +01:00
Harry Míchal
1625ad319f Add a --very-verbose or -vv option
Currently, toolbox(1) offers a --verbose option that only shows debug
information from toolbox(1) itself and the error stream of internal
commands. There's no way to further increase the log level of the
internal commands. It's sometimes very useful to be able to get more
detailed logs from Podman.

This adds a new --very-verbose or -vv option that makes this possible.

This should have been implemented as '--verbose --verbose', which
could be conveniently shortened to '-vv'. This is what flatpak(1)
does. However, due to the lack of built-in command line parsing
facilities in POSIX shell, there's no support for multiple short
options expressed as one single argument. eg., '-vy' doesn't expand to
'-v -y'.

Therefore, a separate --very-verbose or -vv option was added to make
things convenient for the user. It's expected that most people will
refer to this as -vv.

If this option is used, every Podman command in the code is run with
'--log-level debug'. Use wisely, Podman can be 'very verbose'.

https://github.com/containers/toolbox/pull/289
2019-11-19 13:38:41 +01:00
Debarshi Ray
1dca2bea09 Give access to the syslog and systemd journal sockets from the host
This makes the following work from inside a toolbox container:
$ logger "syslog: hello world"
$ python3 <<< "from systemd import journal; \
      journal.send('journal: hello world')"

https://github.com/containers/toolbox/pull/327
2019-11-07 16:24:29 +01:00
Debarshi Ray
ee82b94da4 Give access to the user's systemd journal entries from the host
It's now possible to use journalctl(1) to query the user's systemd
journal entries from the host. However, messages from other users and
the system aren't shown.

https://github.com/containers/toolbox/pull/327
2019-11-07 16:24:25 +01:00
Debarshi Ray
c0879a1691 Give access to /etc/machine-id from the host operating system
The machine ID is necessary to query the host operating system's
systemd journal, and currently toolbox containers have an empty
/etc/machine-id file.

Unlike /etc/resolv.conf, the machine ID is supposed to stay constant
once the host is booted. Therefore, it is safe to bind mount
/etc/machine-id from the host, as opposed to using a symbolic link;
because there's no chance of the file getting atomically updated on
the host and diverging from the bind mount due to being allocated a
new inode. Incidentally, this is also what Flatpak does.

A subsequent commit will use this to enable accessing the host's
systemd journal via journalctl(1) inside toolbox containers.

https://github.com/containers/toolbox/pull/327
2019-11-07 16:24:21 +01:00
Debarshi Ray
929e71b00f Bind mount the system libvirt instance at runtime
For what it's worth, this does alter the mount propagation flags by
adding 'slave'.

Earlier with 'podman create --volume ...' it was:
$ findmnt -o OPTIONS,PROPAGATION /run/libvirt
OPTIONS                           PROPAGATION
rw,nosuid,nodev,seclabel,mode=755 private

Now with 'mount --bind ...' it is:
$ findmnt -o OPTIONS,PROPAGATION /run/libvirt
OPTIONS                           PROPAGATION
ro,relatime,seclabel private,slave

This difference was ignored because it doesn't appear to cause any
real problem.

https://github.com/containers/toolbox/pull/327
2019-11-07 16:24:17 +01:00
Debarshi Ray
9436bbece0 Bind mount the system Flatpak directory at runtime
For what it's worth, this does alter the mount propagation flags by
adding 'slave'.

Earlier with 'podman create --volume ...' it was:
$ findmnt -o OPTIONS,PROPAGATION /var/lib/flatpak
OPTIONS              PROPAGATION
ro,relatime,seclabel private

Now with 'mount --bind -o ro ...' it is:
$ findmnt -o OPTIONS,PROPAGATION /var/lib/flatpak
OPTIONS              PROPAGATION
ro,relatime,seclabel private,slave

This difference was ignored because it doesn't appear to cause any
real problem.

https://github.com/containers/toolbox/pull/327
2019-11-07 16:24:13 +01:00
Debarshi Ray
819bb46aaa Add a helper function to bind mount locations at runtime
Subsequent commits will use this to perform some of the bind mounts in
the toolbox container's entry point, instead of doing them as part of
'podman create ...'.

Anything that's specified during 'podman create ...' gets statically
baked into the container's configuration, and is either difficult or
impossible to change afterwards. This means that toolbox containers
created with older versions of Toolbox keep diverging from those
created with newer versions. Hence making it complicated to keep older
containers working with a newer Toolbox.

In the case of bind mounts, a good chunk of the host's file hierarchy
is already bind mounted by 'podman create ...' under the toolbox
container's /run/host. Therefore, the more granular bind mounts like
$XDG_RUNTIME_DIR and /var/lib/flatpak can be performed by the
container's entry point at runtime using what's already inside
/run/host, and reduce the footprint of the static configuration.

Older containers created with Toolbox 0.0.10 onwards will see two bind
mounts for locations that get moved from 'podman create ...' to the
entry point. The presence of the second mount should be harmless.

Based on an idea from Colin Walters.

https://github.com/containers/toolbox/pull/327
2019-11-07 16:24:06 +01:00
Dusty Mabe
ebb88a76a0 README: add "Goals and Use Cases" section 2019-10-29 17:12:02 +01:00
Debarshi Ray
2a2867789d Prepare 0.0.16 2019-10-29 16:09:44 +01:00
Debarshi Ray
2e7ba83be2 Try to migrate to a supported OCI runtime if 'podman start' suggests so
Toolbox containers using runc as their runtime don't work on host
operating systems using cgroups v2. They need to be migrated to crun.
'podman start' throws a specific error for such containers:
  ERRO[0000]: oci runtime "runc" does not support CGroups V2: use
    system migrate to mitigate
  Error: unable to start container "fedora-toolbox-30": this version
    of runc doesn't work on cgroups v2: OCI runtime error

This error is identified by the phrase "use system migrate to mitigate"
to avoid encoding any assumptions about updating from cgroups v1 to v2
or downgrading in the other direction.

If the migration fails, 'toolbox reset' is suggested as the last hope.

https://github.com/containers/toolbox/pull/309
2019-10-29 14:12:41 +01:00
Debarshi Ray
3496029ed7 Split out the code that calls 'podman start'
A subsequent commit will leverage this to detect 'podman start'
failures caused by attempting to run runc-based toolbox containers on
cgroups v2 sytems, and try to migrate them if possible.

https://github.com/containers/toolbox/pull/309
2019-10-29 13:45:32 +01:00
Debarshi Ray
359fae59be Tweak the debug output
https://github.com/containers/toolbox/pull/309
2019-10-29 13:44:45 +01:00
Debarshi Ray
c7d2eb7a99 Log the Podman version into the debug output
Asking for the Podman version is one of the most common support
questions. So it can't hurt to have it in the debug output, especially
when the version is already being read to decide if migration is
necessary or not.

https://github.com/containers/toolbox/pull/309
2019-10-29 13:40:07 +01:00
Debarshi Ray
2142cdd612 Log the cgroups version into the debug output
The migration to cgroups v2 in Fedora 31 [1] has proved a bit stormy.
This is meant to help users self-diagnose whether their problems might
be originating from the use of cgroups v2.

[1] https://fedoraproject.org/wiki/Changes/CGroupsV2

https://github.com/containers/toolbox/pull/309
2019-10-29 13:34:33 +01:00