Commit graph

665 commits

Author SHA1 Message Date
Debarshi Ray
fac565bc33 cmd/root: Rename a variable
https://github.com/containers/toolbox/pull/542
2020-08-26 23:40:15 +02:00
Debarshi Ray
69cec92abb Prepare 0.0.94 2020-08-24 19:31:28 +02:00
Debarshi Ray
80c02344df Silence SC2086
Otherwise https://www.shellcheck.net/ would complain:
  Line 729:
  if $podman_command image exists localhost/$base_toolbox_image
    >/dev/null 2>&3; then
                                            ^-----------------^
    SC2086: Double quote to prevent globbing and word splitting.

See: https://github.com/koalaman/shellcheck/wiki/SC2086

https://github.com/containers/toolbox/pull/537
2020-08-24 19:30:12 +02:00
Debarshi Ray
1dce87c5f1 libc-wrappers: Tighten the definition of the ppc64le definition
This is based on the output of 'gcc -dM -E - </dev/null' on a ppc64le
system. For what it's worth, the _CALL_ELF macro is defined as 1 on
the big endian variants of the architecture.

The original intention in commit 6ad9c63180 was to support the
architectures that Fedora builds for, and it doesn't care about PowerPC
variants that aren't ppc64le [1].

[1] https://fedoraproject.org/wiki/Architectures/PowerPC

https://github.com/containers/toolbox/pull/536
2020-08-24 19:03:24 +02:00
Debarshi Ray
aa1235f4fb README.md: Mention that sudo(8) should work without a password
It turns out that this doesn't work out-of-the-box on openSUSE and
needs to be explicitly configured.

https://github.com/containers/toolbox/issues/452
2020-08-21 17:41:59 +02:00
Debarshi Ray
6ad9c63180 build: Ensure binaries built on Fedora 33 run on Fedoras 32 & 31
The /usr/bin/toolbox binary is not only used to interact with toolbox
containers and images from the host. It's also used as the entry point
of the containers by bind mounting the binary from the host into the
container. This means that the /usr/bin/toolbox binary on the host must
also work inside the container, even if they have different operating
systems.

In the past, this worked perfectly well with the POSIX shell
implementation because it got intepreted by whichever /bin/sh was
available.

The Go implementation also mostly worked so far because it's largely
statically linked, with the notable exception of the standard C
library. However, recently glibc-2.32, which is used by Fedora 33
onwards, added a new version of the pthread_sigmask symbol [1] as part
of the libpthread removal project:
  $ objdump -T /usr/bin/toolbox | grep GLIBC_2.32
  0000000000000000      DO *UND*	0000000000000000  GLIBC_2.32
    pthread_sigmask

This means that /usr/bin/toolbox binaries built against glibc-2.32 on
newer Fedoras pick up the latest version of the symbol and fail to run
against older glibcs in older Fedoras.

One way to fix this is to disable the use of any C code from Go by
using the CGO_ENABLED environment variable [2]. However, this can
negatively impact packages like "os/user" [3] and "net" [4], where the
more featureful glibc APIs will be replaced by more limited
equivalents written only in Go.

Instead, since glibc uses symbol versioning, it's better to tell the
Go toolchain to avoid linking against any symbols from glibc-2.32.

This was accomplished by a few linker tricks:

  * The GNU ld linker's --wrap flag was used when building the Go code
    to divert pthread_sigmask invocations from Go to another function
    called __wrap_pthread_sigmask.

  * A static library was added to provide this __wrap_pthread_sigmask
    function, which forwards calls to the actual pthread_sigmask API in
    glibc. This library itself was not linked with --wrap, and
    specifies the latest permissible version of the pthread_sigmask
    symbol from glibc for each architecture. Currently, the list of
    architectures covers the ones that Fedora builds for.

  * The Go cmd/link linker was switched to external mode [5]. This
    ensures that the final object file containing all the Go code gets
    linked to the standard C library and the wrapper static library by
    the GNU ld linker for the --wrap flag to kick in.

Based on ideas from Ondřej Míchal.

[1] glibc commit c6663fee4340291c
    https://sourceware.org/git/?p=glibc.git;a=commit;h=c6663fee4340291c

[2] https://golang.org/cmd/cgo/

[3] https://golang.org/pkg/os/user/

[4] https://golang.org/pkg/net/

[5] https://golang.org/src/cmd/cgo/doc.go

https://github.com/containers/toolbox/issues/529
2020-08-21 16:29:37 +02:00
Ondřej Míchal
3fee36c885 images: Add fedora-toolbox image definition for Fedora 34
https://github.com/containers/toolbox/pull/532
2020-08-21 16:13:28 +02:00
Juanje Ojeda
464ea7cb28 README: Add missing parameter at the installation instructions
https://github.com/containers/toolbox/pull/516
2020-08-20 04:22:24 -04:00
Ondřej Míchal
ff1fab0f52 cmd/rmi, pkg/podman: Move 'removeImage' func to podman pkg
The 'removeImage' function should go into 'pkg/podman' because it wraps
around Podman's command. Because it no longer has access to the commands
- toolbox rmi - parameters it has a new forceDelete parameter.

https://github.com/containers/toolbox/pull/519
2020-08-14 11:00:06 +02:00
Ondřej Míchal
2ecaaf8ba3 cmd/rm, pkg/podman: Move 'removeContainer' func to podman pkg
The 'removeContainer' function should go into 'pkg/podman' because it
wraps around Podman's command. Because it no longer has access to the
commands - 'toolbox rm' - parameters it has a new forceDelete parameter.

https://github.com/containers/toolbox/pull/519
2020-08-14 11:00:06 +02:00
Harry Míchal
6741ff3a26 cmd/initContainer: Add more information to errors from redirectPath
https://github.com/containers/toolbox/pull/527
2020-08-13 18:36:36 +02:00
Ondřej Míchal
f246d20fa6 build: Install the tests
Installing the tests will let downstream distributors like Fedora run
them as part of their build and CI systems.

https://github.com/containers/toolbox/pull/511
2020-08-13 11:34:37 +02:00
Ondřej Míchal
6eba08ff9e Add CONTRIBUTING.md
Having some contributing guidelines is good!

I wrote of this mostly from top of my head (but I took inspiration from
projects like Podman or Atom). Maybe some parts are not very clear.
2020-08-12 11:57:41 +02:00
Harry Míchal
09896ebfcd Update issue templates
Bug reporters most of the time don't know what and how much info they
should share with us.

I heavily based the bug report template on Podman's[0].

[0] https://github.com/containers/podman/blob/master/.github/ISSUE_TEMPLATE.md

https://github.com/containers/toolbox/pull/459
2020-08-12 11:51:39 +02:00
Debarshi Ray
2befde83d1 cmd/initContainer: Mount a tmpfs at /tmp to match the host
Podman doesn't mount a tmpfs at /tmp by default - it needs to be
separately requested. However, doing it as part of 'podman create ...'
won't add a tmpfs at /tmp for existing toolbox containers. Therefore,
it's best done as part of the entry point.

The mount options are the same as used by systemd (see tmp.mount) to
provide a tmpfs at the host's /tmp.

For what it's worth, the mount flags do differ slightly from the host.

The host has:
$ findmnt --output OPTIONS,PROPAGATION /tmp
OPTIONS                  PROPAGATION
rw,nosuid,nodev,seclabel shared

The container has:
$ findmnt --output OPTIONS,PROPAGATION /tmp
OPTIONS                                        PROPAGATION
rw,nosuid,nodev,seclabel,uid=100000,gid=100000 private

The uid and gid options don't show up on the host because both are 0,
and hence skipped by the tools.

https://github.com/containers/toolbox/issues/513
2020-08-05 16:29:30 +02:00
Debarshi Ray
679680df59 cmd/run: Make it more obvious when falling back to /bin/bash
Users, who prefer shells other than Bash, tend to get confused when
Toolbox presents a Bash prompt to them. It would be better to be more
upfront about what the problem is, so that users can self-support
themselves.

https://github.com/containers/toolbox/issues/18
2020-08-04 14:11:51 +02:00
Debarshi Ray
238a678264 Prepare 0.0.93 2020-07-24 17:44:51 +02:00
Ondřej Míchal
8168e0f58c cmd/list: Handle API change of Podman 2.1 in podman images
Since Podman 2.1 the field 'Created' of `podman images --format json`
no longer holds a string with human-readable string but holds a Unix
time integer[0].

[0] https://github.com/containers/podman/pull/6815

https://github.com/containers/toolbox/pull/503
2020-07-24 17:31:59 +02:00
Ondřej Míchal
e772207831 cmd/list, cmd/run: Decode image/container JSONs to typed structs
Every time Podman changes their JSON API Toolbox breaks horribly. That
is caused by the combination of decoding the JSON purely by hand and by
the complete lack of type assertions to make the process stable.

I previously didn't know that unmarshalling with Go works on the 'do
the best job it can' logic (thank you Owen!). This makes the need to
check for subtle changes in the names of fields go away and type
checking a bit more bearable (marking the questioned field as
interface{} and then immediatelly type switch).

If even now an existing field does not hold an expected value the field
will remain blank.

To take this a bit further, I created two types (toolboxImage and
toolboxContainer) that are used by both functions for returning the
list of images and containers. Instead of using getters for properties
that need some checks or adjustments this uses a custom unmarshaling
function (Owen's idea; thanks!).

The work around populating the two new types is a bit hacky
(marshalling and already unmarshalled JSON - that was unmarshalled to
[]map[string]interface{} - that is then unmarshalled to either
toolboxImage or toolboxContainer. This is done to prevent a major
refactoring before the 0.1.0 release. This should be changed for the
0.2.0 release.

https://github.com/containers/toolbox/pull/503
2020-07-24 17:31:59 +02:00
Ondřej Míchal
4028c3ae3f pkg/utils: Add function creating human-readable duration strings
https://github.com/containers/toolbox/pull/503
2020-07-24 17:31:57 +02:00
Ondřej Míchal
0e5c30d971 build: Add github.com/docker/go-units
This package will be used for generating a human-readable
representation of elapsed time (e.g., "5 minutes").

https://github.com/containers/toolbox/pull/503
2020-07-24 17:31:51 +02:00
Debarshi Ray
06ef424c71 cmd/run: Adjust for changes in 'podman ps' in Podman 2.0
The key containing the container's name (ie., "Names") is no longer a
string, but a slice of strings.

https://github.com/containers/toolbox/pull/509
2020-07-23 15:28:02 +02:00
Ondřej Míchal
2aad8385e1 cmd/create: Update hint after creating a container
Rewritten version of Toolbox supports newer syntax for entering a
container: `toolbox enter <name-of-container>`. This new workflow should
be shown in the hint after creating a container with `toolbox create`.

https://github.com/containers/toolbox/issues/489
2020-07-22 23:56:26 +02:00
Harry Míchal
61a5f5158c cmd/list: Re-enable highlighting of running containers
The Go implementation did not highlight running containers because it
uses a special table writer from 'text/tabwriter' that does not
support colored output [0].

The trick used here is to use ANSI SGR terminal escape sequences [1]
on every line output by the table writer, including headers, and make
the escape sequences use the same number of characters. eg., even
though it's idiomatic to use "\033[0m" to reset the SGR attributes,
such as color; each non-colored line is prefixed with "\033[0;00m" to
ensure that it has the same number of characters as "\033[1;32m", which
is used for bold green output.

Therefore, not only are the non-colored lines wrapped in a redundant
pair of ANSI SGR terminal escape sequences, they are carefully crafted
to match the escape sequences for colored output.

Tested on VTE and xterm.

[0] https://github.com/golang/go/pull/35017
[1] https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_parameters

https://github.com/containers/toolbox/pull/494
2020-07-22 23:29:44 +02:00
Ondřej Míchal
68c585f8b3 test/system: Don't check the output of 'toolbox run'
For some reason running 'toolbox run echo Hello World' in the tests in
the environment of Zuul is causing problems. Bats saves the output of
ran command to $output that can be then checked if it contains the right
output. This variable sometimes holds in this case the right output
("Hello World") but more often it is blank, causing the tests to fail.

I don't know the culprit, so for now I'm commenting out the checks to
make the CI pass again.

https://github.com/containers/toolbox/issues/410
2020-07-22 14:47:42 +02:00
Ondřej Míchal
8c971b381a playbooks, .zuul: Clarify naming and descriptions
Not all tests are the same and the ones we're currently running are
system tests. Also the mention of 'podman-stable' is not that important
because we're using the version in the 'stable' stream of Fedora
releases.

https://github.com/containers/toolbox/pull/508
2020-07-21 17:09:17 +02:00
Harry Míchal
cf1b74e1db .zuul, playbooks/fedora-32: Enable testing for Fedora 32
New release out, we need to run the tests there.

https://github.com/containers/toolbox/pull/507
2020-07-21 16:31:56 +02:00
Harry Míchal
5a651eae2f playbooks, test/system: Test the Go implementation and update tests
In the Go implementation, when the 'rm' and 'rmi' commands fail to
remove a container or image, they don't use a non-zero exit code.
There's currently no nice fix for this. So, the tests have been
adjusted as a temporary measure.

https://github.com/containers/toolbox/pull/507
2020-07-21 16:31:50 +02:00
Harry Míchal
9c5bc56bbd playbooks: Add Go to the list of build dependencies
Otherwise, the tests on Fedora 31 and Rawhide fail with:
  meson.build:8:0: ERROR: Program(s) ['go'] not found or not executable

https://github.com/containers/toolbox/pull/506
2020-07-21 16:04:40 +02:00
Debarshi Ray
bf815d3d73 cmd/create: Restore the spinner, when pulling an image, to stdout
This doesn't seem to have any user-visible effect. However, since the
line was inadvertently removed, it's good to put it back.

Fallout from 950f510872

https://github.com/containers/toolbox/pull/498
2020-07-10 12:36:03 +02:00
Harry Míchal
950f510872 cmd/create: Show the spinner only when connected to a terminal
It doesn't make sense to show a spinner when the output is redirected
to something other than a terminal. When the output is not connected
to a terminal, the terminal escape sequences and other control
characters used to render the spinner leads to some spurious
characters in the output, which confuses the test suite.

This uses the golang.org/x/crypto/ssh/terminal package to check if the
standard output is connected to a terminal or not, which is also what
Podman uses. The other option was the github.com/mattn/go-isatty
package but using it leads to a slightly bigger binary - 7778323 bytes
versus 7782284.

[0] https://github.com/briandowns/spinner

https://github.com/containers/toolbox/pull/496
2020-07-10 11:56:58 +02:00
Harry Míchal
d8f74b538f .zuul: Style fix
https://github.com/containers/toolbox/pull/495
2020-07-09 16:02:53 +02:00
Harry Míchal
6bcd9fd0c4 .zuul, playbooks/fedora-30: Drop testing for Fedora 30
Fedora 30 reached it's End of Life on the 26th of May, 2020 [1]; and
Toolbox requires Go 1.13 which is not available in Fedora 30.

[1] https://fedoraproject.org/wiki/End_of_life

https://github.com/containers/toolbox/pull/495
2020-07-09 16:02:49 +02:00
Debarshi Ray
85ada1e77c README.md: List out dependencies and installation instructions
Based on work done by Natanael.

https://github.com/containers/toolbox/pull/492
2020-07-07 19:36:58 +02:00
Mario Sebastian Chacon
9488c95038 cmd/root: Use the correct verb format for string
https://github.com/containers/toolbox/pull/480
2020-07-07 18:43:58 +02:00
Morten Linderud
0e5592bada build: Ensure reproducible builds
The Go compiler embeds full paths to modules and dependencies into the
binary. This prevents people from reconstructing a bit-for-bit identical
toolbox binary without going through several hoops.

Without patch:

    $ strings build/src/toolbox | grep "$HOME" | wc -l
    105

With patch:

    $ strings build/src/toolbox | grep "$HOME" | wc -l
    0

https://github.com/containers/toolbox/pull/448

Signed-off-by: Morten Linderud <morten@linderud.pw>
2020-07-07 18:11:38 +02:00
Ondřej Míchal
843d4ce2cc pkg/podman: Speed things up by caching the Podman version at runtime
We call `podman version` every time we want to check Podman's version.
This slows Toolbox down considerably. It's more efficient if we call
`podman version` just once and remember the output during runtime and
use the cached value.

https://github.com/containers/toolbox/pull/481
2020-07-03 19:09:39 +02:00
Debarshi Ray
bcacb1196c pkg/podman: Rename a variable
The subsequent commit will introduce a global variable called
podmanVersion, so this will prevent the local variable from colliding
with the global.

https://github.com/containers/toolbox/pull/481
2020-07-03 19:06:09 +02:00
Debarshi Ray
cb5c77eae5 Prepare 0.0.92 2020-07-03 15:54:02 +02:00
Debarshi Ray
b5552d3351 build, pkg/version: Embed the version from Meson into the binary
This removes the need to manually update the version in the Go source
code when making a release.

https://github.com/containers/toolbox/pull/487
2020-07-03 15:28:36 +02:00
Debarshi Ray
ad87a30caf pkg/version: Use a string, not a struct, to for the version information
The subsequent commit will automatically embed the project's version
encoded in Meson into the toolbox binary during the build. This will
remove the need to manually update the version information in the Go
source code. Consolidating the version information reduces the chances
of human error while making a new release. Note, how the 0.0.91 release
forgot to update the version in the Go sources.

This will be done by feeding the version string from Meson into
go-build-wrapper, which will use Go's -X linker flag to embed it into
the final toolbox binary. Since Meson stores the version information
as a string, it's convenient to have the Go code do the same.

https://github.com/containers/toolbox/pull/487
2020-07-03 15:28:36 +02:00
Debarshi Ray
982f10e29b pkg/version: Mark variable as private since there's an accessor for it
Clients of this package should use the GetVersion function to get the
version as a string. The variable that actually stores the version
information is an implementation detail and meant to be private.

https://github.com/containers/toolbox/pull/487
2020-07-03 15:28:28 +02:00
Debarshi Ray
38d6d4702c pkg/utils: Make it build on aarch64
The syscall.Dup2 wrapper isn't defined on aarch64, which breaks the
build as:
  ../../pkg/utils/utils.go:551:12: undefined: syscall.Dup2

https://github.com/containers/toolbox/pull/486
2020-06-30 18:32:49 +02:00
Debarshi Ray
387872c6a3 Prepare 0.0.91 2020-06-27 16:27:02 +02:00
Harry Míchal
dad7e99ea5 cmd/list, cmd/rm: Adjust for changes in 'podman ps' in Podman 2.0
These keys in 'podman ps --format json' were renamed in Podman 2.0:
  * "ID" to "Id"
  * "Created" to "CreatedAt"
  * "Status" to "State"

The key containing the container's name (ie., "Names") is no longer a
string, but a slice of strings.

https://github.com/containers/toolbox/pull/471
2020-06-25 02:16:02 +02:00
Harry Míchal
d0af91502e cmd/list, cmd/rmi: Adjust for changes in 'podman images' in Podman 2.0
The key containing the image's hash in 'podman images --format json'
was renamed to "Id" in Podman 2.0.

https://github.com/containers/toolbox/pull/471
2020-06-25 02:14:57 +02:00
Harry Míchal
5967ab6be5 cmd/list: Use utils.ShortID instead of open-coding the shortening
https://github.com/containers/toolbox/pull/479
2020-06-24 18:01:21 +02:00
Jose Miguel Parrella
89ff98d8c3 cmd/run, sh: Show an error if $PWD is missing inside the container
It might happen that the current working directory in the present
environment is missing inside the container that's going to be used.
So far, Toolbox would fail silently, which wasn't very obvious.

Some changes by Debarshi Ray and Harry Míchal.

https://github.com/containers/toolbox/issues/369
2020-06-23 19:26:59 +02:00
Debarshi Ray
1daa158385 images: Add gvfs-client to Fedoras 31, 32 and 33
The gvfs-client package is necessary for GIO-based processes inside
toolbox containers to use the GVfs backend and volume monitor daemons,
and it comes preinstalled on Fedora Silverblue and Workstation.

Only the images for currently maintained Fedoras (ie., 31, 32 and 33)
were updated.

https://github.com/containers/toolbox/pull/466
2020-06-10 15:40:27 +02:00
Debarshi Ray
dcaf06a950 cmd/create: Don't use gh.c/coreos/go-systemd
Fedora's golang-github-coreos-systemd package is still at version 19,
while the support for Go modules is only available from version 22.
Since all that's necessary is to retrieve the 'Listen' property from
the sssd-kcm.socket unit, it's quite easy to do so directly using Go's
D-Bus bindings.

The systemdNeedsEscape and systemdPathBusEscape functions are copied
from github.com/coreos/go-systemd's needsEscape and PathBusEscape
functions respectively.

The github.com/coreos/go-systemd code is licensed under the ASL 2.0,
the same as Toolbox. It's copyright notice lists 'CoreOS, Inc.', but
since CoreOS has been acquired by Red Hat [1], it should be covered by
Toolbox's existing copyright notices that list 'Red Hat, Inc'.

[1] https://www.redhat.com/en/about/press-releases/red-hat-acquire-coreos-expanding-its-kubernetes-and-containers-leadership

https://github.com/containers/toolbox/pull/444
2020-05-14 23:54:43 +02:00