build: Ensure binaries built on Fedora 33 run on Fedoras 32 & 31

The /usr/bin/toolbox binary is not only used to interact with toolbox
containers and images from the host. It's also used as the entry point
of the containers by bind mounting the binary from the host into the
container. This means that the /usr/bin/toolbox binary on the host must
also work inside the container, even if they have different operating
systems.

In the past, this worked perfectly well with the POSIX shell
implementation because it got intepreted by whichever /bin/sh was
available.

The Go implementation also mostly worked so far because it's largely
statically linked, with the notable exception of the standard C
library. However, recently glibc-2.32, which is used by Fedora 33
onwards, added a new version of the pthread_sigmask symbol [1] as part
of the libpthread removal project:
  $ objdump -T /usr/bin/toolbox | grep GLIBC_2.32
  0000000000000000      DO *UND*	0000000000000000  GLIBC_2.32
    pthread_sigmask

This means that /usr/bin/toolbox binaries built against glibc-2.32 on
newer Fedoras pick up the latest version of the symbol and fail to run
against older glibcs in older Fedoras.

One way to fix this is to disable the use of any C code from Go by
using the CGO_ENABLED environment variable [2]. However, this can
negatively impact packages like "os/user" [3] and "net" [4], where the
more featureful glibc APIs will be replaced by more limited
equivalents written only in Go.

Instead, since glibc uses symbol versioning, it's better to tell the
Go toolchain to avoid linking against any symbols from glibc-2.32.

This was accomplished by a few linker tricks:

  * The GNU ld linker's --wrap flag was used when building the Go code
    to divert pthread_sigmask invocations from Go to another function
    called __wrap_pthread_sigmask.

  * A static library was added to provide this __wrap_pthread_sigmask
    function, which forwards calls to the actual pthread_sigmask API in
    glibc. This library itself was not linked with --wrap, and
    specifies the latest permissible version of the pthread_sigmask
    symbol from glibc for each architecture. Currently, the list of
    architectures covers the ones that Fedora builds for.

  * The Go cmd/link linker was switched to external mode [5]. This
    ensures that the final object file containing all the Go code gets
    linked to the standard C library and the wrapper static library by
    the GNU ld linker for the --wrap flag to kick in.

Based on ideas from Ondřej Míchal.

[1] glibc commit c6663fee4340291c
    https://sourceware.org/git/?p=glibc.git;a=commit;h=c6663fee4340291c

[2] https://golang.org/cmd/cgo/

[3] https://golang.org/pkg/os/user/

[4] https://golang.org/pkg/net/

[5] https://golang.org/src/cmd/cgo/doc.go

https://github.com/containers/toolbox/issues/529

meson.build

@@ -1,10 +1,15 @@
 project(
   'toolbox',
+  'c',
   version: '0.0.93',
   license: 'ASL 2.0',
   meson_version: '>= 0.40.0',
 )
 
+cc = meson.get_compiler('c')
+add_project_arguments('-pthread', language: 'c')
+add_project_link_arguments('-pthread', language: 'c')
+
 go = find_program('go')
 go_md2man = find_program('go-md2man')
 shellcheck = find_program('shellcheck', required: false)

src/go-build-wrapper

@@ -16,9 +16,9 @@
 #
 
 
-if [ "$#" -ne 3 ]; then
+if [ "$#" -ne 4 ]; then
     echo "go-build-wrapper: wrong arguments" >&2
-    echo "Usage: go-build-wrapper [SOURCE DIR] [OUTPUT DIR] [VERSION]" >&2
+    echo "Usage: go-build-wrapper [SOURCE DIR] [OUTPUT DIR] [VERSION] [libc-wrappers.a]" >&2
     exit 1
 fi
 
@@ -27,5 +27,5 @@ if ! cd "$1"; then
     exit 1
 fi
 
-go build -trimpath -ldflags "-X github.com/containers/toolbox/pkg/version.currentVersion=$3" -o "$2"
+go build -trimpath -ldflags "-extldflags '-Wl,--wrap,pthread_sigmask $4' -linkmode external -X github.com/containers/toolbox/pkg/version.currentVersion=$3" -o "$2"
 exit "$?"

src/libc-wrappers/libc-wrappers.c

@@ -0,0 +1,42 @@
+/*
+ * Copyright © 2020 Red Hat Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+
+#include <signal.h>
+
+
+#if defined __aarch64__
+__asm__(".symver pthread_sigmask,pthread_sigmask@GLIBC_2.17");
+#elif defined __arm__
+__asm__(".symver pthread_sigmask,pthread_sigmask@GLIBC_2.4");
+#elif defined __i386__
+__asm__(".symver pthread_sigmask,pthread_sigmask@GLIBC_2.0");
+#elif defined __powerpc64__
+__asm__(".symver pthread_sigmask,pthread_sigmask@GLIBC_2.17");
+#elif defined __s390x__
+__asm__(".symver pthread_sigmask,pthread_sigmask@GLIBC_2.2");
+#elif defined __x86_64__
+__asm__(".symver pthread_sigmask,pthread_sigmask@GLIBC_2.2.5");
+#else
+#error "Please specify symbol version for pthread_sigmask"
+#endif
+
+
+int
+__wrap_pthread_sigmask (int how, const sigset_t *set, sigset_t *oldset)
+{
+  return pthread_sigmask (how, set, oldset);
+}

src/libc-wrappers/meson.build

@@ -0,0 +1,8 @@
+sources = files(
+  'libc-wrappers.c',
+)
+
+libc_wrappers = static_library(
+  'c-wrappers',
+  sources,
+)

src/meson.build

@@ -1,3 +1,5 @@
+subdir('libc-wrappers')
+
 go_build_wrapper_file = files('go-build-wrapper')
 go_build_wrapper_program = find_program('go-build-wrapper')
 
@@ -27,7 +29,9 @@ custom_target(
     meson.current_source_dir(),
     meson.current_build_dir(),
     meson.project_version(),
+    libc_wrappers.full_path(),
   ],
+  depends: libc_wrappers,
   input: sources,
   install: true,
   install_dir: get_option('bindir'),