This is meant to roughly replicate the build environments used by
downstream distributors to build toolbox(1). These can be restricted in
odd ways compared to a fully featured environment where toolbox(1) is
actually going to be used. eg., the inability to use podman(1) in the
case of Fedora or not having subordinate user and group ID ranges in the
case of openSUSE.
It's important to ensure that toolbox(1) can be built by downstream
distributors without any unnecessary hassle.
https://github.com/containers/podman/issues/17657https://github.com/containers/toolbox/issues/1246
The name of a node in a nodeset is meant to be a human-readable name. A
name with an obscure prefix like 'ci-node-' makes it look more profound
than it really is.
https://github.com/containers/toolbox/pull/1206
The 'unit tests' are no longer just unit tests. They also run a bunch
of static analysis tools like ShellCheck, codespell, gofmt and 'go vet'.
Since newer versions of these tools are generally better at catching
problems in the codebase, it will be better to run the 'unit tests' on
Fedora Rawhide with the latest versions than older stable Fedoras.
The timeout for the 'unit tests' need to be increased because Fedora
Rawhide is slower than stable Fedoras. Currently, the timeout for the
'unit tests' running on Fedora 36 is 10 minutes. Increasing it to 20
minutes when running on Fedora Rawhide wasn't enough, so maybe 30 will
be sufficient.
Note that this is only feasible because the Fedora Rawhide builds are
now more robust against stale DNF caches [1]. Otherwise, it wouldn't
have been wise to use Fedora Rawhide to test anything which isn't also
being tested elsewhere, because the Fedora Rawhide builds might have
stayed broken for extended periods of time due to reasons completely
unrelated to Toolbx.
[1] Commit 995c6d175ehttps://github.com/containers/toolbox/pull/1201https://github.com/containers/toolbox/pull/1206
With the recent expansion of the test suite, it's necessary to increase
the timeout for Fedora Rawhide nodes to prevent the CI from timing out.
https://github.com/containers/toolbox/pull/1195
On a couple of occasions the relevant tests didn't get triggered because
some files weren't listed [1], and on another a commit forgot to update
the list of files [2].
The objective of the CI is to reduce stress for the maintainers, and
make it easy for contributors to find out if their changes work or not.
Missing tests don't help with that, and there's no need to optimize the
tests like this unless there's a real problem to be solved.
[1] Commit deca452b27
Commit 5c27d73021
[2] Commit b1743c4927
This reverts commit c28d902089.
https://github.com/containers/toolbox/pull/1168
The -Dmigration_path_for_coreos_toolbox option enables a different code
path that's currently not tested by the CI at all. In fact, since it's
a build-time option, the corresponding code path is not even built by
the CI.
To properly support the -Dmigration_path_for_coreos_toolbox option, it
needs to be covered by the CI. This is a step in that direction by
running the unit tests on it.
https://github.com/containers/toolbox/pull/1095
Defining the YAML anchor as part of the Rawhide tests, instead of the
Fedora 34 test, will prevent it from getting lost by mistake when
Fedora 34 reaches its End of Life.
https://github.com/containers/toolbox/pull/971
Currently, the CI has been frequently timing out when running the unit
tests. It's possible that the current 5 minute timeout isn't enough,
because it's significantly lower than the 20 minute timeout on stable
Fedoras for the system tests.
Increase the timeout to 10 minutes to see if that makes the CI more
stable.
https://github.com/containers/toolbox/pull/970
Currently, the CI has been frequently timing out on Fedora Rawhide
nodes, and it's not clear why that is. One possibility is that this is
due to Rawhide using Linux kernels that are built with debugging
enabled, which makes it slower than released Fedoras. So it might be a
matter of just increasing the timeout.
Currently, the timeout for stable Fedoras is 20 minutes, and that for
Rawhide is 22 minutes. An attempt to increase the Rawhide timeout to 30
minutes didn't succeed, so maybe 45 minutes will be sufficient.
https://github.com/containers/toolbox/pull/964
Not all file are equal when it comes to testing. Unit tests are related
strictly to the source code and documentation changes do not concern it.
System tests have a wider range of influence but documentation and some
other areas also do not concern them.
I'm unsure about the effect of this change on the periodic pipeline
execution.
https://github.com/containers/toolbox/pull/948
We need to know if the latest changes in the libc (that is dynamically
linked to the binary) causes problems in containers based on older
releases of Fedora.
The estimate of the version numbers is very crude and does not follow
the upstream schedule. That should not be a problem, though.
A part of an existing test has been reused and made into a helper
function to implement this.
This increases the run time of the test suite on Rawhide which already
takes longer than the same test suite on released versions of Fedora.
Make up for it by increasing the timeout by 2 minutes.
https://github.com/containers/toolbox/pull/899
There is no significant benefit in keeping this configuration separated.
Now the to-be installed packages are tracked in a single place and the
test playbooks only call the relevant tests.
This was pointed out by in 6063eb27b9https://github.com/containers/toolbox/pull/898
We now have some Go unit tests[0] and we should use them. By adding a
new test case to Meson, the existing CI job called "shellcheck" has no
longer an accurate name. With this it has been renamed to "unit-test".
Also, the job is now more important and therefore should also be used
for gating.
[0] https://github.com/containers/toolbox/pull/474https://github.com/containers/toolbox/pull/730
Not all tests are the same and the ones we're currently running are
system tests. Also the mention of 'podman-stable' is not that important
because we're using the version in the 'stable' stream of Fedora
releases.
https://github.com/containers/toolbox/pull/508
Toolbox requires Go 1.13, while Fedora 30 only has Go 1.12.17.
Therefore the test environment needs to be upgraded to something more
recent.
Otherwise, the test fails with:
note: module requires Go 1.13
The name of the go-md2man package changed in Fedora 31, and hence had
to be updated.
https://github.com/containers/toolbox/pull/442
This adds several .yaml files that specify jobs (those in folder
playbooks) and one that serves as the main config (.zuul.yaml).
Tests and builds are currently executed on every change in PRs (ie.,
check and gating) and periodically (according to the documentation
this pipeline should be run at least once a day).
There are 4 tests in total:
1. 'ninja test' - does the same thing that Travis did
2. Fedora 30 - runs the system tests with current Podman and Toolbox
in Fedora 30
3. Fedora 31 - the same but for Fedora 31
4. Fedora Rawhide - the same but for Fedora Rawhide
https://github.com/containers/toolbox/issues/68
