Building foreign-arch images
The CKI pipeline is powered by the internal PSI OpenShift cluster running on
x86_64 machines. For compiling kernels on
x86_64, the native compiler
toolchain is used. For all other architectures, the kernel is cross-compiled.
While this setup is supported by the RHEL cross-compiler team for the kernel
itself, this is not the case for the kernel tools.
Up to now, this was not a problem as the CKI pipeline did not build those RPMs. As that is going to change in the near future, we would like to build native versions of our container images to bootstrap the effort.
Installing the necessary pieces in the builder image
We use a Fedora-based builder container with
to build other images. To make that work for non-native architectures,
surprisingly few steps are needed.
One bit of the puzzle is qemu-user-static. It allows to transparently use
qemu for foreign architectures. For a Fedora container, the only package that
needs to be added to the container image is
RUN dnf install qemu-user-static
The other bit is the host setup. The documentation for qemu-user-static
recommends running the
multiarch/qemu-user-static setup container to prepare
the host. Internally, the container uses the upstream qemu-binfmt-conf.sh to
binfmt interpreter. As we don’t want to run a container during
our builds, we add the script directly to our builder container image via
ADD https://raw.githubusercontent.com/qemu/qemu/master/scripts/qemu-binfmt-conf.sh \ /usr/local/bin/qemu-binfmt-conf.sh RUN chmod +x /usr/local/bin/qemu-binfmt-conf.sh
Building via buildah
Building foreign-arch container images with
buildah is actually pretty easy
once you know how to do it. The DevConf.CZ 2020 presentation on building
multi-arch container images with buildah by Nalin Dahyabhai is the best
introduction that I could find.
As we want to build our image in a priviledged container managed by
gitlab-runner, the qemu-binfmt-conf.sh script needs to be invoked as first
part of the build process before starting the actual build via
qemu-binfmt-conf.sh --qemu-suffix -static --qemu-path /usr/bin --persistent yes
--arch arm64 (for selecting the architecture of the resulting
--override-arch arm64 (for selecting the architecture of the image
FROM) to buildah is enough to build
arm64 images instead of native ones.
The code in this post can be found in the container images repository.