Cross-compile multi-arch containers

Question

I'm trying to build an ARM (arm32v7) container, but using an x86_64 host. While I know there are some pretty cool things like Resin using Qemu shenanigans, and Multiarch for doing crossbuilding of generic containers, I have a slight issue: The container I'm trying to build starts off as multiarch, and so Docker always chooses the x86 image in the FROM instruction.

I want to build an ARM container from a Multi-arch Rust image on an x86 host. The problem is, I can't find any documentation to explicitly say I want to start with the ARM container and build from that, not the x86 container. Additionally, the tags on the image don't disambiguate, so I can't use those to select the starting container.

I've tried editing the /etc/docker/daemon.json file to contain:

{
    "labels": [ "os=linux", "arch=arm32v7" ],
    "experimental": true
}

but that hasn't helped at all. docker pull still retrieves the x86 images. The purpose of all this is to boost compile times for containers ultimately running on Raspberry Pi; compile times are super slow as it stands.

Are there any ways to explicitly say that I want to build starting with the ARM image?

Answer 1

It is possible to build simple Docker containers for another architecture ("cross-compile") by using an appropriate base image for that architecture. By simple, I mean images that don't need a RUN command in their Dockerfile to be built. This is because Docker doesn't have the ability to actually run commands in a container for another architecture. While this sounds restrictive, it can be quite powerful when combined with multi-stage builds to cross-compile code.

Let's walk through this step-by-step. First off, let's enable experimental mode for our Docker client to enable docker manifest by adding the following option to ~/.docker/config.json:

{
    "experimental": "enabled"
}

We can then use docker manifest inspect debian:stretch to show the fat manifest that contains a digest for the image in the architecture we want to build for. For example, the arm32v7 image has "architecture": "arm" and "variant": "v7" specified under the platform key. Using jq, we can extract the digest for this image programatically:

docker manifest inspect debian:stretch | jq -r '.manifests[] | select(.platform.architecture == "arm" and .platform.variant == "v7") | .digest'`

This digest can then be used in the FROM command in a Dockerfile:

FROM debian@sha256:d01d682bdbacb520a434490018bfd86d76521c740af8d8dbd02397c3415759b1

It is then possible to COPY cross-compiled binary into the image. This binary could come from a cross-compiler on your machine or from another container in a multi-stage build. To get rid of the hard-coded digest in the Dockerfile's FROM line, it's possible to externalise it through a Docker build argument (ARG).