Docker Multi-Stage Build

On June 13, 2017 took place the Paris Container Day. At the opening of the conferences they unveiled a new docker feature : multi-stage build. That's the subject of this article.

The multi-stage build requires version 17.06 of Docker-CE (released in June 2017). This feature allowed us to review how we managed the build of our applications and Dockerfiles.

What is Docker ?

First, I'll make a quick reminder, what is Docker?

Docker is a containerization technology that allows the creation and use of Linux containers. Linux containers allow you to package and isolate applications with their complete runtime environment. Containerized applications are easier to move from one environment to another (development, test, production, etc.), while retaining all of their functions. These containerized applications must be transformed into a docker image for easy sharing.

Some applications, to be used, need a build step before being usable (Java, C/C++, Rust...). Before multi-stage build, when we wanted to build an application, we had two possibilities :

1. create an image with everything needed to build the application, build it and deliver it (I will call this fat-container)
2. launch a container that build the application and then place the application in the final image (I will call this builder)

Now I will show you some examples, they are available on github and will be based on a rust application that displays Hello Docker!. It’s a choice I made that can be easily replaced by java.
Sources are here jawg/blog-resources/docker-multi-stage-build.

git clone https://github.com/jawg/blog-resources
cd blog-resources/docker-multi-stage-build/rust-lang

Example of fat-container

Here is the content of fat.dockerfile

FROM rust:1-stretch
# Choose a workdir
WORKDIR /usr/src/app
# Copy sources
COPY . .
# Build app (bin will be in /usr/src/app/target/release/rust-lang-docker-multistage-build)
RUN cargo build --release
# Default command, run app
CMD /usr/src/app/target/release/rust-lang-docker-multistage-build

To build this image you can execute the command:

docker build -t jawg/docker-multi-stage-build:rust-fat-container -f fat.dockerfile .
# To run the binary
docker run -ti --rm jawg/docker-multi-stage-build:rust-fat-container

At the time this article is written, the final image is 1.57GB. What is quite big for an image that we would like to share. Much of the size of this is due to the base image rust 1-stretch which 1.48GB.
It is to avoid large images that we prefer to create images with builders.

Example of builder

To use a builder, you have two choices

1. have the application installed on the computer
2. use a docker containing everything needed
   In the example, I will use a docker. Here is the command to build the rust application, the result will be in target/release/rust-lang-docker-multistage-build

docker run -ti --rm -v $(pwd):/usr/src/app -w /usr/src/app rust:1-stretch cargo build --release

Here is the content of runner.dockerfile, it will take the generated binary and put it in /bin to be in ${PATH}.

FROM debian:stretch-slim
# Copy bin from builder to this new image
COPY target/release/rust-lang-docker-multistage-build /bin/
# Default command, run app
CMD rust-lang-docker-multistage-build

To build this image you can execute the command

docker build -t jawg/docker-multi-stage-build:rust-runner-container -f runner.dockerfile .
# To run the binary
docker run -ti --rm jawg/docker-multi-stage-build:rust-runner-container

Now we have an image that is much smaller than before, it is 60.5MB. It is much better, but you can still find drawbacks to this, it generates files in this directory with root as owner and this requires the execution of two commands. So we will go to multi-stage.

Example of multi-stage-build

For this example, everything will be done in a single Dockerfile multi-stage-build.dockerfile. The process is the same as the builder, we need to make the binary and move it to the final image.
Here is the content of multi-stage-build.dockerfile.

FROM rust:1-stretch as builder
# Choose a workdir
WORKDIR /usr/src/app
# Copy sources
COPY . .
# Build app (bin will be in /usr/src/app/target/release/rust-lang-docker-multistage-build)
RUN cargo build --release

FROM debian:stretch-slim
# Copy bin from builder to this new image
COPY --from=builder /usr/src/app/target/release/rust-lang-docker-multistage-build /bin/
# Default command, run app
CMD rust-lang-docker-multistage-build

To build this image you can execute the command

docker build -t jawg/docker-multi-stage-build:rust-multi-stage-container -f multi-stage-build.dockerfile .
# To run the binary
docker run -ti --rm jawg/docker-multi-stage-build:rust-multi-stage-container

With this version, without any surprises our image is still 60.5MB, normal since the binary does not change, it's the way it was generated that changes. We also have another advantage, our workspace remains clean because everything is included in a container.

You may have noticed that with each build of your images, you had to wait for some libraries to be downloaded. This is a step that takes time and fortunately can be overridden. For this, we can take advantage of the docker cache.

Example of multi-stage-build-cached

When a docker image is built, each step is actually a container that is launched and saved, these are intermediate images.
When you rebuild an image, docker will check if the files you use for its build have changed. If so, it will start again from the step that has been modified.
To take advantage of the cache, here we will pre-register the dependencies. So let's start from the Dockerfile multi-stage-build.dockerfile.

During development, what changes most often are sources, so we must perform the download dependencies before adding sources. To do this we first add the Cargo.toml which contains the dependencies. We can then download the sources (or make an empty build here).
This will only be after we can copy the sources and build our application.

That's what it looks like now

FROM rust:1-stretch as builder
# Choose a workdir
WORKDIR /usr/src/app
# Create blank project
RUN USER=root cargo init
# Copy Cargo.toml to get dependencies
COPY Cargo.toml .
# This is a dummy build to get the dependencies cached
RUN cargo build --release
# Copy sources
COPY src src
# Build app (bin will be in /usr/src/app/target/release/rust-lang-docker-multistage-build)
RUN cargo build --release

FROM debian:stretch-slim
# Copy bin from builder to this new image
COPY --from=builder /usr/src/app/target/release/rust-lang-docker-multistage-build /bin/
# Default command, run app
CMD rust-lang-docker-multistage-build

This technique can very well be transposed to java (with the pom.xml of maven or build.gradle of gradle) or the nodejs (with its package.json).