Keep your buildsystem clean: builds inside docker containers

As Build-Engineer you might know those days when dev-team „A“ comes to you with „Hey, let’s try this new fance version of SBT“ while dev-team B says „No, we really need this ant in exactly this version with that configuration and plugins“ and dev-team C mentions „Well, we have this build script in Python that was developed years ago and nobody knows really how it works“. So you start with poluting your Jenkins-instance with a lot of more or less well known build tools. Time comes and you are faced with possibly outdated tools or with tools/scripts you don’t know enough to efficiently help these guys with problems, so you need to let devs configure the build tools on your build machine. Normally this doesn’t lead to better maintainability of your sysytem…

Here comes the idea of isolated „toolchain“ or „build“-containers that can easily be started by your build machine. This actually is not my idea. I heard about this concept from Lothar Schulz at Continuous Lifecycle in Mannheim this year, but I made up my implementation of that idea 🙂

The basic thinking behind is abstraction of artifact-build from the job-run itself. The artifact-build (generation of jar, war, ear, zip, tgz, pex, …) will be encapsulated inside a „throw-away“ docker container so Jenkins (or other build machines) has not to be aware about what goes on inside the job-run.

First of all the requirements:

  • Jenkins 2.33 (not really required as the solution does not depende on a specific Jenkins version)
  • Docker 1.12.3
  • Oracle Java (used for the Maven build itself)
  • Maven

For this example I created two Docker images. One with java-1.8/maven-3.3.9 and one with java-1.8/maven-2.2.1:

FROM debian:jessie
MAINTAINER Christoph Burmeister

ENV JAVA_HOME="/jdk1.8.0_112"
ENV MVN_HOME="/apache-maven-3.3.9
ENV PATH="${PATH}:${JAVA_HOME}/bin:${MVN_HOME}/bin"

# update and upgrade
RUN apt-get update && apt-get -y upgrade
 
# install several tools via apt
RUN apt-get -y install apt-utils wget

# get and install jdk
RUN wget --quiet --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz \
&& tar -xzf /jdk-8u112-linux-x64.tar.gz \
&& rm -f /jdk-8u112-linux-x64.tar.gz

# get and install maven
RUN wget --quiet http://mirror.dkd.de/apache/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz \
&& tar -xzf /apache-maven-3.3.9-bin.tar.gz \
&& rm -f /apache-maven-3.3.9-bin.tar.gz

 
CMD ["mvn", "-f", "/workspace/pom.xml", "clean", "package"] 
FROM debian:jessie
MAINTAINER Christoph Burmeister

ENV JAVA_HOME="/jdk1.8.0_112"
ENV MVN_HOME="/apache-maven-2.2.1
ENV PATH="${PATH}:${JAVA_HOME}/bin:${MVN_HOME}/bin"

# update and upgrade
RUN apt-get update && apt-get -y upgrade
 
# install several tools via apt
RUN apt-get -y install apt-utils wget

# get and install jdk
RUN wget --quiet --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.tar.gz \
&& tar -xzf /jdk-8u112-linux-x64.tar.gz \
&& rm -f /jdk-8u112-linux-x64.tar.gz

# get and install maven
RUN wget --quiet http://archive.apache.org/dist/maven/binaries/apache-maven-2.2.1-bin.tar.gz \
&& tar -xzf /apache-maven-2.2.1-bin.tar.gz \
&& rm -f /apache-maven-2.2.1-bin.tar.gz

 
CMD ["mvn", "-f", "/workspace/pom.xml", "clean", "package"] 

Both Dockerfiles (and so the resulting images) are very similar and you could of course exclude parts to a new base image, but for now, let’s assume that here we have two completely different build tools.

christoph@athena:~/toolcontainer-example$ ll
total 143288
drwxrwxr-x  3 christoph christoph      4096 Nov 26 19:57 ./
drwxr-xr-x 24 christoph christoph      4096 Nov 26 16:59 ../
-rw-rw-r--  1 christoph christoph       870 Nov 26 19:57 Dockerfile_builder_java18_mvn221
-rw-rw-r--  1 christoph christoph       881 Nov 26 19:41 Dockerfile_builder_java18_mvn339
christoph@athena:~/toolcontainer-example$ 
christoph@athena:~/toolcontainer-example$ 
christoph@athena:~/toolcontainer-example$ sudo docker build \
    -f Dockerfile_builder_java18_mvn339 \
    -t builder_java18_mvn339 \
    .
[..]
christoph@athena:~/toolcontainer-example$ sudo docker build \
    -f Dockerfile_builder_java18_mvn221 \
    -t builder_java18_mvn221 \
    .
[..]
christoph@athena:~/toolcontainer-example$ sudo docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
builder_java18_mvn221    latest              c711cd0c4f03        12 minutes ago      550.8 MB
builder_java18_mvn339    latest              d86da0fc85fd        15 minutes ago      557.6 MB

For playing around with these images you can run them in interactive mode to become root in the container:

christoph@athena:~/toolcontainer-example$ sudo docker run \
    -it \
    -v /data/jenkins/jobs/example-java-maven/workspace:/workspace builder_java18_mvn339 \
    /bin/bash

There you can see three directories under „/“ that have been added by docker build: a jdk-installation, a maven-installation and a workspace directory where the the working directory of the job „/data/jenkins/jobs/example-java-maven/workspace“ has been mounted to.

As you can see in the Dockerfiles, the last CMD will be automatically executed when the images are run without interactive mode:

christoph@athena:~/toolcontainer-example$ sudo docker run \
    -v /tmp/example-java-maven/:/workspace builder_java18_mvn339

If you don’t need the maven-console log, you can run the container with „-d“ in detached mode. Anyway, if the Maven-build has been finished the running container will also exit. Because of the mounted workspace, the generated artifact in target directory will survive the end of container’s life. The result is a clean build that does not depend on Jenkins.

So everything Jenkins need to do is to run the correct buildcontainer. No tool installation no configuration files or similar required anymore. And the best: every dev-team can provide their own build-container, no matter if the want to use make, maven, ant, sbt, gradle, leiningen, python, gulp, grunt, composer or whatever-fancy-build-tool.

Of course, if you are working with a central Artifact-Repository like Nexus or Artifactory, you could extend this scenario by passing a configuration (e.g. settings.xml for maven, ivy.xml for ant, npmrc for npm, …) in a separate directory via „-v“ parameter to the buildcontainer. And if you really need it, you could of course share a local maven-repository between the containers. This would minimize the build-times because for now, every container starts with an empty repository and for large dependency trees this has effects on buildtime.