Building containers with Docker in Docker and Jenkins

Today I wanted to share some of my experiences I have when I am at home experimenting with Docker. I really love Docker it makes my life of deploying my software a lot easier. However as a hobbyist in my home setup I face one challenge and that is limited funds. I don’t have a nice build farm at home, and I do not want to run a cloud build server 24/7.

However what I want to have is a reliable Continuous Integration environment on hardware that is not always on and might be portable. Based on this I set out to run Jenkins as a Docker container which is relatively simple. You can simply run ”’docker run -d -p”8080:8080″ jenkinsci/jenkins”’ and voila a running Jenkins installation. For most of my challenges this indeed seems to be the case, I can create portable projects using Jenkins pipelines where I just need to point Jenkins to my github account.

Docker in Docker with Jenkins

The main challenge I faced was that I actually also wanted to build new containers as part of the CI setup. But the problem here is that in order to build a Docker container there needs to be a running Docker engine in the Jenkins host. I actually would like to have access to a Docker engine inside my already existing Docker container so I can build new containers.

There are actually multiple solutions to the problem:
1. Really run a Docker engine inside the Jenkins Docker container
2. Map the Docker from the Jenkins Docker host to be accesible inside the Jenkins container

There is an obvious downside to nr 1, and many other blogs do not recommend using this ever…. So this left me with solution nr 2. using the already running Docker inside my parent host environment.

Solving the problem

In order to make this work I need to do a few things:

1: Install the Docker client inside the Jenkins container
The first part is actually the hardest, I can make the docker engine of the host system available but I still need the client tools installed. The default Jenkins container does ofcourse not contain these tools. As I had to make some more modifications to my Jenkins container I set out to simply extend the current container.

In order to install the Docker CLI, you can use the below Dockerfile to extend the official Jenkins 2.x container with the latest docker client that is available.

from jenkinsci/jenkins:latest

USER root
RUN apt-get update -qq
RUN apt-get install -qqy apt-transport-https ca-certificates
RUN apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
RUN echo deb https://apt.dockerproject.org/repo debian-jessie main > /etc/apt/sources.list.d/docker.list
RUN apt-get update -qq
RUN apt-get install -qqy docker-engine

USER jenkins

Next thing I just need to build this container once on the host system:
docker build -t myjenkins .

2: Start the Jenkins container with a mapper Docker socket
In order to make the Docker from the host system available I need to make the API available to the Jenkins docker container. You can do this by mapping the docker socket that is available on the parent system. I have created a small docker-compose file where I map both my volumes and the docker socket as following:

jenkins:
  container_name: jenkins
  image: myjenkins:latest
  ports:
    - "8080:8080"
  volumes:
    - /Users/devuser/dev/docker/volumes/jenkins:/var/jenkins_home
    - /var/run:/var/run:rw

Please not specially mapping the ‘/var/run’ with rw privileges, this is needed to make sure the Jenkins container has access to the host systems docker.sock.

3: Building a container
In order to demonstrate what better way than to create a Jenkins pipeline project that can build the Jenkins Docker container itself 🙂

In essence it is quite simple, using Jenkins 2.x you can create a Jenkins Pipeline project and supply the project with this Jenkinsfile in the configuration:

node {
   stage 'Checkout'

   git credentialsId: 'aa123c45-6abc-78e9-a1b2-583c1234e123', url: 'git@bitbucket.org:myorg/jenkins-repo.git'

   stage 'Build Jenkins Container'
   sh "docker pull jenkinsci/jenkins:latest"
   sh "docker build -t myjenkins -f ./jenkins/Dockerfile ."
}

This above Jenkinsfile checks out a git repository where I have put the Dockerfile for the Jenkins container. It uses the credentials that are already saved inside Jenkins for providing the credentials for GIT.

Because I always want the latest Jenkins 2.x container, i first pull in the latest version from Docker hub. The next stage is to simply run the Docker build and voila we have a completely build Jenkins container.

If i now run docker images in the host I see the following:

REPOSITORY                                                         TAG                 IMAGE ID            CREATED             SIZE
myjenkins                                                          latest              f89d0d6ba4a5        24 seconds ago        1.07 GB

Conclusion

When using standard home setups where hardware does not run fulltime you have some limitations. However with Docker and Jenkins you can create a very nice and effective Docker CI pipeline.

In some of the next few blog posts I will detail more about how to create a fully fledged Kubernetes deployment using Jenkins and Docker.

Advertisements

4 thoughts on “Building containers with Docker in Docker and Jenkins

  1. Hi, thanks for the great article. Recently I was working on several problem, but for kitchen.ci tooling accessing host’s dockerd. I’ve learned that it is possible to mount socket directly as in
    – /var/run/docker.sock:/var/run/docker.sock
    So you don’t need to provide access to whole /var/run to container
    Cheers,
    Yury

    Like

    1. Hi,

      Unfortunately on my mac’s that does not work because i get permission errors. I have not really tried resolving that particular permission problem due to difference in owner between the container and the host system. Could very well that its resolvable and then this would for sure be the preferred solution.

      Cheers,
      Renze

      Like

  2. Hey, thanks for your post about mapping docker sock to Jenkins. The other thing I need for my Jenkins is to be able to be a bastion and ssh into other hosts to build. What are your thoughts on how to manage this with credentials and networking etc?

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s