Continuous Integration With Docker
Simon Waite
So it turns out I actually liked setting up a dev environment in a recent role. So I decided to update my personal setup as I had let things rot after Jenkins decided to self-destruct after an update.
So what did I use?
git-o-lite- A tiny no-frills git-based admin based on, git. Whilst not the most user friendly, You don’t have to wrestle with a web browser, log in and click. You just modify the admin repository, commit and all is good! Yes, even post-commit hooks! It’s so satisfying seeing a message from ‘git push’ telling you that it’s triggered a build!
Where?http://gitolite.com/gitolite/index.html
How?http://gitolite.com/gitolite/install.html#qi
gitweb- another low-tech, no frills setup. It’s just for me, and the occasional collaborator - so it’s all good!
Where? apt-get install gitweb
How? fiddling with gitolite-suexec-wrapper.sh, pointing /etc/apache2/suexec/www-data at the correct location and a healthy amount of symlinks so apt updates will work. (Sorry - too many site-specific details to write things up)
docker- I get tired of installing substantial apps to evaluate them only to discover they have a large vulnerability surface (What do all those ports do? Are they encrypted? Why are they all listening on 0.0.0.0 ?) or worse - decide that they’ll stomp all over your file system and need odd permissions. For me, docker promises a nice jail and private networking (If set up correctly) between swarm nodes.
Where? apt-get docker.io ( Sure you could also get it from https://get.docker.com/ which might work better for you, especially on debian/raspberry pis)
How? Lot’s of implementation specific things, however finding the right docker documentation is key:https://docs.docker.com/engine/swarm/
jenkins- Not only do they support docker, Jenkins has a habit of completely wrecking setups - solution; use docker container have its configuration folder on the host so everything can be backed up properly, and things can be rolled back easily. No more fighting between package management and auto-updates!
Where?https://hub.docker.com/_/jenkins/
How?
docker run -d --restart=always -p 127.0.0.1:8080:8080 -p 50000:50000 -p 50001:50001 --name jenkins -v /home/jenkins/volume:/var/jenkins_home jenkins
jenkins-slaves- A custom docker image custom ubuntu distro, with all the dev tools I need. If it all fails horribly, the images can be thrown away and downloaded automatically or regenerated. Three of them, using docker-swarm. So even if the nodes fail they’ll get restarted and if I add more compute to the mix, it’s super easy to scale up. (Install docker, connect to swarm, ask swam to add another jenkins-slave service)
Where? Custom DockerFile
How?
DockerFile:
FROM your-repo/your-base
WORKDIR /root
RUN mkdir /root/.ssh
COPY id_rsa /root/.ssh/
COPY id_rsa.pub /root/.ssh/
COPY known_hosts /root/.ssh/
COPY slave.jar /root/
RUN chmod 600 /root/.ssh /root/.ssh/*
RUN git config --global user.email "[email protected]" && git config --global user.name "Jenkins Slave"
Docker Runes
docker service create --name SLAVENAME --replicas=1 --with-registry-auth your-repo/your-slave-image java -jar slave.jar -jnlpUrl https://example.com/computer/SLAVENAME/slave-agent.jnlp -secret COMPUTER-SECRET
nexus3- I used an older version of this recently (2016) also, but the latest community edition supports hosting private docker repos and private nuget repos too. Perfect what I need. Quite a clunky java web app though. Containerised, with its files on the host, not for backing up though, as I can build what I need from source.
Where?https://hub.docker.com/r/sonatype/nexus3/
How?
docker run -d --restart=always -p 127.0.0.1:8081:8081 -p 127.0.0.1:30715:30715 --name nexus -v /home/nexus3/volume:/nexus-data sonatype/docker-nexus3
apache2/let’s encrypt- Probably the most annoying part of setting all this up. However let’s encrypt was a breeze. The main issues were organising the ports of the docker containers and setting up reverse proxies and reacquainting myself with some annoying changes since I last set up an Apache2 hosting platform a few years ago.
Where?
How? Lot’s of implementation specific things, so not appropriate for here.
But a basic reverse proxy config snippet will help:
RequestHeader set X-Forwarded-Proto “https”
RequestHeader set X-Forwarded-Port “443”
ProxyRequests off
ProxyPreserveHost on
ProxyPass / http://127.0.0.1:5000/
ProxyPassReverse / http://127.0.0.1:5000/
If you’re having problems with your directory stanzas try adding this:
Require all granted
Overall this probably took a week or two to set up, including learning docker (including DockerFiles), let’s encrypt, and evaluating potential CI and repository services (Travis, and nuget/docker-repo).
Summary
Projects get built automatically once submitted to git, and I’m slowly moving over projects and tidying them up, and my own nuget repo looks a bit like this in Xamarin Studio.
**Future work.**It would be nice to be able to set up a unity3d/unreal/xamarin builder at some point, get documentation auto-built, and hang a dedicated OSX/Windows builder off the swarm. Perhaps even an extra build step to push the source (but not my private makefiles) to github. But I’m in no hurry for these things as yet.