Making the build of Docker images re-usable

Development
#1

Hi everyone,

I recently had the opportunity to work on the packaging of a custom XWiki WAR within a Docker image, while trying to rely as much as possible on the standard Docker build we provide in the platform *.

As part of this process, I observed that it was not that easy nor that efficient to replace the XWiki Standard WAR. There are basically two approaches :

  • Either we need to re-build the whole docker image from scratch, starting from the base image of Tomcat.
  • Or we need to take the image we want to build upon, remove the XWiki Standard WAR from this image, put the customized WAR, and put back the configuration files and the JDBC driver.

This second solution is a bit easier to maintain because some steps, such as the installation of LibreOffice are already taken care of in the standard image. However, it has the disadvantage of generating very big images, because docker images are made out of layers, each layer being a patch on top of its parent layer. We then have 3 big layers in our image :

  • A first layer where the XS WAR is unzipped in the container
  • A second layer where the XS WAR is removed from the container
  • A third layer where the custom WAR is unzipped

I would have loved to just deploy sort of a “patch” on top of the XS WAR : adding and removing only the files that are relevant compared to what’s already in the XS WAR, however unfortunately I did not find any Maven artefact type that would be able to do that. If you have suggestions, I’ll be more than happy to know about them.

With the help of @gsautner, we tried to look at how it would be possible to make the build of the XWiki Docker images more modular. Much like the build of xwiki-platform-distribution-debian-common which produce an intermediate artefact, we looked into how the build of XWiki’s Docker images could be modified to be re-usable downstream.

Currently, the build of the XWiki Docker image is roughly divided in 5 parts :

  • The installation of prerequisites (LibreOffice + utilities needed to install the WAR)
  • The installation of the WAR in Tomcat’s ROOT webapp + the installation of configuration files
  • The installation of the JDBC driver
  • Touch ups on the WAR (deployment of configuration files, update of the distribution ID)
  • The installation of scripts for starting the docker image properly (such as docker-entrypoint.sh)

The two steps that are in italic (WAR install + touch ups on the WAR) are most probably the ones that would be valuable to override as part of projects deploying custom WARs in Docker images. With that in mind, one approach would be to update the build steps of our docker image to produce two intermediary images :

  • The first intermediary image could contain the following steps :
    • The installation of prerequisites (LibreOffice + utilities needed to install the WAR)
    • The installation of scripts for starting the docker image properly (such as docker-entrypoint.sh)
  • The second intermediary image would contain the setup of the database (there would be 1 intermediate image per database variant), with the following steps :
    • The installation of the JDBC driver
    • The installation of a default Hibernate configuration in /etc/xwiki
  • Finally, we would be left with two last steps that could be overridden more easily, by starting from the second intermediary image :
    • The installation of the WAR in Tomcat’s ROOT webapp + the installation of configuration files
    • Touch ups on the WAR (deployment of configuration files, update of the distribution ID)

WDYT ? Do you think that it is interesting to go forward with these changes, or does it bring too much complexity ? To note that, in that case, we would probably need to consider two changes to how we build the image :

  • On one hand, deploy configuration files in /etc/xwiki instead of updating the WAR. /etc/xwiki could actually be defined as a volume.
  • On the other hand, deploy the JDBC drivers in the Tomcat lib folder, insetad of the WAR WEB-INF/lib folder.

Thanks,
Clément

* The project through which this work was done is accessible here ; the code related to the creation of a Docker image based on a different WAR is accessible here. We use jib to perform operations on top of base docker images as part of a Maven build.

#2

Hi Clement. I would try to avoid having to create and maintain intermediary images if possible.

I can think of 3 ideas:

  1. We could define an ARG for the location of the XWiki WAR so that you could provide a local one (using file:// for example or using https:// if your custom WAR is available from a server). Another ARG coud be defined for the distribution id. See Using parameterized docker builds – Schneide Blog
  2. Maybe you could try using multi-stage builds to avoid creating too many layers, you’d depend on the XWiki image and copy stuff from it. See Multi-stage builds | Docker Documentation
  3. Explore the usage of tools such as docker slim to reduce the image size, see GitHub - slimtoolkit/slim: Slim(toolkit): Don't change anything in your container image and minify it by up to 30x (and for compiled languages even more) making it secure too! (free and open source)

Let’s continue the brainstorming!

#3

Thanks a lot Vincent for the helpful links ! I’ll look into them !