Docker container’s configuration

As we saw in the previous post, Docker basic usage seems pretty simple. Let’s try to dig a little deeper into the configuration management.

From an OPS perspective, we can consider containers as application black boxes, without caring about how they are built, and simply use them. However there are several aspects of a container that must be configured to ensure their correct integration in an information system.

Container run-time configuration

It is possible, and thus recommended, to limit CPU and memory usage in LXC, and therefore Docker containers. Docker run command line provides both limitations with the -m and -c options. But we might need to configure the container application itself to adapt its memory usage or processes/threads counts to match those settings.

We’ll try to illustrate this by improving the previous memcached example.

Docker provide a simple way to provide configuration options to containers by setting environment variables:

$ docker run -e MY_VAR=666 -t -i /bin/bash
[email protected]:/$ echo $MY_VAR        
[email protected]:/$ exit

Now we can adapt our startup script and Dockerfile to use this feature:

# Dockerfile

RUN dpkg-divert --local --rename --add /sbin/initctl
RUN ln -s /bin/true /sbin/initctl

RUN apt-get install -y memcached

EXPOSE 11211
CMD    ["/"]
USER daemon

And the startup script

# Default values if env vars are not set

exec memcached -m $mem -c $conn

If we rebuild the container and launch it will by default start with a 64MB memory usage and 1024 connections, but we can also force other values:

$ docker run -d -m=130m -e MEMCACHED_MEM=128 -e MEMCACHED_CON=512

A simple ps command confirm the variables handling.

  \_ lxc-start -n 9215e32d9bee4c632623b9d4cfff226f260069c28f94307989788e48c8fbf9a9 -f /var/lib/docker/containers/9215e32d9bee4c632623b9d4cfff226f260069c28f94307989788e48c8fbf9a9
      \_ memcached -m 128 -c 512

Applications that can natively rely on environment variables get reconfigured easily this way. For others it might be a little trickier:

# by default, start 4 workers unless overriden by the environment variable
exec /usr/sbin/nginx -g "worker_processes $workers;"

Inter-container communications

Topology-dependent variables must be injected into containers as soon as we try to make them communicate with each-other. Docker links feature simply set environment variables to provide such information.

In the following example, we’ll consider two nginx instances, one acting as a static content provider (web1), one acting as a reverse-proxy (rp1) in front of he first one:


Consider that we have two Docker images providing both roles: nginx:latest and nginx/rp:latest. We won’t get into much details about how the containers are built, this is not the point.

Let’s start the first static Nginx whose configuration is stand-alone, and name the running container web1:

$ docker run -d -name=web1

Now let’s start a simple bash into the reverse-proxy nginx and linking it to web1:

$ docker run -link=web1:web -t -i /bin/bash
[email protected]:/$ export | grep WEB
declare -x WEB_ENV_NGINX_WORKERS="1"
declare -x WEB_NAME="/insane_fermi/web"
declare -x WEB_PORT="tcp://"
declare -x WEB_PORT_80_TCP="tcp://"
declare -x WEB_PORT_80_TCP_ADDR=""
declare -x WEB_PORT_80_TCP_PORT="80"
declare -x WEB_PORT_80_TCP_PROTO="tcp"

All WEB_PORT_ variables can be used to configure our upstream to point to web1 nginx instance. Unfortunately, Nginx upstream configuration can’t be easily based on environment variables, this is the worst case scenario, and we will have to play with sed:

# Tweak a env variable to help templating a NGinx configuration file by removing the tcp:// prefix

# use a sed to create the final site.conf NGinx configuration file
sed -e "s/##NGINX_UPSTREAM_URL##/$NGINX_UPSTREAM_URL/g" /site.conf.tpl > /etc/nginx/sites-enabled/site.conf

with a template file site.conf.tpl which might look like:

upstream upstream_server  {
      server ##NGINX_UPSTREAM_URL##;

server {
    listen       80;
    location / {
     proxy_pass  http://upstream_server;
     proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
     proxy_redirect off;
     proxy_buffering off;
     proxy_set_header        Host            $host;
     proxy_set_header        X-Real-IP       $remote_addr;
     proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

We simply have to start the second nginx instance and publicly expose its HTTP port:

$ docker run -d -link=web1:web -name=rp1 -p=80:80

Wonderful, we now have a working chain. However two concerns remain:

  • On a multi-host toplogy (container running onto several physical or virtual hosts), the link feature won’t apply. Fortunately, if you can provide those exact same environment variables by other means, you will be able to make things work
  • What happens if you stop, destroy and recreate web1? Docker will allocate a new container which will unlikely have the same IP address: failure! Your environments variables used to start the reverse-proxy point to nothing interesting, you have to restart all the depending containers.

Tools like fig or MaestroNG have been written on top of Docker to help you handle such situations, MaestroNG being able to handle multi-hosts design. If you want to add even more control over the configuration and orchestration of your Docker containers, you still can code on top of frameworks like capistrano to do this.



  • Docker is doing well with cross-container communications by using env variables.
  • Some real simple topology management (links) can provide a static way of managing links.


  • On a larger scale, or in complicated toplogies, with lots of links and containers, it might make sense to use some orchestration tools on top of Docker to provide some high-level deployment management (rolling deployment, zero-downtime deployment…).
  • In very dynamic and elastic architectures, the use of distributed services registry such as Zookeeper, etcd or serf must be considered.

Un commentaire sur “Docker container’s configuration”

  • Alternatively, you can use Tiller ( which provides a standardized, easy to use way of generating configuration files. Many containers already use it, as it can generate configuration files from a variety of sources: environment variables; a HTTP webservice; zookeeper cluster; or even just plain YAML files on disk. I wrote it specifically so that you can build a single Docker container, but ship configuration files for different environments (or let users specify parameters at run time). In your example above, you'd just use standard ERb templates, so you could do something like: upstream upstream_server { server ; } Or whatever, and it would get filled in from environment variables. But you could use the defaults plugin to provide default values if the user omits them, and so on. There's some more examples on my blog, or linked from the on the github page.
    1. Laisser un commentaire

      Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *

      Ce formulaire est protégé par Google Recaptcha