Docker Part IV - Running Multi-Container Application using Docker Compose

Hello, I am Malathi Boggavarapu and welcome back to my blog. I hope you had great learning sessions from my previous posts about Docker and how Docker works with several examples and pictures for easy illustration.

Now let's take a look at how we build and run a Multi-container application using Docker Compose. In the last section of this course, i provided a real world usecase and explained how we deploy application to Wildfly server and start database server just by using single docker-compose file.

Running Multi-Container Application using Docker Compose 

So far we have talked about a Single Container application. But in typical application you will have Webserver, Database server and Messaging server and each of those should be scaled up and down. That's where Docker compose comes in.

Docker Compose is the ability to define and run multi-container applications. The configuration of each container is defined in one or more files. The default file name is docker-compose.yml. It's a YAML definition where you  define what your services and what your containers in those services will look like. We also have docker-compose.override.yml file which gives ability to override the definition. We can have multiple files by specifying using -f. Just like Docker build, you should have standard directory where you put Docker compose and that's where the context is read from and it start copying files accordingly. It is very good for reducing Impedance mismatch between Dev, Staging and CI.

For example, if i  want to fireup the CouchBase server. Let's see at the Docker Compose file for this simple usecase.

version: "1"
services:
          db:
          image: couchbase
          volumes:
                     - ~/couchbase:/opt/couchbase/var
          ports:
                - 8091:8091
                - 8092:8092              

Now this version 1 have services. Service name is db and image that we are using is couchbase. Now if the container dies i still want to maintain the data. This is the stateful container. So i want to do the volume mapping. There is a concept of volume mapping in Docker where you say "everything in /opt/couchbase/var , map it to my home directory /couchbase". So all the data is written actually to my home directory. So if the container dies, i can fireup a new container and automatically map to the same volume and the state is maintained outside. I am not discussing about the volume mapping in depth in this course. And i am also doing port mapping here. As you already know the first port would be the localhost port and second port would be the container port. Container port is mapped towards the localhost port. Please watch  my post on Docker part III to learn about ports in docker-compose file.

Now we look at the usecase where we want to build the application with two containers.

So essentially we have one wildfly, one Couchbase. Couchbase is a NoSql database where you have your JSON documents stored over there. Couchbase also offers N1QL which allows you to write Sql queries on JSON documents. So i want to query my JSON documents using Sql like capabilities and expose Couchbase JSON documents as a REST endpoint on WildFly. So essentially a JavaEE application deployed on Wildfly using Sql like language to query Couchbase and exposing them as REST endpoints. That's what the application here is.

So the compose definition for the application should look like below.

version:"1"
services:
  db:
    image: couchbase
    ports: 
          - 8091:8091
          - 8092:8092
   web:
        image: wildfly
        environment: 
                - COUCHBASE_URI=db
        ports:
              - 8080:8080
              - 9990:9990

You can see two Services db and web. For service db, we have defined image as couchbase and also the ports . And we also had defined image as wildfly for service web and one important thing to be noted here is environment for the service web. environment is pointing to COUCHBASE_URI and again it is pointing to db service. So the db resolution is done at runtime by DNS.

So what's going to happen in this case is, it is going to download the couchbase image, wildfly image. In wildfly image there is already a Java EE application build in which is reading COUCHBASE_URI environment variable which is then pointing to db. So this Compose file brings up two containers and two containers are connected to each other.

Reducing Impedance Mismatch between Dev/Prod

So let's take a look at examples for clear understanding. I have a docker-compose.yml and docker-compose.override.yml. See below.

docker-compose.yml

web:
     image: jboss/wildfly
     ports:
            -8080:8080

docker-compose.override.yml

web:
     ports:
            - 9080: 8080

These two files are in same directory and now i say docker-compose up -d. It is going to startup wildfly but its going to be on port 9080. Because docker-compose.override.yml contains 9080 port, it actually overrides the port in docker-compose.yml file. 

Now why would i want to override?

Usually the configuration information about databases or some of the services would differ in Dev and Production. So when we are deploying an application we can have override file so that all the configuration related to prod will be overriden to the actual docker-compose file. Let's take an example where we create two files docker-compose.yml and production.yml files.

docker-compose.yml

db-dev: 
        image: couchbase
       ports: 
           -  ...
web:
     image: wildfly
    environment:
          - COUCHBASE_URI=db-dev:8093
    ports: 
           - 8080:8080

production.yml

web:
      environment:
             - COUCHBASE_URI=db_prod:8093
      ports:
             - 80:8080
      db_prod:
             image: ...

If i fire up the application using docker-compose up -f docker-compose.yml -f production.yml -d, it will override the ports from production.yml to docker-compose.yml file.

Service Discovery with Application Server and Database

Real world use-case

We will take a real world usecase that explains how to deploy the application to wildfly server and start the database server. We use wildfly application server to deploy the application and database as couchbase. Now we will look into how to write docker-compose file which contains two services. One for Wildfly application server and one for couchbase database. Since application will interact with database server, first database server should be Up and running before application server starts running. Otherwise it would throw exception and application would fail to start.

So let's see the docker-compose file which is comprised with  two containers.

version: "2"
services:
      db: 
         container_name: "db"
         image: couchbase:latest
         ports:
              - 8091:8091
     web:
          image: wildfly:latest
          depends_on:
                - db
          environment:
               - COUCHBASE_URI=db
          ports:
                -8080:8080

The above shown docker-compose file  is as usual as the example that i have discussed above but we have one important attribute to notice here. We use depends_on attribute. So Docker will fire up the database container First and web container second. Here the containers are initialized and fired up according to the attribute depends_on but there should also be a check at application level to see whether the database in the container is Up and running. That is very important part to understand. Because the database server may take some time to be Up and Running and start accepting the application requests.

Now if we run docker-compose up -d, It downloads both the images and then it fire up the containers. In this particular case both the containers come up rather quickly but couchbase container takes few seconds before it is ready to start accepting requests. Wildfly container comes up in 3 seconds. So when the application server is trying to connect to the database and the database is not Up yet, wildfly (application server) fails in that case. So we will give it a few seconds and then we restart just the web container, just the web service part of it and if we do that, by that time database is fired up and is ready to accept the requests.

This gives the ability where you start building the logic in the applicaiton where if the application is trying to connect to database and database is failing, you retry in few seconds. This is the real scenario that would happen in the real world.

Later we run docker ps, it gives the containers that were running currently. So copy the container_id of wildfly server and run the below command

docker logs [container_id]

For example if we say container_id is 08fe457891 and we run docker logs 08fe457891, it shows the log data of wildfly server. Similarly you can check the logs for database server too.

Now open the browser and try localhost:8080 and see that your wildfly server is running. And later you can check whether the application that is deployed into the server is running as well.

If we check the Dockerfile for the image wildfly, it should be as follows

FROM jboss/wildfly:latest
COPY javaeeSample/target/books.war /opt/jboss/wildfly/standalone/deployments/books.war

I am taking the books.war file from the local directory and bundling it and deploying to the /opt/jboss/wildfly/standalone/deployments directory.

So that's all about this session. Hope the course is helpful. Please comment on the post and make the session interactive.  The more the session is interactive, the more we learn.

Happy Learning!!