Docker, Spring Boot and JAVA_OPTS

While adjusting some environment variables recently, I came across an odd issue with Docker, Spring Boot and JAVA_OPTS. JAVA_OPTS comes from the Tomcat/Catalina world and when searching for “Docker and javaopts” on Google you’ll find many references to just adding JAVA_OPTS to the Docker environment. After some testing, I found this to be incorrect when running a Spring Boot jar in a Docker container, I’ll explain why and give a solution in this post.

Before I start, let’s setup a basic test environment that prints out the current memory setup so we can test in various situations. I’ve created this repo as a test case and you can refer back to it when needed.

Test Application Setup

Looking at the sample code, we have a basic pom file that imports Spring Boot dependencies:

Looking at the sample code, we have a basic pom file that imports Spring Boot dependencies:

<?xml version=”1.0" encoding=”UTF-8"?>
<project xmlns=”http://maven.apache.org/POM/4.0.0" xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation=”http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
 <modelVersion>4.0.0</modelVersion>

<groupId>vc.c4.javaopts</groupId>
 <artifactId>example-javaopts</artifactId>
 <version>0.0.1-SNAPSHOT</version>

<name>Spring Boot Tomcat JAVA_OPTS</name>
 <description>Spring Boot Tomcat JAVA_OPTS Example</description>
 <url>https://github.com/cl4r1ty/spring-boot-javaopts/</url>

<properties>
 <java.version>1.7</java.version>
 <main.basedir>${basedir}/../..</main.basedir>
 <spring-boot.version>1.3.1.RELEASE</spring-boot.version>
 </properties>

<dependencyManagement>
 <dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-dependencies</artifactId>
 <version>${spring-boot.version}</version>
 <type>pom</type>
 <scope>import</scope>
 </dependency>
 </dependencies>
 </dependencyManagement>

<dependencies>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter</artifactId>
 </dependency>
 <dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-test</artifactId>
 <scope>test</scope>
 </dependency>
 </dependencies>

<build>
 <plugins>
 <plugin>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-maven-plugin</artifactId>
 <version>${spring-boot.version}</version>
 <executions>
 <execution>
 <goals>
 <goal>repackage</goal>
 </goals>
 </execution>
 </executions>
 </plugin>
 </plugins>
 </build>

</project>

The main application is a single Java file that prints out memory details:

package vc.c4.javaopts;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

import java.text.NumberFormat;

@SpringBootApplication
public class JavaOptsApp {

private static Log logger = LogFactory.getLog(JavaOptsApp.class);

public static void main(String[] args) throws Exception {
 SpringApplication.run(JavaOptsApp.class, args);
 Runtime runtime = Runtime.getRuntime();

final NumberFormat format = NumberFormat.getInstance();

final long maxMemory = runtime.maxMemory();
 final long allocatedMemory = runtime.totalMemory();
 final long freeMemory = runtime.freeMemory();
 final long mb = 1024 * 1024;
 final String mega = “ MB”;

logger.info(“========================== Memory Info ==========================”);
 logger.info(“Free memory: “ + format.format(freeMemory / mb) + mega);
 logger.info(“Allocated memory: “ + format.format(allocatedMemory / mb) + mega);
 logger.info(“Max memory: “ + format.format(maxMemory / mb) + mega);
 logger.info(“Total free memory: “ + format.format((freeMemory + (maxMemory — allocatedMemory)) / mb) + mega);
 logger.info(“=================================================================\n”);
 }

}

Running this app in IntelliJ with no memory settings will show some basic output with memory settings:

========================== Memory Info ==========================
Free memory: 221 MB
Allocated memory: 245 MB
Max memory: 3,641 MB
Total free memory: 3,617 MB
=================================================================

Changing the memory values to -Xmx3g -Xms3g we get a different output:

========================== Memory Info ==========================
Free memory: 2,713 MB
Allocated memory: 2,944 MB
Max memory: 2,944 MB
Total free memory: 2,713 MB
=================================================================

Now that we have a baseline, let’s move onto the Docker stage.

Docker

The first thing we’ll do is setup a basic Dockerfile similar to the one in the Spring Boot guide. Since the jar will be built in the target folder we grab it from there and place a renamed copy into the container.

# Base java:8
FROM java:8

# Add jar to container
ADD /target/example*.jar javaopts.jar

# Entry in json format
ENTRYPOINT [“java”, “-jar”, “/javaopts.jar”]

To generate the jar, just run maven real quick.

mvn clean package

The jar will then be in the target folder ready for building the Docker container.

$ ls -ahl target
total 11760
drwxr-xr-x 7 clarity clarity 238B Dec 22 12:42 .
drwxr-xr-x 13 clarity clarity 442B Dec 22 12:42 ..
drwxr-xr-x 3 clarity clarity 102B Dec 22 12:42 classes
-rw-r — r — 1 clarity clarity 5.7M Dec 22 12:42 example-javaopts-0.0.1-SNAPSHOT.jar
-rw-r — r — 1 clarity clarity 3.3K Dec 22 12:42 example-javaopts-0.0.1-SNAPSHOT.jar.original
drwxr-xr-x 3 clarity clarity 102B Dec 22 12:42 maven-archiver
drwxr-xr-x 3 clarity clarity 102B Dec 22 12:42 maven-status

With the jar and the Dockerfile ready we can build the container from the base folder of the repository.

$ docker build -t spring-boot-javaopts .
Sending build context to Docker daemon 6.254 MB
Step 1 : FROM java:8
 — -> d4849089125b
Step 2 : ADD /target/example*.jar javaopts.jar
 — -> 518b9e05c3a9
Removing intermediate container cf4ff4db572a
Step 3 : ENTRYPOINT java -jar /javaopts.jar
 — -> Running in 15f032c1cc5c
 — -> 29a4656cdd5c
Removing intermediate container 15f032c1cc5c
Successfully built 29a4656cdd5c

We can see the newly built image by listing our Docker images.

spring-boot-javaopts$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
spring-boot-javaopts latest 29a4656cdd5c 8 seconds ago 648 MB

With our newly built container that will run Spring boot application let’s run it for the first time! Note: I’ll be cutting out some of the output lines to save space since we only care about the memory detail output.

$ docker run spring-boot-javaopts

2015–12–22 20:44:12.516 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : ========================== Memory Info ==========================
2015–12–22 20:44:12.516 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Free memory: 17 MB
2015–12–22 20:44:12.517 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Allocated memory: 30 MB
2015–12–22 20:44:12.517 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Max memory: 485 MB
2015–12–22 20:44:12.518 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Total free memory: 471 MB
2015–12–22 20:44:12.518 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : =================================================================

First run looks good. Let’s try setting the JAVA_OPTS as we’re used to!

$ docker run -e JAVA_OPTS=’-Xmx3g -Xms3g’ spring-boot-javaopts

2015–12–22 20:45:40.030 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : ========================== Memory Info ==========================
2015–12–22 20:45:40.030 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Free memory: 16 MB
2015–12–22 20:45:40.031 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Allocated memory: 30 MB
2015–12–22 20:45:40.031 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Max memory: 485 MB
2015–12–22 20:45:40.032 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Total free memory: 471 MB
2015–12–22 20:45:40.034 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : =================================================================

Woah! The Xmx and Xms settings didn’t have any affect on the Java settings in the container! Something must be wrong here and it’s not immediately obvious if all you search for is for “Docker and javaopts” on Google. Spring Boot is smart enough to handle many passed in environment variables but those are application specific. For example, when you look for a value x.y.z, Spring Boot will look in application.properties by default for x.y.z and in the environment variables for X_Y_Z. Due to this, passing in the environment variable -e X_Y_Z=1234 can be used in your application when using a Docker container. However, since JAVA_OPTS are used by Java and not in the application we run into this problem.

After some searching, I came across the fact that JAVA_OPTS are very specific to Catalina (Tomcat). Looking in the bin folder of a tomcat install you’ll find a shell script that handles passing JAVA_OPTS into the exec lines. With this info, we can now look to change the Dockerfile to adjust for handling passed in environment variables. I named this new Dockerfile ExecDockerfile] so I can have both in the same repo.

The change needed to enable JAVA_OPTS is to execute the Java line with an ENTRYPOINT exec command and place the environment variable into the line.

# Base java:8
FROM java:8

# Add jar to container
ADD /target/example*.jar javaopts.jar

# Entry with exec
ENTRYPOINT exec java $JAVA_OPTS -jar /javaopts.jar

Let’s rebuild the container but specify the new ExecDockerfile specifically using the -f flag.

$ docker build -f ExecDockerfile -t spring-boot-javaopts .
Sending build context to Docker daemon 6.254 MB
Step 1 : FROM java:8
 — -> d4849089125b
Step 2 : ADD /target/example*.jar javaopts.jar
 — -> Using cache
 — -> 518b9e05c3a9
Step 3 : ENTRYPOINT exec java $JAVA_OPTS -jar /javaopts.jar
 — -> Running in 87f0d69e6171
 — -> a5622e7d302e
Removing intermediate container 87f0d69e6171
Successfully built a5622e7d302e

Let’s run the container again.

$ docker run -e JAVA_OPTS=’-Xmx3g -Xms3g’ spring-boot-javaopts

2015–12–22 21:04:03.662 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : ========================== Memory Info ==========================
2015–12–22 21:04:03.662 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Free memory: 2,740 MB
2015–12–22 21:04:03.663 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Allocated memory: 2,969 MB
2015–12–22 21:04:03.663 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Max memory: 2,969 MB
2015–12–22 21:04:03.663 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : Total free memory: 2,740 MB
2015–12–22 21:04:03.664 INFO 1 — — [ main] vc.c4.javaopts.JavaOptsApp : =================================================================

Conclusion

By starting the jar with ENTRYPOINT exec java $JAVA_OPTS, we’ve successfully enabled changing memory settings using Docker environment variables. If you think you’re changing the memory settings currently but are not passing in $JAVA_OPTS into your start line in some fashion, I suggest you look at your Dockerfile config.