Java Performance - When 4294967296 bytes of address space isn't enough

the more memory you give to the JVM the more likely you are to get java.lang.OutOfMemoryError: unable to create new native thread exceptions when you have many threads.

Absurd but true: to create more threads you have to reduce the memory allocated to the JVM. Another option is to host the JVM in your own process using JNI.

I've done some interesting work lately, improving performance on a server designed to handle thousands of stateful concurrent connections. One of the major problems I've been seeing is the mass proliferation of threads. I was seeing memory errors like

java.lang.OutOfMemoryError unable to create new native thread

The problem with this message is that the server actually had plenty of memory to go around. The max heap was set to 1G, but it was only using up perhaps a few hundred megabytes of that. Initially, I thought that, perhaps, the memory being allocated for the thread's stack wasn't being reported against the heap and was causing these out of memory errors. I remembered reading somewhere that the default Window's thread stack size was 1M. According to MSDN, httpmsdn.microsoft.comlibrarydefault.aspurl=libraryen-usdllprocbasethread_stack_size.asp, what really happens, is that the operating system will reserve a certain amount of address space for each thread, but only commit a percentage of it to physical memory. The actual amount of memory committed and address space reserved isn't documented there, though it does note that if you change the initial amount committed to be larger than the default, the operating system will reserve the same amount of address space, rounded up to the nearest megabyte - hence using -Xss to change the initial stack size (even just slightly) can cause a much larger amount of address space to be reserved - potentially an entire megabyte.

After running some tests with the JVM, I've come to the conclusion that Sun's JDK 1.4 allocates about 256K of address space per thread. I also ran some tests with JDK 1.5, which seem to indicate that it was allocating about 1M of address space per thread. Those numbers come from an assumption of a 2G address space per process under Windows, and the following statistics for the maximum number of threads I could create

JDK1.4

-Xmx750 = 4580 threads.

-Xmx1000 = 3608 threads.

-Xmx1500M = 1663 threads

JDK1.5

-Xmx750M = 1129 threads

-Xmx1000M = 880 threads

-Xmx1500M = 384 threads

As you can see, the maximum number of threads I can create decreases as the heap size gets larger. This is because the JVM immediately reserves the address space specified for the maximum heap size - That's probably because it needs that memory to be contiguous.

This tells us two things

1) There are very finite limits on the maximum number of threads that you can create in an application, and those numbers are very small. This only reinforces the mantra that server's should be designed to operate with a fairly small and constant number of threads.

2) If you're really stuck creating a lot of threads, you can maximize the number available to you by reducing the address space allocated by the VM through a lower maximum heap size. You should also avoid changing the thread stack size, as a larger stack size can result in a much larger consumption of address space. Finally, unless things change, you should be prepared to run under JDK1.4 for a long time.