A few days ago I came across an interesting article, Where the printf() Rubber Meets the Road, describing how the printf function ‘works’ on the low level.
Commonly asked by Java beginners is the question, “How does System.out.println() work?”; the above blog post inspired me to do some research into this question. In this blog post I’ll attempt to provide an explanation of what System.out.println() does behind the scenes.
Most of the relevant code can be found on OpenJDK, which is the primary implementation of Java itself. System.out.println() may be implemented completely differently on other Java platforms.
I will warn you now that this article is very long and not for the easily bored.
Our first step in figuring out how System.out.println works is by first understanding what System.out is and how it came to be.
Let’s take a look through the OpenJDK’s Mercurial online repository. Digging around a bit, we find System.java. In this file System.out is declared:
But when we find the code for nullPrintStream():
So nullPrintStream() simply returns null or throws an exception. This can’t be it. What’s going on here?
The answer can be found in the function initializeSystemClass(), also in System.java:
There’s a lot of stuff going on in this code. I’m going to refer back to this two lines of code later, but setOut0() is what actually initializes System.out.
The function setOut0() is a native function. We can find its implementation in System.c:
This is pretty standard JNI code that sets System.out to the argument passed to it.
At first all this deal with setting System.out to nullPrintStream() and later setting it with JNI seems entirely unnecessary. But this is actually justified.
In Java, static fields are initialized first, and everything else comes after. So even before the JVM and the System class is fully initialized, the JVM tries to initialize System.out.
Unfortunately at this point the rest of the JVM isn’t properly initialized so it’s impossible to reasonably set System.out at this point. The best that could be done would be to set it to null.
The System class, along with System.out is properly initialized in initializeSystemClass() which is called by the JVM after static and thread initialization.
There is a problem, however. System.out is final, meaning we cannot simply set it to something else in initializeSystemClass(). There’s a way around that, however. Using native code, it is possible to modify a final variable.
Wait, what’s a FileDescriptor?
Notice this line of code:
A FileOutputStream object is created from something referred to as FileDescriptor.out.
The FileDescriptor class, though part of java.io, is rather elusive. It can’t be found in the java.io directory in OpenJDK.
This is because FileDescriptor is much lower level than most of the Java standard library. While most .java files are platform independent, there are actually different implementations of FileDescriptor for different platforms.
We’ll be using the Linux/Solaris version of FileDescriptor.java.
A FileDescriptor object is very simple. Essentially all it really holds is an integer. It holds some other data too, which aren’t really important. The constructor of FileDescriptor takes an integer and creates a FileDescriptor containing that integer.
The only use of a FileDescriptor object is to initialize a FileOutputStream object.
Let’s see how FileDescriptor.out is defined:
FileDescriptor.out is defined as 1, in as 0, and err as 2. The basis of these definitions are from a very low level somewhere in Unix.
We now know how System.out is initialized. For now, we’re going to leave behind the FileDescriptor; we only need to know what it does.
A tour through java.io
Now we redirect our attentions to the println() function of PrintStream.
PrintStream is a comparably higher level class, capable of writing many different kinds of data, flushing and handling errors for you without much effort.
Let’s see how println() is defined in PrintStream.java:
Following the call stack to print():
Going deeper, and looking at write():
Internally, the PrintStream object (System.out) contains three different objects to do its work:
- The OutputStreamWriter object (charOut), writing character arrays into a stream
- The BufferedWriter object (textOut), writing not only character arrays but also strings and text
- A BufferedOutputStream object (out), passed all the way down the call stack and used much lower then at the PrintStream level
We can see that PrintStream.write() calls BufferedWriter.write() and flushes both buffers. I’m not sure why it’s necessary to flush the charOut buffer, so I’m going to ignore that.
Moving back to BufferedWriter:
As its name suggests, BufferedWriter is buffered. Data is stored in a data buffer until it’s written all at once, or flushed. Buffered IO is much faster than simply writing to the hardware one byte at a time.
The function BufferedWriter.write() doesn’t actually write anything. It only stores something in an internal buffer. The flushing is not done here, but back at PrintStream.write().
Let’s go to flushBuffer(), in the same file:
We find yet another write() call, on a Writer object (out). The out object here is the charOut object of PrintStream, and has the type OutputStreamWriter. This object is also the same object as charOut in PrintStream.
Let’s look at OutputStreamWriter.write() in OutputStreamWriter.java:
This now transfers the job to another object, se. This object is of type sun.nio.cs.StreamEncoder. We’re going to leave the java.io directory for a while.
Let’s see the implementation of StreamEncoder.write() inStreamEncoder.java:
Moving on to StreamEncoder.implWrite():
Again this calls another function, writeBytes(). Here’s the implementation:
We’re done with StreamEncoder. This class essentially processes or encodes character streams, but ultimately delegates the task of writing the bytes back to BufferedOutputStream.
Let’s take a look at the code for write() in BufferedOutputStream.java:
And BufferedOutputStream passes the baton again, this time to FileOutputStream. Remember when we instantiated fdOut as a FileOutputStream? Well, this is it, passed down through dozens of system calls.
Believe it or not, FileOutputStream is the final layer before JNI. We see the function write() in FileOutputStream.java:
We’ve reached the end of the Java part. But we’re not quite finished.
A Review of the java.io call stack
This is a ‘contains’ chart:
Also here’s the entire call stack:
Stepping into the JNI
After FileOutputStream, the writing of bytes to the console is handled natively. Much of this native code is platform dependent: there are different versions of the code for Windows and Linux. We’re going to deal with the Linux versions first.
The native implementation of writeBytes() is defined inFileOutputStream_md.c.
The field fos_fd is the integer stored in the FileDescriptor object that we’ve visited so long ago. So for the out stream, fos_fd should be 1.
We’re just calling a method, writeBytes, with the additional argument of fos_id. The implementation of writeBytes() is defined in io_util.c:
The writing here is done by a method called IO_Write. At this point, what happens next becomes platform dependent, as IO_Write is defined differently for Windows and Linux.
The Linux Way
The linux way of handling IO uses the HPI (Hardware Platform Interface). Thus, the method is defined as JVM_Write in io_util_md.h:
The code form JVM_Write is defined in the JVM itself. The code is not Java, nor C, but it’s C++. The method can be found in jvm.cpp:
The writing is now done by various HPI methods. Although you could go further, I’m going to stop here, since we’re now so far from where we started.
The Way of Windows
In Windows, the method IO_Write is routed away from the HPI layer. Instead, it’s redefined as handleWrite in io_util_md.h.
The implementation for handleWrite() is defined in io_util_md.c:
The WriteFile function is in the Windows API. The Windows API is not open source, so we would have to stop here.
We’ve taken a tour through the entire call stack of System.out.println(), from the instantiation of System.out to the path through java.io, all the way down to JNI and HPI level IO handling.
We haven’t even reached the end. No doubt there’s dozens more levels underneath the HPI layer and the WriteFile API call.
Perhaps the better answer to the question, “how does System.out.println() work” would be “it’s magic”.