01-JAVA I/O - OverView

The InputStream, OutputStream, Reader and Writer

A program that needs to read data from some source needs an InputStream or a Reader. A program that needs to write data to some destination needs an OutputStream or a Writer. This is also illustrated in the diagram below:

An InputStream or Reader is linked to a source of data. An OutputStream or Writer is linked to a destination of data.

InputStream

The class java.io.InputStream is the base class for all Java IO input streams. If you are writing a component that needs to read input from a stream, try to make our component depend on an InputStream, rather than any of it’s subclasses (e.g. FileInputStream). Doing so makes your code able to work with all types of input streams, instead of only the concrete subclass.

Depending on InputStream only isn’t always possible, though. If you need to be able to push back data into the stream, you will have to depend on a PushbackInputStream - meaning your stream variable will be of this type. Otherwise your code will not be able to call the unread() method on the PushbackInputStream.

You typically read data from an InputStream by calling the read() method. The read() method returns a int containing the byte value of the byte read. If there is no more data to be read, the read() method typically returns -1;

Here is a simple example:

InputStream input = new FileInputStream("c:\\data\\input-file.txt");

int data = input.read();

while(data != -1){
  data = input.read();
}

OutputStream

The class java.io.OutputStream is the base class of all Java IO output streams. If you are writing a component that needs to write output to a stream, try to make sure that component depends on an OutputStream and not one of its subclasses.

Here is a simple example pushing some data out to a file:

OutputStream output = new FileOutputStream("c:\\data\\output-file.txt");
output.write("Hello World".getBytes());
output.close();

PipeOutputStream and PipeInputStream

Pipes in Java IO provides the ability for two threads running in the same JVM to communicate. Therefore pipes can also be sources or destinations of data.

Creating a pipe using Java IO is done via the PipedOutputStream and PipedInputStream classes. A PipedInputStream should be connected to a PipedOutputStream. The data written to the PipedOutputStream by one thread can be read from the connected PipedInputStream by another thread.

简单来说Java I/O 提供的PipeStream 可以实现 两个线程之间的数据流通信。

下面是演示代码

值得注意的是：如果用同一个线程写入和读取，会发生触发死锁。
Remember, when using the two connected pipe streams, pass one stream to one thread, and the other stream to another thread. The read() and write() calls on the streams are blocking, meaning if you try to use the same thread to both read and write, this may result in the thread deadlocking itself.

/**
 * @Classname PipeDemo01
 * @Description 演示Pipe的简单使用方法
 * @Date 2021/3/5 22:09
 * @Created by YoungLiu
 */
public class PipeDemo01 {
    public static void main(String[] args) throws IOException {
        final PipedOutputStream os = new PipedOutputStream();
        final PipedInputStream is = new PipedInputStream(os);


        Thread thread = new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    os.write("Hello world pipe".getBytes());
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        });

        Thread thread1 = new Thread(new Runnable() {
            @Override
            public void run() {
                try {
                    int data = is.read();
                    while(data!=-1){
                        System.out.println((char) data);
                        data=is.read();
                    }
                } catch (IOException e) {
                    e.printStackTrace();
                }

            }
        });

        thread.start();
        thread1.start();

    }
}

Reading Arrays via InputStream or Reader

To make such a component read from the data from an array, you will have to wrap the byte or char array in an ByteArrayInputStream or CharArrayReader. This way the bytes or chars available in the array can be read through the wrapping stream or reader.
Here is a simple example

byte[] bytes = new byte[1024];

//here stimulate write data into byte array

InputStream input = new ByteArrayInputStream(bytes);
//read first byte
int data = input.read();
while(data != -1){
	//do something with data

	// read next byte
	data  = input.read();
}

Writing to Arrays via OutputStream or Writer

It is also possible to write data to an ByteArrayOutputStream or CharArrayWriter. All you have to do is to create either a ByteArrayOutputStream or CharArrayWriter, and write your data to it, as you would to any other stream or writer. Once all the data is written to it, simply call the method toByteArray() or toCharArray, and all the data written is returned in array form.

Here is a simple example

ByteArrayOutputStream output = new ByteArrayOutputStream();

output.write("This text is converted to bytes".getBytes("UTF-8"));

byte[] bytes = output.toByteArray();

Java System.in, System.out, and System.err

Java has 3 streams called System.in, System.out, and System.err which are commonly used to provide input to, and output from Java applications. Most commonly used is probably System.out for writing output to the console from console programs (command line applications).

System.in, System.out and System.err are initialized by the Java runtime when a Java VM starts up, so you don’t have to instantiate any streams yourself (although you can exchange them at runtime). I will explain each of these streams in deeper detail later in this tutorial

System.in

System.in is an InputStream which is typically connected to keyboard input of console programs. In other words, if you start a Java application from the command line, and you type something on the keyboard while the CLI console (or terminal) has focus, the keyboard input can typically be read via System.in from inside that Java application. However, it is only keyboard input directed to that Java application (the console / terminnal that started the application) which can be read via System.in. Keyboard input for other applications cannot be read via System.in .

System.in is not used as often since data is commonly passed to a command line Java application via command line arguments, files, or possibly via network connections if the application is designed for that. In applications with GUI the input to the application is given via the GUI. This is a separate input mechanism from System.in.

System.out

System.out is a PrintStream to which you can write characters. System.out normally outputs the data you write to it to the CLI console / terminal. System.out is often used from console-only programs like command line tools as a way to display the result of their execution to the user. This is also often used to print debug statements of from a program (though it may arguably not be the best way to get debug info out of a program).

System.err

System.err is a PrintStream. System.err works like System.out except it is normally only used to output error texts. Some programs (like Eclipse) will show the output to System.err in red text, to make it more obvious that it is error text.

System.out and System.err sample

try {
  InputStream input = new FileInputStream("c:\\data\\...");
  System.out.println("File opened...");

} catch (IOException e){
  System.err.println("File opening failed:");
  e.printStackTrace();
}

Exchanging System Streams

Even if the 3 System streams are static members of the java.lang.System class, and are pre-instantiated at JVM startup, you can change what streams to use for each of them. Just set a new InputStream for System.in or a new OutputStream for System.out or System.err, and all further data will be read / written to the new stream.

To set a new System stream, use one of th emethods System.setIn(), System.setOut() or System.setErr(). Here is a simple example:

OutputStream output = new FileOutputStream("c:\\data\\system.out.txt");
PrintStream printOut = new PrintStream(output);

System.setOut(printOut);

Now all data written to System.out should be redirected into the file “c:\data\system.out.txt”. Keep in mind though, that you should make sure to flush System.out and close the file before the JVM shuts down, to be sure that all data written to System.out is actually flushed to the file

Write Performance

It is faster to write an array of bytes to a Java OutputStream than writing one byte at a time. The speedup can be quite significant - up to 10 x higher or more. Therefore it is recommended to use the write(byte[]) methods whenever possible.

The exact speedup you get depends on the underlying OS and hardware of the computer you run the Java code on. The speedup depends on issues like memory speed, hard disk speed and buffer sizes, or network card speed and buffer sizes, depending on which destination the OutputStream sends its data to.

Transparent Buffering via BufferedOutputStream

You can get transparent buffering of bytes written to a Java OutputStream by wrapping it in a Java BufferedOutputStream . All bytes written to the BufferedOutputStream will first get buffered inside an internal byte array in the BufferedOutputStream. When the buffer is full, the buffer is flushed to the underlying OutputStream all at once. Here is an example of wrapping a Java OutputStream in a BufferedOutputStream:

int bufferSize = 8 * 1024;

OutputStream outputStream =
    new BufferedOutputStream(
          new FileOutputStream("c:\\data\\output-file.txt"), bufferSize);

flush()

The Java OutputStream’s flush() method flushes all data written to the OutputStream to the underlying data destination. For instance, if the OutputStream is a FileOutputStream then bytes written to the FileOutputStream may not have been fully written to disk yet. The data might be buffered in OS memory somewhere, even if your Java code has written it to the FileOutputStream. By calling flush() you can assure that any buffered data will be flushed (written) to disk (or network, or whatever else the destination of your OutputStream has). Here is an example of flushing data written to a Java OutputStream by calling its flush() method:

outputStream.flush();

Close an OutputStream

Once you are done writing data to a Java OutputStream you should close it. You close an OutputStream by calling its close() method. Here is an example of closing a Java OutputStream:

OutputStream outputStream = new FileOutputStream("c:\\data\\output-text.txt");

while(hasMoreData()) {
    int data = getMoreData();
    outputStream.write(data);
}
outputStream.close();

Convert OutputStream to Writer

The Java OutputStream is a byte based stream. You can convert a OutputStream to a character based Writer using the Java OutputStreamWriter class. Here is an example of converting a Java OutputStream to a Writer using OutputStreamWriter:

OutputStream outputStream       = new FileOutputStream("c:\\data\\output.txt");
Writer       outputStreamWriter = new OutputStreamWriter(outputStream);

outputStreamWriter.write("Hello World");

FileInputStream

The Java FileInputStream class, java.io.FileInputStream, makes it possible to read the contents of a file as a stream of bytes. The Java FileInputStream class is a subclass of Java InputStream. This means that you use the Java FileInputStream as an InputStream (FileInputStream behaves like an InputStream).

Java FileInputStream Example

InputStream input = new FileInputStream("c:\\data\\input-text.txt");

int data  = input.read();
while(data !=-1){
// do sth with data...
doSomethingWithData(data);

data = input.read()
}

input.close();

FileInputStream Constructors

The FileInputStream class has a three different constructors you can use to create a FileInputStream instance. I will cover the first two here.

The first constructor takes a String as parameter. This String should contain the path in the file system to where the file to read is located. Here is a code example

String path = "C:\\user\\data\\thefile.txt";

FileInputStream fileInputStream = new FileInputStream(path);

Notice the path String. It needs double backslashes (\) to create a single backslash in the String, because backslash is an escape character in Java Strings. To get a single backslash you need to use the escape sequence \.

On unix the file path could have looked like this:

String path = "/home/jakobjenkov/data/thefile.txt";

The second FileInputStream constructor takes a File object as parameter. The File object has to point to the file you want to read. Here is an example:

String path = "C:\\user\\data\\thefile.txt";
File   file = new File(path);

FileInputStream fileInputStream = new FileInputStream(file);

Which of the constructors you should use depends on what form you have the path in before opening the FileInputStream. If you already have a String or File, just use that as it is. There is no particular gain in converting a String to a File, or a File to a String first.

Read()

The read() method of a FileInputStream returns an int which contains the byte value of the byte read. If the read() method returns -1, there is no more data to read in the FileInputStream, and it can be closed. That is, -1 as int value, not -1 as byte value. There is a difference here!

You use the read() method just like the read() method of an InputStream. Here is an example of reading all data in a Java FileInputStream :

FileInputStream fileInputStream =
    new FileInputStream("c:\\data\\input-text.txt");


int data = fileInputStream.read();
while(data != -1) {
    // do something with data variable

    data = fileInputStream.read(); // read next byte
}

read(byte[])

FileInputStream fileInputStream = new FileInputStream("c:\\data\\input-text.txt");

byte[] data      = new byte[1024];
int    bytesRead = fileInputStream.read(data, 0, data.length);

while(bytesRead != -1) {
  doSomethingWithData(data, bytesRead);
//Notice that read(data, 0, data.length) is equivalent to read(data) .
  bytesRead = fileInputStream.read(data, 0, data.length);
}

Read Performance

Reading an array of bytes at a time is faster than reading a single byte at a time from a Java FileInputStream. The difference can easily be a factor 10 or more in performance increase, by reading an array of bytes rather than reading a single byte at a time.

The exact speedup gained depends on the size of the byte array you read, and the OS, hardware etc. of the computer you are running the code on. You should study the hard disk buffer sizes etc. of the target system before deciding. However buffer sizes of 8KB and up will give a good speedup. However, once your byte array exceeds the capacity of the underlying OS and hardware, you won’t get a bigger speedup from a bigger byte array.

You will probably have to experiment with different byte array size and measure read performance, to find the optimal byte array size.

FileOutputStream

Overwriting vs. Appending the File

When you create a Java FileOutputStream pointing to a file that already exists, you can decide if you want to overwrite the existing file, or if you want to append to the existing file. You decide that based on which of the FileOutputStream constructors you choose to use.

This constructor which takes just one parameter, the file name, will overwrite any existing file:

OutputStream output = new FileOutputStream("c:\\data\\output-text.txt");

There is a constructor that takes 2 parameters too: The file name and a boolean. The boolean indicates whether to append to the file or not. Thus, a value of true means that you want to append to the file, whereas a value of false means you want to overwrite the file. Here are two Java FileOutputStream constructor examples:

OutputStream output = new FileOutputStream("c:\\data\\output-text.txt", true); //append

OutputStream output = new FileOutputStream("c:\\data\\output-text.txt", false); //overwrite

When you leave out the second boolean parameter and thus just use the constructor that takes a file path, the default mode is to overwrite any existing file on the given path.

Write()

To write data to a Java FileOutputStream you can use its write() method. The write() method takes an int which contains the byte value of the byte to write. Thus, only the lower 8 bit of the passed int actually gets written to the FileOutputStream destination. Here is an example of writing data to a Java FileOutputStream using its write() method:

OutputStream outputStream = new FileOutputStream("c:\\data\\output-text.txt");

outputStream.write(123);

Also you can write byte arrays by write() method, instead of write single byte value of byte.

OutputStream outputStream = new FileOutputStream("c:\\data\\output-text.txt");

byte bytes =  new byte[]{1,2,3,4,5};

outputStream.write(bytes);

Write Performance

It is faster to write an array of bytes to a Java FileOutputStream than writing one byte at a time. The speedup can be quite significant - up to 10 x higher or more. Therefore it is recommended to use the write(byte[]) methods whenever possible.

The exact speedup you get depends on the underlying OS and hardware of the computer you run the Java code on. The speedup depends on issues like memory speed, hard disk speed and buffer sizes.

Transparent Buffering via BufferedOutputStream

You can get transparent buffering of bytes written to a Java FileOutputStream by wrapping it in a Java BufferedOutputStream . All bytes written to the FileOutputStream will first get buffered inside an internal byte array in the BufferedOutputStream. When the buffer is full, the buffer is flushed to disk all at once. Here is an example of wrapping a Java FileOutputStream in a BufferedOutputStream:

int bufferSize = 8 * 1024;
FileOutputStream output = new BufferedOutputStream(
                      new FileOutputStream("c:\\data\\output-file.txt"),
                          bufferSize
);

flush()

When you write data to a Java FileOutputStream the data may get cached internally in the memory of the computer and written to disk at a later time. For instance, every time there is X amount of data to write, or when the FileOutputStream is closed.

If you want to make sure that all written data is written to disk without having to close the FileOutputStream you can call its flush() method. Calling flush() will make sure that all data which has been written to the FileOutputStream so far, is fully written to disk too. Here is an example of calling the Java FileOutputStream flush() method:

OutputStream outputStream = new FileOutputStream("c:\\data\\output-text.txt");

byte bytes =  new byte[]{1,2,3,4,5};

outputStream.write(bytes);

outputStream.flush()

Close OutputStream

Close OutputStream when you have been finished your job is necessary.

Here is the simple sample.

output.close();

Convert FileOutputStream to Writer

The Java FileOutputStream is a byte based stream. You can convert a FileOutputStream to a character based Writer using the Java OutputStreamWriter class. Here is an example of converting a Java FileOutputStream to a Writer using OutputStreamWriter:

FileOutputStream outputStream       = new FileOutputStream("c:\\data\\output.txt");
Writer       outputStreamWriter = new OutputStreamWriter(outputStream);

outputStreamWriter.write("Hello World");

RandomAccessFile

The Java RandomAccessFile class in the Java IO API allows you to move navigate a file and read from it or write to it as you please. You can replace existing parts of a file too. This is not possible with the FileInputStream or FileOutputStream.

Creating a RandomAccessFile

Before you can work with the RandomAccessFile class you must instantiate it. Here is how that looks:

RandomAccessFile file = new RandomAccessFile("c:\\data\\file.txt", "rw");

Notice the second input parameter to the constructor: “rw”. This is the mode you want to open file in. “rw” means read / write mode. The different access modes supported by the Java RandomAccessFile is covered in the next section.

Access Modes

Mode Description
r Read mode. Calling write methods will result in an IOException.
rw Read and write mode.
rwd Read and write mode - synchronously. All updates to file content is written to the disk synchronously.
rws Read and write mode - synchronously. All updates to file content or meta data is written to the disk synchronously.

to know more detail please see this website: http://tutorials.jenkov.com/java-io/randomaccessfile.html

File

The Java File class only gives you access to the file and directory meta data. If you need to read or write the content of files, you should do so using either FileInputStream, FileOutputStream or RandomAccessFile.

Create a File

Before you can do anything with the file system or File class, you must create a Java File instance. Here is an example of creating a Java File instance:

File file = new File("c:\\data\\input-file.txt");

Check if File or Directory Exists

You can check if a file referenced by a Java File object exists using the File exists() method. Here is an example of checking if a file exists:

File file = new File("c:\\data\\input-file.txt");

boolean fileExists = file.exists();

The above code also works for directories. The only change you need to make to check if a directory exists is to pass a file system path to a directory to the Java File constructor, intead of a path to a file. Here is an example of checking if a directory exists:

File file = new File("c:\\data");

boolean fileExists = file.exists();

Create a Directory if it Does Not Exist

You can use the Java File class to create directories if they don’t already exists. The File class contains the method mkdir() and mkdirs() for that purpose.

The mkdir() method creates a single directory if it does not already exist. Here is an example of creating a single directory via the Java File class:

File file = new File("c:\\users\\jakobjenkov\\newdir");

boolean dirCreated = file.mkdir();

Provided that the directory c:\users\jakobjenkov already exists, the above code will create a subdirectory of jakobjenkov named newdir. The mkdir() returns true if the directory was created, and false if not.

The mkdirs() will create all directories that are missing in the path the File object represents. Here is an example of creating multiple directories via the Java File class:

File file = new File("c:\\users\\jakobjenkov\\newdir");

boolean dirCreated = file.mkdirs();

Provided that the C drive exists, this example will create all the directories in the path c:\users\jakobjenkov\newdir. The mkdirs() method will return true if all the directories were created, and false if not.

File Length

The Java File class enables you to read the length in bytes of a file. To read the length of a file, call the File length() method. Here is an example of reading the length of a file via the Java File length() method:

File file = new File("c:\\data\\input-file.txt");

long length = file.length();

Rename or Move File or Directory

To rename (or move) a file, call the method renameTo() on the File class. Here is a simple example:

File file = new File("c:\\data\\input-file.txt");

boolean success = file.renameTo(new File("c:\\data\\new-file.txt"));

As briefly mentioned earlier, the renameTo() method can also be used to move a file to a different directory. The new file name passed to the renameTo() method does not have to be in the same directory as the file was already residing in.

The renameTo() method returns boolean (true or false), indicating whether the renaming was successful. Renaming of moving a file may fail for various reasons, like the file being open, wrong file permissions etc.

The Java File renameTo() method also works for directories, by the way. Just pass a path to a directory to the File constructor, instead of a path to a file.

Delete File or Directory

To delete a file call the Java File delete() method. Here is a simple example:

File file = new File("c:\\data\\input-file.txt");

boolean success = file.delete();

The delete() method returns boolean (true or false), indicating whether the deletion was successful. Deleting a file may fail for various reasons, like the file being open, wrong file permissions etc.

The Java File delete() method also works for directories, meaning you can also delete directories with it.

Delete Directory and Subdirectories Recursively

The Java File delete() method can only delete a directory if the directory is empty. To delete a directory that contains files and subdirectories you must iterate through the directory and delete all files and subdirectories first, before you can delete the root directory. This iteration has to be carried out recursively, so you also delete all content of subdirectories and their subdirectories. Otherwise the deletion of the root directory will fail. Here is a Java method that can delete a directory and all its subdirectories and their files recursively:

public static boolean deleteDir(File dir){
    File[] files = dir.listFiles();
    if(files != null){
        for(File file : files){
            if(file.isDirectory()){
                deleteDir(file);
            } else {
                file.delete();
            }
        }
    }
    return dir.delete();
}

Read List of Files in Directory

You can obtain a list of all the files in a directory by calling either the Java File list() method or the listFiles() method. The list() method returns an array of String’s with the file and / or directory names of directory the File object points to. The listFiles() returns an array of File objects representing the files and / or directories in the directory the File points to.

Here is an example of listing all files in a directory via the Java File list() and listFiles() methods:

File file = new File("c:\\data");

String[] fileNames = file.list();

File[]   files = file.listFiles();

PipeInputStream

The PipedInputStream class makes it possible to read the contents of a pipe as a stream of bytes. Pipes are communication channels between threads inside the same JVM. Pipes are explained in more detail in my tutorial about Java IO Pipes.

PipeInputStream Example

InputStream input = new PipedInputStream(pipedOutputStream);

int data = input.read();
while(data != -1) {
  //do something with data...
  doSomethingWithData(data);

  data = input.read();
}
input.close();

PipeOutputStream

PipeOutputStream is likely to PipeInputStream.

ByteArrayInputStream

The Java ByteArrayInputStream class, java.io.ByteArrayInputStream, of the Java IO API enables you to read data from byte arrays as streams of bytes. In other words, the ByteArrayInputStream class can turn a byte array into an InputStream. The ByteArrayInputStream class is a subclass of the InputStream class, so you can use a ByteArrayInputStream as an InputStream. The ByteArrayInputStream also has a set of additional methods that are specific to the ByteArrayInputStream class. I will cover some of these methods in this tutorial.

The Java ByteArrayInputStream can be handy if your data is stored in an array, but you have a component that can only process it as an InputStream. The ByteArrayInputStream can thus wrap the byte array, and turn it into a stream.

Create a ByteArrayInputStream

To use a Java ByteArrayInputStream you must first create an instance of the ByteArrayInputStream class. To the constructor you pass the byte array you want to read as an InputStream. Here is an example of creating a ByteArrayInputStream instance:

byte[] bytes = ... //get byte array from somewhere.

ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);

This example creates a ByteArrayInputStream which can read all bytes in the byte array passed to its constructor.

You can also tell the ByteArrayInputStream to only read part of the given byte array. You can pass an extra offset and length to the constructor which specifies which section of the byte array to read. Here is how that looks:

byte[] bytes = ... //get byte array from somewhere.

int offset = 20;
int length = 45;

ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes, offset, length);

reset()

The reset() method of the ByteArrayInputStream resets how far it has read into the byte array. The index will be reset back to the last mark set on the ByteArrayInputStream. By default, if no mark has been explicitly set, the ByteArrayInputStream has marked position 0, or the position at the offset passed to its constructor. Here is an example of using the ByteArrayInputStream reset() method:

byte[] bytes = "abcdef".getBytes();

ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(bytes);

int data = byteArrayInputStream.read();  // read 'a'
    data = byteArrayInputStream.read();  // read 'b'
    data = byteArrayInputStream.read();  // read 'c'

    byteArrayInputStream.mark(1024);     // mark set before reading 'd'
    data = byteArrayInputStream.read();  // read 'd'
    data = byteArrayInputStream.read();  // read 'e'
    data = byteArrayInputStream.read();  // read 'f'

    byteArrayInputStream.reset();        // reset to mark before 'd'
    data = byteArrayInputStream.read();  // read 'd'
    data = byteArrayInputStream.read();  // read 'e'
    data = byteArrayInputStream.read();  // read 'f'

skip()

The Java ByteArrayInputStream skip() method enables you to skip over a number of bytes from the underlying byte array. You pass as parameter the number of characters you want to skip over. Here is an example of skipping over a number of bytes using the ByteArrayInputStream skip() method:

BufferedInputStream

The Java BufferedInputStream class, java.io.BufferedInputStream, provides transparent reading of chunks of bytes and buffering for a Java InputStream, including any subclasses of InputStream. Reading larger chunks of bytes and buffering them can speed up IO quite a bit. Rather than read one byte at a time from the network or disk, the BufferedInputStream reads a larger block at a time into an internal buffer. When you read a byte from the Java BufferedInputStream you are therefore reading it from its internal buffer. When the buffer is fully read, the BufferedInputStream reads another larger block of data into the buffer. This is typically much faster than reading a single byte at a time from an InputStream, especially for disk access and larger data amounts.