java.io和java.nio性能简单对比

我从java1.3开始学习java，后来主要用1.4，再后来1.5和1.6中的很多新特性，都停留在“知道”的状态，比如nio，虽然据说可以提升性能，但并没有真正深入使用和测试过，工作操作文件的情况不多，所以关注也不多，即便用到，也还是习惯性的用java.io。 今天看到的这篇文章，虽然测试手段非常简单，所得结论也难免有些片面 ，但依然说明，在顺序访问的时候，NIO的性能相对java.io有很大的提升。

也许应该update一下自己的知识了，否则就要OUT，或者早已经OUT了。
下次操作文件或者写socket要用NIO了。

---
转自：http://links.techwebnewsletters.com/ctt?kn=28&m=34038811&r=MzI1Mjc3MDAzOAS2&b=0&j=NTc5NjM4MTAS1&mt=1&rt=0
以下为翻译的内容：

最近我在工作中用到了java i/o相关功能。因为对java.io的了解更多（毕竟面世较早），所以一开始我使用的是java.io包下的类，后来为了测试一下是不是能够通过NIO提高文件操作性能，于是转向了java.nio。我得到的结论让我感到有些震惊，下面是对比测试的一些细节：

   1、在java.io的测试代码中，我使用RandomAccessFile直接向文件写数据，并搜索到特定的位置执行记录的插入、读取和删除。
   2、在java.nio的初步测试代码中，使用FileChannel对象。NIO之所以比java.io更加高效，是因为NIO面向的是data chunks，而java.io基本上是面向byte的。
   3、为了进一步挖掘NIO的能力，我又改用MappedByteBuffer执行测试，这个类是构建在操作系统的虚拟内存机制上的。根据java文档所说，这个类在性能方面是最好的。



为了进行测试，我写了一个模拟员工数据库的小程序，员工数据的结构如下：

   

[java] view plain copy print ?

1. class Employee { 
2.         String last; // the key 
3.         String first; 
4.         int id; 
5.         int zip; 
6.         boolean employed; 
7.         String comments; 
8.     } 

class Employee { String last; // the key String first; int id; int zip; boolean employed; String comments; }

员工数据写入文件，并将last name作为索引key，日后可以通过这个key从文件中加载该员工对应的数据。无论使用IO、NIO还是MappedByteBuffers，首先都需要打开一个RandomAccessFile。以下代码在用户的home目录下创建一个名为employee.ejb的文件，设置为可读可写，并初始化对应的Channel和MappedByteBuffer：
   

[java] view plain copy print ?

1. String userHome = System.getProperty("user.home"); 
2.     StringBuffer pathname = new StringBuffer(userHome); 
3.     pathname.append(File.separator); 
4.     pathname.append("employees.ejb"); 
5.     java.io.RandomAccessFile journal = 
6.         new RandomAccessFile(pathname.toString(), "rw"); 
7.   
8.     //下面这一句是为了NIO 
9.     java.nio.channels.FileChannel channel = journal.getChannel(); 
10.     
11.     //下面这两句是为了使用MappedByteBuffer 
12.     journal.setLength(PAGE_SIZE); 
13.     MappedByteBuffer mbb = 
14.         channel.map(FileChannel.MapMode.READ_WRITE, 0, journal.length() ); 

String userHome = System.getProperty("user.home"); StringBuffer pathname = new StringBuffer(userHome); pathname.append(File.separator); pathname.append("employees.ejb"); java.io.RandomAccessFile journal = new RandomAccessFile(pathname.toString(), "rw"); //下面这一句是为了NIO java.nio.channels.FileChannel channel = journal.getChannel(); //下面这两句是为了使用MappedByteBuffer journal.setLength(PAGE_SIZE); MappedByteBuffer mbb = channel.map(FileChannel.MapMode.READ_WRITE, 0, journal.length() );


使用channel.map进行映射后，当该文件被追加了新的数据时，之前的MappedByteBuffer是看不到这些数据的。因为我们想测试读和写，所以当文件中追加写入新的记录后，需要重新做映射才能使得MappedByteBuffer读取新数据。为了提高效率，降低重新映射的次数，每次空间不够的时候，我们将文件扩张特定的大小（比如说1K）以防止每次追加新记录都要重新映射。


下面是写入员工记录的对比测试：

使用java.io的代码：
   

[java] view plain copy print ?

1. public boolean addRecord_IO(Employee emp) { 
2.         try { 
3.             byte[] last = emp.last.getBytes(); 
4.             byte[] first = emp.first.getBytes(); 
5.             byte[] comments = emp.comments.getBytes(); 
6.             
7.             // Just hard-code the sizes for perfomance 
8.             int size = 0; 
9.             size += emp.last.length(); 
10.             size += 4; // strlen - Integer 
11.             size += emp.first.length(); 
12.             size += 4; // strlen - Integer 
13.             size += 4; // emp.id - Integer 
14.             size += 4; // emp.zip - Integer 
15.             size += 1; // emp.employed - byte 
16.             size += emp.comments.length(); 
17.             size += 4; // strlen - Integer 
18.             long offset = getStorageLocation(size); 
19.             // 
20.             // Store the record by key and save the offset 
21.             // 
22.             if ( offset == -1 ) { 
23.                 // We need to add to the end of the journal. Seek there 
24.                 // now only if we're not already there 
25.                 long currentPos = journal.getFilePointer(); 
26.                 long jounralLen = journal.length(); 
27.                 if ( jounralLen != currentPos ) 
28.                     journal.seek(jounralLen); 
29.                     
30.                 offset = jounralLen; 
31.             }else { 
32.                 // Seek to the returned insertion point 
33.                 journal.seek(offset); 
34.             } 
35.             // Fist write the header 
36.             journal.writeByte(1); 
37.             journal.writeInt(size); 
38.             // Next write the data 
39.             journal.writeInt(last.length); 
40.             journal.write(last); 
41.             journal.writeInt(first.length); 
42.             journal.write(first); 
43.             journal.writeInt(emp.id); 
44.             journal.writeInt(emp.zip); 
45.             if ( emp.employed ) 
46.                 journal.writeByte(1); 
47.             else 
48.                 journal.writeByte(0); 
49.             journal.writeInt(comments.length); 
50.             journal.write(comments); 
51.             // Next, see if we need to append an empty record if we inserted 
52.             // this new record at an empty location 
53.             if ( newEmptyRecordSize != -1 ) { 
54.                 // Simply write a header 
55.                 journal.writeByte(0); //inactive record 
56.                 journal.writeLong(newEmptyRecordSize); 
57.             } 
58.             employeeIdx.put(emp.last, offset); 
59.             return true; 
60.         } 
61.         catch ( Exception e ) { 
62.             e.printStackTrace(); 
63.         } 
64.         return false; 
65.     } 

public boolean addRecord_IO(Employee emp) { try { byte[] last = emp.last.getBytes(); byte[] first = emp.first.getBytes(); byte[] comments = emp.comments.getBytes(); // Just hard-code the sizes for perfomance int size = 0; size += emp.last.length(); size += 4; // strlen - Integer size += emp.first.length(); size += 4; // strlen - Integer size += 4; // emp.id - Integer size += 4; // emp.zip - Integer size += 1; // emp.employed - byte size += emp.comments.length(); size += 4; // strlen - Integer long offset = getStorageLocation(size); // // Store the record by key and save the offset // if ( offset == -1 ) { // We need to add to the end of the journal. Seek there // now only if we're not already there long currentPos = journal.getFilePointer(); long jounralLen = journal.length(); if ( jounralLen != currentPos ) journal.seek(jounralLen); offset = jounralLen; }else { // Seek to the returned insertion point journal.seek(offset); } // Fist write the header journal.writeByte(1); journal.writeInt(size); // Next write the data journal.writeInt(last.length); journal.write(last); journal.writeInt(first.length); journal.write(first); journal.writeInt(emp.id); journal.writeInt(emp.zip); if ( emp.employed ) journal.writeByte(1); else journal.writeByte(0); journal.writeInt(comments.length); journal.write(comments); // Next, see if we need to append an empty record if we inserted // this new record at an empty location if ( newEmptyRecordSize != -1 ) { // Simply write a header journal.writeByte(0); //inactive record journal.writeLong(newEmptyRecordSize); } employeeIdx.put(emp.last, offset); return true; } catch ( Exception e ) { e.printStackTrace(); } return false; }


使用java.nio的代码：

   

[java] view plain copy print ?

1. public boolean addRecord_NIO(Employee emp) { 
2.         try { 
3.             data.clear(); 
4.             byte[] last = emp.last.getBytes(); 
5.             byte[] first = emp.first.getBytes(); 
6.             byte[] comments = emp.comments.getBytes(); 
7.             data.putInt(last.length); 
8.             data.put(last); 
9.             data.putInt(first.length); 
10.             data.put(first); 
11.             data.putInt(emp.id); 
12.             data.putInt(emp.zip); 
13.             byte employed = 0; 
14.             if ( emp.employed ) 
15.                 employed = 1; 
16.             data.put(employed); 
17.             data.putInt(comments.length); 
18.             data.put(comments); 
19.             data.flip(); 
20.             int dataLen = data.limit(); 
21.             header.clear(); 
22.             header.put((byte)1); // 1=active record 
23.             header.putInt(dataLen); 
24.             header.flip(); 
25.             long headerLen = header.limit(); 
26.             int length = (int)(headerLen + dataLen); 
27.             long offset = getStorageLocation((int)dataLen); 
28.             // 
29.             // Store the record by key and save the offset 
30.             // 
31.             if ( offset == -1 ) { 
32.                 // We need to add to the end of the journal. Seek there 
33.                 // now only if we're not already there 
34.                 long currentPos = channel.position(); 
35.                 long jounralLen = channel.size(); 
36.                 if ( jounralLen != currentPos ) 
37.                     channel.position(jounralLen); 
38.                 offset = jounralLen; 
39.             } 
40.             else { 
41.                 // Seek to the returned insertion point 
42.                 channel.position(offset); 
43.             } 
44.             // Fist write the header 
45.             long written = channel.write(srcs); 
46.             // Next, see if we need to append an empty record if we inserted 
47.             // this new record at an empty location 
48.             if ( newEmptyRecordSize != -1 ) { 
49.                 // Simply write a header 
50.                 data.clear(); 
51.                 data.put((byte)0); 
52.                 data.putInt(newEmptyRecordSize); 
53.                 data.flip(); 
54.                 channel.write(data); 
55.             } 
56.             employeeIdx.put(emp.last, offset); 
57.             return true; 
58.         } 
59.         catch ( Exception e ) { 
60.             e.printStackTrace(); 
61.         } 
62.         return false; 
63.     } 

public boolean addRecord_NIO(Employee emp) { try { data.clear(); byte[] last = emp.last.getBytes(); byte[] first = emp.first.getBytes(); byte[] comments = emp.comments.getBytes(); data.putInt(last.length); data.put(last); data.putInt(first.length); data.put(first); data.putInt(emp.id); data.putInt(emp.zip); byte employed = 0; if ( emp.employed ) employed = 1; data.put(employed); data.putInt(comments.length); data.put(comments); data.flip(); int dataLen = data.limit(); header.clear(); header.put((byte)1); // 1=active record header.putInt(dataLen); header.flip(); long headerLen = header.limit(); int length = (int)(headerLen + dataLen); long offset = getStorageLocation((int)dataLen); // // Store the record by key and save the offset // if ( offset == -1 ) { // We need to add to the end of the journal. Seek there // now only if we're not already there long currentPos = channel.position(); long jounralLen = channel.size(); if ( jounralLen != currentPos ) channel.position(jounralLen); offset = jounralLen; } else { // Seek to the returned insertion point channel.position(offset); } // Fist write the header long written = channel.write(srcs); // Next, see if we need to append an empty record if we inserted // this new record at an empty location if ( newEmptyRecordSize != -1 ) { // Simply write a header data.clear(); data.put((byte)0); data.putInt(newEmptyRecordSize); data.flip(); channel.write(data); } employeeIdx.put(emp.last, offset); return true; } catch ( Exception e ) { e.printStackTrace(); } return false; }

使用MappedByteBuffer的代码如下:

  

[java] view plain copy print ?

1. public boolean addRecord_MBB(Employee emp) { 
2.         try { 
3.             byte[] last = emp.last.getBytes(); 
4.             byte[] first = emp.first.getBytes(); 
5.             byte[] comments = emp.comments.getBytes(); 
6.             int datalen = last.length + first.length + comments.length + 12 + 9; 
7.             int headerlen = 5; 
8.             int length = headerlen + datalen; 
9.             // 
10.             // Store the record by key and save the offset 
11.             // 
12.             long offset = getStorageLocation(datalen); 
13.             if ( offset == -1 ) { 
14.                 // We need to add to the end of the journal. Seek there 
15.                 // now only if we're not already there 
16.                 long currentPos = mbb.position(); 
17.                 long journalLen = channel.size(); 
18.                 if ( (currentPos+length) >= journalLen ) { 
19.                     //log("GROWING FILE BY ANOTHER PAGE"); 
20.                     mbb.force(); 
21.                     journal.setLength(journalLen + PAGE_SIZE); 
22.                     channel = journal.getChannel(); 
23.                     journalLen = channel.size(); 
24.                     mbb = channel.map(FileChannel.MapMode.READ_WRITE, 0, journalLen); 
25.                     currentPos = mbb.position(); 
26.                 } 
27.                 if ( currentEnd != currentPos ) 
28.                     mbb.position(currentEnd); 
29.                 offset = currentEnd;//journalLen; 
30.             } 
31.             else { 
32.                 // Seek to the returned insertion point 
33.                 mbb.position((int)offset); 
34.             } 
35.             // write header 
36.             mbb.put((byte)1); // 1=active record 
37.             mbb.putInt(datalen); 
38.             // write data 
39.             mbb.putInt(last.length); 
40.             mbb.put(last); 
41.             mbb.putInt(first.length); 
42.             mbb.put(first); 
43.             mbb.putInt(emp.id); 
44.             mbb.putInt(emp.zip); 
45.             byte employed = 0; 
46.             if ( emp.employed ) 
47.                 employed = 1; 
48.             mbb.put(employed); 
49.             mbb.putInt(comments.length); 
50.             mbb.put(comments); 
51.             currentEnd += length; 
52.             // Next, see if we need to append an empty record if we inserted 
53.             // this new record at an empty location 
54.             if ( newEmptyRecordSize != -1 ) { 
55.                 // Simply write a header 
56.                 mbb.put((byte)0); 
57.                 mbb.putInt(newEmptyRecordSize); 
58.                 currentEnd += 5; 
59.             } 
60.             employeeIdx.put(emp.last, offset); 
61.             return true; 
62.         } 
63.         catch ( Exception e ) { 
64.             e.printStackTrace(); 
65.         } 
66.         return false; 
67.     } 

public boolean addRecord_MBB(Employee emp) { try { byte[] last = emp.last.getBytes(); byte[] first = emp.first.getBytes(); byte[] comments = emp.comments.getBytes(); int datalen = last.length + first.length + comments.length + 12 + 9; int headerlen = 5; int length = headerlen + datalen; // // Store the record by key and save the offset // long offset = getStorageLocation(datalen); if ( offset == -1 ) { // We need to add to the end of the journal. Seek there // now only if we're not already there long currentPos = mbb.position(); long journalLen = channel.size(); if ( (currentPos+length) >= journalLen ) { //log("GROWING FILE BY ANOTHER PAGE"); mbb.force(); journal.setLength(journalLen + PAGE_SIZE); channel = journal.getChannel(); journalLen = channel.size(); mbb = channel.map(FileChannel.MapMode.READ_WRITE, 0, journalLen); currentPos = mbb.position(); } if ( currentEnd != currentPos ) mbb.position(currentEnd); offset = currentEnd;//journalLen; } else { // Seek to the returned insertion point mbb.position((int)offset); } // write header mbb.put((byte)1); // 1=active record mbb.putInt(datalen); // write data mbb.putInt(last.length); mbb.put(last); mbb.putInt(first.length); mbb.put(first); mbb.putInt(emp.id); mbb.putInt(emp.zip); byte employed = 0; if ( emp.employed ) employed = 1; mbb.put(employed); mbb.putInt(comments.length); mbb.put(comments); currentEnd += length; // Next, see if we need to append an empty record if we inserted // this new record at an empty location if ( newEmptyRecordSize != -1 ) { // Simply write a header mbb.put((byte)0); mbb.putInt(newEmptyRecordSize); currentEnd += 5; } employeeIdx.put(emp.last, offset); return true; } catch ( Exception e ) { e.printStackTrace(); } return false; }


接下来，调用每种方法插入100,000条记录, 耗时对比如下:
    * With java.io: ~10,000 milliseconds
    * With java.nio: ~2,000 milliseconds
    * With MappedByteBuffer: ~970 milliseconds


使用NIO的性能改善效果非常明显，使用MappedByteBuffer的性能，更是让人吃惊。

使用三种方式读取数据的性能对比如下：
    * With java.io: ~6,900 milliseconds
    * With java.nio: ~1,400 milliseconds
    * With MappedByteBuffer: ~355 milliseconds

和写入的时候情况差不多，NIO有很明显的性能提升，而MappedByteBuffer则有惊人的高效率。从java.io迁移到nio并使用MappedByteBuffer，通常可以获得10倍以上的性能提升。