自从第一个Java版本开始,很多开发人员一直都在尝试让Java获得最少和C/C++一样的表现。JVM提供商尽他们最大的努力去实现一些新的JIT算法,但是还是有很多需要做的,特别是在我们使用Java的方法上。
例如,在对象<->文件序列化上就差距很大--尤其在读写内存对象上。我将就这个主题做一些解释和分享。
所有的测试都是在下面这个对象上执行的:
1 | public class TestObject implements Serializable { |
3 | private long longVariable; |
4 | private long [] longArray; |
5 | private String stringObject; |
6 | private String secondStringObject; |
为了简单起见,我将只贴出写入方法(尽管读取类似),完整的源码在我的GitHub上可以找到(http://github.com/jkubrynski/serialization-tests)
最标准的java序列化(我们都是从这里学起的)是这样的:
01 | public void testWriteBuffered(TestObject test, String fileName) throws IOException { |
02 | ObjectOutputStream objectOutputStream = null ; |
04 | FileOutputStream fos = new FileOutputStream(fileName); |
05 | BufferedOutputStream bos = new BufferedOutputStream(fos); |
06 | objectOutputStream = new ObjectOutputStream(bos); |
07 | objectOutputStream.writeObject(test); |
09 | if (objectOutputStream != null ) { |
10 | objectOutputStream.close(); |
提升标准序列化速度的最简单方法时使用RandomAccessFile对象:
01 | public void testWriteBuffered(TestObject test, String fileName) throws IOException { |
02 | ObjectOutputStream objectOutputStream = null ; |
04 | RandomAccessFile raf = new RandomAccessFile(fileName, "rw" ); |
05 | FileOutputStream fos = new FileOutputStream(raf.getFD()); |
06 | objectOutputStream = new ObjectOutputStream(fos); |
07 | objectOutputStream.writeObject(test); |
09 | if (objectOutputStream != null ) { |
10 | objectOutputStream.close(); |
更高深点的技术是使用Kryo框架,新旧版本的差距是很大的,我做过测试。因为性能比较上并没有体现出特别引人注意的差异,所以我将使用2.x版本,因为它对用户更友好而且更快些。
01 | private static Kryo kryo = new Kryo(); |
03 | public void testWriteBuffered(TestObject test, String fileName) throws IOException { |
06 | RandomAccessFile raf = new RandomAccessFile(fileName, "rw" ); |
07 | output = new Output( new FileOutputStream(raf.getFD()), MAX_BUFFER_SIZE); |
08 | kryo.writeObject(output, test); |
最后一个方案是在Martin Thompson的文章中提到的(Native C/C++ Like Performance For Java Object Serialisation),介绍了怎样在Java中像C++那样和内存打交道。
01 | public void testWriteBuffered(TestObject test, String fileName) throws IOException { |
02 | RandomAccessFile raf = null ; |
04 | MemoryBuffer memoryBuffer = new MemoryBuffer(MAX_BUFFER_SIZE); |
05 | raf = new RandomAccessFile(fileName, "rw" ); |
06 | test.write(memoryBuffer); |
07 | raf.write(memoryBuffer.getBuffer()); |
08 | } catch (IOException e) { |
TestObject写入方法如下:
01 | public void write(MemoryBuffer unsafeBuffer) { |
02 | unsafeBuffer.putLong(longVariable); |
03 | unsafeBuffer.putLongArray(longArray); |
05 | boolean objectExists = stringObject != null ; |
06 | unsafeBuffer.putBoolean(objectExists); |
08 | unsafeBuffer.putCharArray(stringObject.toCharArray()); |
10 | objectExists = secondStringObject != null ; |
11 | unsafeBuffer.putBoolean(objectExists); |
13 | unsafeBuffer.putCharArray(secondStringObject.toCharArray()); |
直接内存缓冲区类(已简化了的,仅仅为了展示这个思想)
01 | public class MemoryBuffer { |
03 | public static final Unsafe unsafe = UnsafeUtil.getUnsafe(); |
05 | private final byte [] buffer; |
07 | private static final long byteArrayOffset = unsafe.arrayBaseOffset( byte []. class ); |
08 | private static final long longArrayOffset = unsafe.arrayBaseOffset( long []. class ); |
几个小时的Caliper测试结果如下:
| Full trip [ns] | Standard deviation [ns] |
Standard | 207307 | 2362 |
Standard on RAF | 42661 | 733 |
KRYO 1.x | 12027 | 112 |
KRYO 2.x | 11479 | 259 |
Unsafe | 8554 | 91 |
在最后我们可以得出一些结论:
- Unsafe序列化比标准的java.io.Serizlizable快了23倍
- 使用RandomAccessFile可以使标准的有缓冲序列化加速将近4倍
- Kryo-dynamic序列化大约比手写实现的直接缓冲满了35%
最后,就像我们看到的那样,还是没有绝对的答案。对于我们中的大多数人来说,获得3000ns(0.003ms)的速度提升是不值得为每个需要序列化的对象来写单独实现的。在标准的方案中,我们大多数选择Kryo 。然而,在惜时如金的低延时系统中,这个选择将会是完全不同的。
kryo实例:
1. 依赖库
kryo-1.03.jar
2. 代码
public class Student {
private String name;
private int age;
public Student(){
}
public Student(String name, int age) {
super();
this.name = name;
this.age = age;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public int getAge() {
return age;
}
public void setAge(int age) {
this.age = age;
}
}
public class Grade {
public String name = "wang";
public int f1 = 100;
}
public enum Color {
RED, GREEN, BLUE
}
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.serialize.ArraySerializer;
import com.esotericsoftware.kryo.serialize.CollectionSerializer;
import com.esotericsoftware.kryo.serialize.EnumSerializer;
import com.esotericsoftware.kryo.serialize.MapSerializer;
public class BinaryTest {
public static void main(String[] args) {
Kryo k = new Kryo();
k.register(Student.class);
ByteBuffer buf = ByteBuffer.allocate(1024);
k.writeObject(buf, new Student("wang", 20));
byte[] serialValue = readBuf(buf);
Student student = k.readObject(ByteBuffer.wrap(serialValue), Student.class);
System.out.println(student.getName());
System.out.println(student.getAge());
//数组
buf.clear();
k.register(String[].class, new ArraySerializer(k));
String[] arr1 = {"111", "222"};
k.writeObject(buf, arr1);
serialValue = readBuf(buf);
String[] arr2 = k.readObject(ByteBuffer.wrap(serialValue), String[].class);
System.out.println(arr2[0]);
//集合
buf.clear();
k.register(ArrayList.class, new CollectionSerializer(k));
List<Student> list = new ArrayList<Student>();
list.add(new Student("wang", 20));
list.add(new Student("wang", 20));
k.writeObject(buf, list);
serialValue = readBuf(buf);
List<Student> list2 = k.readObject(ByteBuffer.wrap(serialValue), ArrayList.class);
System.out.println(list.get(0).getName());
//MAP
buf.clear();
k.register(HashMap.class, new MapSerializer(k));
Map<Integer, String> map1 = new HashMap<Integer, String>();
map1.put(1, "11");
map1.put(2, "22");
k.writeObject(buf, map1);
serialValue = readBuf(buf);
Map<Integer, String> map2 = k.readObject(ByteBuffer.wrap(serialValue), HashMap.class);
System.out.println(map2.get(1));
//枚举
buf.clear();
k.register(Color.class, new EnumSerializer(Color.class));
Color color = Color.RED;
k.writeObject(buf, color);
serialValue = readBuf(buf);
Color color2 = k.readObject(ByteBuffer.wrap(serialValue), Color.class);
System.out.println(color2);
buf.clear();
k.register(Grade.class);
k.writeObject(buf, new Grade());
serialValue = readBuf(buf);
Grade g = k.readObject(ByteBuffer.wrap(serialValue), Grade.class);
System.out.println("g.name = " + g.name);
System.out.println(g.f1);
}
public static byte[] readBuf(ByteBuffer buf){
int size = buf.position();
byte[] newBuf = new byte[size];
for(int i=0; i<size; i++){
newBuf[i] = buf.get(i);
}
return newBuf;
}
}