1.6 Hessian序列化
支持跨语言的二进制序列化协议,相对于Java默认序列化,具有更好的性能与易用性
1.6.1 核心类
AbstractSerializerFactory、AbstractHessianOutput、AbstcactSerializer、AbstractHessianInput、AbstractDeserializer,和本质输入输出的ByteArrayOutputStream、ByteArrayInputStream
1.6.2 抽取通用序列化/反序列化方法
/**
* @author pdc
*/
public class HessianSerializer implements ISerializer {
public byte[] serialize(Object obj) {
if (obj == null)
throw new NullPointerException();
try {
ByteArrayOutputStream os = new ByteArrayOutputStream();
HessianOutput ho = new HessianOutput(os);
ho.writeObject(obj);
return os.toByteArray();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public <T> T deserialize(byte[] data, Class<T> clazz) {
if (data == null) throw new NullPointerException();
try {
ByteArrayInputStream is = new ByteArrayInputStream(data);
HessianInput hi = new HessianInput(is);
return (T) hi.readObject();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
1.7 protobuf/protostuff序列化
1.7.1 protobuf
是Google的一种数据交换格式,独立于语言、平台;是一个纯粹的应用层协议,可以和各种运输层协议一起使用
优缺点:
1.7.1.1 开发步骤:
1.省略,根据自己的OS百度即可
2.编写addressbook.proto
3.执行命令,生成相关的序列化工具类:
./protoc --java_out=./ ./addressbook.proto
4.将生成的AddressBookProtos复制到工程里面,编写应用的序列化与反序列化工具:
/**
* @author pdc
*/
public class ProtoBufSerializer{
public static void main(String[] args) throws Exception {
//构建一个Person对象
AddressBookProtos.Person person = AddressBookProtos.Person
.newBuilder()
.setEmail("kongxuan@163.com")
.setId(10000)
.setName("kongxuan")
.addPhone(
AddressBookProtos.Person.PhoneNumber.newBuilder().setNumber("13300000000")
.setType(AddressBookProtos.Person.PhoneType.HOME).build()).build();
//序列化
System.out.println(person.toByteString());
System.out.println(Arrays.toString(person.toByteArray()));
// 反序列化方法一
AddressBookProtos.Person newPerson = AddressBookProtos.Person.parseFrom(person.toByteString());
System.out.println(newPerson);
// 反序列化方法二
newPerson = AddressBookProtos.Person.parseFrom(person.toByteArray());
System.out.println(newPerson);
}
}
在此基础上可以抽取出通用的序列化/反序列化方法:
/**
* @author pdc
*/
public class ProtoBufSerializer implements ISerializer {
public <T> byte[] serialize(T obj) {
try {
if (!(obj instanceof GeneratedMessageV3)) {
throw new UnsupportedOperationException("not supported obj type");
}
return (byte[]) MethodUtils.invokeMethod(obj, "toByteArray");
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
public <T> T deserialize(byte[] data, Class<T> cls) {
try {
if (!GeneratedMessageV3.class.isAssignableFrom(cls)) {
throw new UnsupportedOperationException("not supported obj type");
}
Object o = MethodUtils.invokeStaticMethod(cls, "getDefaultInstance");
return (T) MethodUtils.invokeMethod(o, "parseFrom", new Object[]{data});
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
}
1.7.2 protostuff
protobuf需要预先编写.proto IDL文件,再通过protobuf提供的编译器生成对应于各种语言的代码
但是对于Java来说,Java具有反射和动态代码生成的能力,这个预编译过程不是必需的,可以在代码执行时实现
protostuff就是实现此功能的框架,基于protobuf,实现了无须预编译就能对JavaBean进行序列化/反序列化能力
1.7.2.1 抽取通用序列化/反序列化方法
/**
* @author pdc
*/
public class ProtoStuffSerializer implements ISerializer {
private static Map<Class<?>, Schema<?>> cachedSchema = new ConcurrentHashMap<>();
/**
* 反序列化实例化对象时,使用Objenesis
*/
private static Objenesis objenesis = new ObjenesisStd(true);
/**
* 线程安全
* @param cls
* @param <T>
* @return
*/
private static <T> Schema<T> getSchema(Class<T> cls) {
Schema<T> schema = (Schema<T>) cachedSchema.get(cls);
if (schema == null) {
schema = RuntimeSchema.createFrom(cls);
cachedSchema.put(cls, schema);
}
return schema;
}
public <T> byte[] serialize(T obj) {
Class<T> cls = (Class<T>) obj.getClass();
LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
try {
Schema<T> schema = getSchema(cls);
return ProtostuffIOUtil.toByteArray(obj, schema, buffer);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
buffer.clear();
}
}
public <T> T deserialize(byte[] data, Class<T> cls) {
try {
//如果使用Java反射,则代码为
//T message = (T)cls.getConstructors()[0].newInstance();
//这种做法要求对象保留无参构造,且调用构造函数不会造成额外的作用
//影响了通用性和健壮性
T message = (T) objenesis.newInstance(cls);
Schema<T> schema = getSchema(cls);
ProtostuffIOUtil.mergeFrom(data, message, schema);
return message;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
1.8 Thrift序列化
虽有Thrift RPC框架,但也可以单独使用Thrift进行序列化/反序列化操作
1.8.1 开发步骤
1.编写数据结构的 .thrift文件
2.使用Thrift提供的代码生成工具,生成.thrift文件对应的Java类
3.使用TSerializer和TDeserializer对类对象进行序列化/反序列化操作
/**
* @author pdc
* User为自定义对象
*/
public class ThriftSerializer implements ISerializer {
public <T> byte[] serialize(T obj) {
try {
if (!(obj instanceof TBase)) {
throw new UnsupportedOperationException("not supported obj type");
}
TSerializer serializer = new TSerializer(new TCompactProtocol.Factory());
return serializer.serialize((TBase) obj);
} catch (TException e) {
throw new RuntimeException(e);
}
}
public <T> T deserialize(byte[] data, Class<T> cls) {
try {
if (!TBase.class.isAssignableFrom(cls)) {
throw new UnsupportedOperationException("not supported obj type");
}
//可以指定如下序列化协议
//TBinaryProtocol TJSONProtocol TCompactProtocol
TBase o = (TBase) cls.newInstance();
TDeserializer tDeserializer = new TDeserializer(new TCompactProtocol.Factory());
tDeserializer.deserialize(o, data);
return (T) o;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
1.9 Avro序列化
虽可以用作RPC框架,但也可以单独用来进行序列化/反序列化操作
1.9.1 主要不同点:
动态类型、无标记数据、不用手动分配的字段ID
1.9.2 特点
性能高、基本代码少、产出数据量精简
1.9.3 特性
1.9.4 序列化编码方式
1.二进制编码
性能更好,序列化后产生的码流更小
2.JSON编码
可读性好,适合在开发调试阶段使用
1.9.5 IDL
1.9.6 Schema
如果仅仅使用Avro的序列化/反序列化功能,则可以使用JSON格式来定义所需要序列化的数据结构Schama
定义文件以.avsc后缀结尾,JSON相关键如下:
1.9.7 序列化/反序列化实现
-
方式1
编写序列化类的Schame并生成对应的对象;使用此对象,序列化生成byte[],再反序列化生成新的对象
/**
* @author pdc
*/
public class AvroSerializer implements ISerializer {
public static void main(String[] args) throws IOException {
User userAvro = new User();
userAvro.setAge(28);
userAvro.setEmail("1142439493@qq.com");
userAvro.setName("pdc");
//1.先自动生成代码,再序列化方式
//1.1 序列化
DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
BinaryEncoder binaryEncoder = EncoderFactory.get().directBinaryEncoder(outputStream, null);
userDatumWriter.write(userAvro, binaryEncoder);
byte[] data = outputStream.toByteArray();
//1.2 反序列化
DatumReader<User> userDatumReader = new SpecificDatumReader<User>(User.class);
BinaryDecoder binaryDecoder = DecoderFactory.get().directBinaryDecoder(new ByteArrayInputStream(data), null);
User userAvroCopy = userDatumReader.read(new User(), binaryDecoder);
System.out.println("age:" + userAvroCopy.getAge() + " email:" + userAvroCopy.getEmail() + " name:" + userAvroCopy.getName());
}
}
-
方式2
直接使用Schema文件进行序列化与反序列化
/**
* @author pdc
*/
public class AvroSerializer implements ISerializer {
public static void main(String[] args) throws IOException {
//2.直接根据avsc文件进行序列化的操作方式
Schema schema = new Schema.Parser().parse(new File("src/main/avro/user.avsc"));
GenericRecord userAvro2 = new GenericData.Record(schema);
userAvro2.put("age", 18);
userAvro2.put("email", "1142439493@qq.com");
userAvro2.put("name", "pdc");
//2.1 序列化
DatumWriter<GenericRecord> datumWriter = new GenericDatumWriter<GenericRecord>(schema);
ByteArrayOutputStream outputStream2 = new ByteArrayOutputStream();
BinaryEncoder binaryEncoder2 = EncoderFactory.get().directBinaryEncoder(outputStream2, null);
datumWriter.write(userAvro2, binaryEncoder2);
byte[] data2 = outputStream2.toByteArray();
//2.2 反序列化
DatumReader<User> userDatumReader2 = new SpecificDatumReader<User>(User.class);
BinaryDecoder binaryDecoder2 = DecoderFactory.get().directBinaryDecoder(new ByteArrayInputStream(data2), null);
User userAvroCopy2 = userDatumReader2.read(new User(), binaryDecoder2);
System.out.println("age:" + userAvroCopy2.getAge() + " email:" + userAvroCopy2.getEmail() + " name:" + userAvroCopy2.getName());
}
}
-
方式3
将序列化的结果保存到文件中
/**
* @author pdc
*/
public class AvroSerializer implements ISerializer {
public static void main(String[] args) throws IOException {
//3.以文件的形式将序列化
//3.1 序列化
File file = new File("users.avro");
DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
dataFileWriter.create(userAvro.getSchema(), file);
dataFileWriter.append(userAvro);
dataFileWriter.append(userAvroCopy2);
dataFileWriter.close();
//3.2 从文件中反序列化
DataFileReader<User> dataFileReader = new DataFileReader<User>(file, userDatumReader);
User user = null;
while (dataFileReader.hasNext()) {
user = dataFileReader.next(user);
System.out.println("age:" + user.getAge() + " email:" + user.getEmail() + " name:" + user.getName());
}
}
}
1.9.8 抽取通用序列化/反序列化方法
/**
* @author pdc
*/
public class AvroSerializer implements ISerializer {
public static void main(String[] args) throws IOException {
byte[] data_1 = new AvroSerializer().serialize(userAvro);
User user1 = new AvroSerializer().deserialize(data_1, User.class);
System.out.println("age:" + user1.getAge() + " email:" + user1.getEmail() + " name:" + user1.getName());
}
public <T> byte[] serialize(T obj) {
try {
if (!(obj instanceof SpecificRecordBase)) {
throw new UnsupportedOperationException("not supported obj type");
}
DatumWriter userDatumWriter = new SpecificDatumWriter(obj.getClass());
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
BinaryEncoder binaryEncoder = EncoderFactory.get().directBinaryEncoder(outputStream, null);
userDatumWriter.write(obj, binaryEncoder);
return outputStream.toByteArray();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public <T> T deserialize(byte[] data, Class<T> clazz) {
try {
if (!SpecificRecordBase.class.isAssignableFrom(clazz)) {
throw new UnsupportedOperationException("not supported clazz type");
}
DatumReader userDatumReader = new SpecificDatumReader(clazz);
BinaryDecoder binaryDecoder = DecoderFactory.get().directBinaryDecoder(new ByteArrayInputStream(data), null);
return (T) userDatumReader.read(clazz.newInstance(), binaryDecoder);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
1.10 JBoss Marshalling序列化框架介绍
Java对象序列包,兼容Java原生的序列化机制,对Java原生序列化机制进行了优化,提升了性能
在保持跟Serializble接口兼容的同时增加了一些可调的参数和附加特性,可通过工厂类进行配置
是一个很好的替代原生Java序列化的框架
1.10.1 抽取通用序列化/反序列化方法
/**
* @author pdc
*/
public class MarshallingSerializer implements ISerializer {
final static MarshallingConfiguration configuration = new MarshallingConfiguration();
//获取序列化工厂对象,参数serial标识创建的是java序列化工厂对象
final static MarshallerFactory marshallerFactory = Marshalling.getProvidedMarshallerFactory("serial");
static {
configuration.setVersion(5);
}
public byte[] serialize(Object obj) {
final ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try {
final Marshaller marshaller = marshallerFactory.createMarshaller(configuration);
marshaller.start(Marshalling.createByteOutput(byteArrayOutputStream));
marshaller.writeObject(obj);
marshaller.finish();
} catch (IOException e) {
e.printStackTrace();
}
return byteArrayOutputStream.toByteArray();
}
public <T> T deserialize(byte[] data, Class<T> clazz) {
try {
ByteArrayInputStream byteArrayInputStream = new ByteArrayInputStream(data);
final Unmarshaller unmarshaller = marshallerFactory.createUnmarshaller(configuration);
unmarshaller.start(Marshalling.createByteInput(byteArrayInputStream));
Object object = unmarshaller.readObject();
unmarshaller.finish();
return (T) object;
} catch (Exception e) {
e.printStackTrace();
}
return null;
}
}
1.11 序列化框架的选型
1.技术层面考虑
2.其他层面考虑
3.选型建议: