Kryo 学习总结

 kryo是一个序列化框架

InstantiatorStrategy即初始化策略,默认kryo在反序列化对象时需要对象的类有一个零参数构造器,该构造器可以是private的,kryo通过反射调用该构造器来实例化对象。如果没有这样一个构造器,就需要使用kryo.setInstantiatorStrategy(new StdInstantiatorStrategy())了,该策略通过jvm api创建对象,这会创建一个完全空的对象(即不执行任何代码中的初始化工作),如果对象在实例化时需要一些初始化操作(比如在构造代码块中执行一些计算逻辑),这种策略就不可行了。
比较好的策略是kryo.setInstantiatorStrategy(new Kryo.DefaultInstantiatorStrategy(new StdInstantiatorStrategy()));,即先尝试通过零参数构造实例化对象,如果类中没有零参数构造器,则会使用StdInstantiatorStrategy策略。
 
References即引用,对A对象序列化时,默认情况下kryo会在每个成员对象第一次序列化时写入一个数字,该数字逻辑上就代表了对该成员对象的引用,如果后续有引用指向该成员对象,则直接序列化之前存入的数字即可,而不需要再次序列化对象本身。这种默认策略对于成员存在互相引用的情况较有利,否则就会造成空间浪费(因为没序列化一个成员对象,都多序列化一个数字),通常情况下可以将该策略关闭,kryo.setReferences(false);
 
Registration即注册,kryo在序列化对象时,首先会序列化其类的全限定名,由于我们通常序列化的对象都是有限范围内的类的实例,这样重复序列化同样的类的全限定名是低效的。通过注册kryo可以将类的全限定名抽象为一个数字,即用一个数字代表全限定名,这样就要高效一些。kryo.register(SomeClass.class);,注册方法的完整签名为public Registration register (Class type, Serializer serializer, int id),我们通常只需要使用其重载方法即可public Registration register (Class type),serializer和id在kryo内部会指定。
 
PS:使用kryo序列化时,可以使用transient关键字忽略某字段。


序列化方法

kryo大体有三种序列化方法,每种方式都有其优势和劣势。
1,kryo.writeObject,这种方法只会序列化对象实例,而不会记录对象所属类的任何信息。优势是最节省空间,劣势是在反序列化时,需要提供类作为模板才能顺利反序列。反序列化时使用readObject。
2,直接kryo.writeClassAndObject,这种方法会先将对象所属类的全限定名序列化,然后再依次序列化对象实例的成员。优势是完全动态序列化,整个kryo周期都不需要提供类信息。反序列化时使用readClassAndObject
3,先注册,再kryo.writeClassAndObject,这种方式时最理想的,其结合了前两种优势,又有效规避了劣势,事先将需要序列化的类注册给kryo(此时类和唯一id绑定),之后使用writeClassAndObject序列化时,只会序列化注册id,而不会序列化类的全限定名了,这样大大节省了空间(通常只比writeObject多一个字节)。反序列化时使用readClassAndObject。注意序列化和反序列的kryo的注册信息应当保持一致。

实例:

KryoUtils.java

package com.eastcom.second.stream.util;

import java.util.HashMap;
import java.util.LinkedHashMap;

import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.PooledObjectFactory;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.apache.commons.pool2.impl.SoftReferenceObjectPool;

import com.eastcom.second.stream.data.Radius;
import com.eastcom.second.stream.data.RadiusSerializer;
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import com.esotericsoftware.kryo.serializers.MapSerializer;

/**
 * 
 * KryoUtils为序列化类,使用了了对象池的“软引入对象池工厂”。
 * 
 * 用于序列化对象。
 * 
 * @author Administrator
 * 
 *
 */
public class KryoUtils implements PooledObjectFactory<Kryo> {

	public static class KryoResigerIds {

		public static final int HASH_MAP = 1001, LINKED_HASH_MAP = 1002, IMMUTABLE_PAIR = 1004, DATE = 1005,
				HASH_BASED_TABLE = 1006, RADIUS = 1101;

	}

	private final static SoftReferenceObjectPool<Kryo> kryoPool;

	static {

		PooledObjectFactory<Kryo> factory = new KryoUtils();

		kryoPool = new SoftReferenceObjectPool<Kryo>(factory);

	}

	public static Kryo create(boolean registrationRequired) {

		MapSerializer mapSerializer = new MapSerializer();

		Kryo kryo = new Kryo();

		kryo.register(HashMap.class, mapSerializer, KryoResigerIds.HASH_MAP);
		kryo.register(LinkedHashMap.class, mapSerializer, KryoResigerIds.LINKED_HASH_MAP);

		kryo.register(Radius.class, new RadiusSerializer(), KryoResigerIds.RADIUS);

		kryo.setReferences(false);
		kryo.setRegistrationRequired(registrationRequired);

		return kryo;
	}

	public static Kryo create() {

		return create(false);
	}

	public static void serialize(Kryo kryo, Output output, Object obj) {

		kryo.writeClassAndObject(output, obj);
	}

	public static byte[] serialize(Object obj) {
		try {
			Kryo kryo = kryoPool.borrowObject();
			byte[] bs = serialize(kryo, obj);
			kryoPool.returnObject(kryo);
			return bs;
		} catch (Exception e) {
			// TODO Auto-generated catch block
			throw new RuntimeException(e);
		}
	}

	public static byte[] serialize(Kryo kryo, Object obj) {
		Output output = new Output(50, Integer.MAX_VALUE);
		kryo.writeClassAndObject(output, obj);
		output.close();
		return output.toBytes();
	}

	public static <T> T deserialize(byte[] data) {

		try {
			Kryo kryo = kryoPool.borrowObject();
			T res = deserialize(kryo, data);
			kryoPool.returnObject(kryo);
			return res;
		} catch (Exception e) {
			throw new RuntimeException(e);
		}

	}

	public static <T> T deserialize(Kryo kryo, byte[] bs) {
		T obj = deserialize(kryo, new Input(bs));
		return obj;
	}

	public static <T> T deserialize(Kryo kryo, Input input) {
		@SuppressWarnings("unchecked")
		T obj = (T) kryo.readClassAndObject(input);
		return obj;

	}

	public void activateObject(PooledObject<Kryo> arg0) throws Exception {
		// TODO Auto-generated method stub

	}

	public void destroyObject(PooledObject<Kryo> arg0) throws Exception {
		// TODO Auto-generated method stub

	}

	public PooledObject<Kryo> makeObject() throws Exception {
		// TODO Auto-generated method stub
		Kryo kryo = create();

		return new DefaultPooledObject<Kryo>(kryo);
	}

	public void passivateObject(PooledObject<Kryo> arg0) throws Exception {
		// TODO Auto-generated method stub

	}

	public boolean validateObject(PooledObject<Kryo> arg0) {
		// TODO Auto-generated method stub
		return true;
	}

}


其中用到commons-pool2创建kryo对象池。采用了类注册false,但是已经有些采用了注册,并且为其设置了特定的数据。如果注册设置为true,则需要序列化的对象如果没有注册则会报错。已知的设置采用注册并且为设置数组编号,这样就不用全报名类名为标识此对象,提高 了序列化效率。

Radius.java

package com.eastcom.second.stream.data;

import java.io.Serializable;

import com.eastcom.second.stream.enums.RadiusType;

/**
 * radius 日志话单
 * 
 * @author Louyj
 *
 */
public class Radius implements Serializable {

	private static final long serialVersionUID = 1L;

	/*
	 * 1 报文时间
	 */
	private long datetime; // 时间戳类型

	/*
	 * 2 BRAS设备IP地址
	 */
	private String srcIp;

	/*
	 * 3 BRAS设备端口号
	 */
	private int srcPort;
	/*
	 * 4 日志提供厂家
	 */
	private String factory;

	/*
	 * 5 BRAS归属地市
	 */
	private String city;

	/*
	 * 6 报文类别
	 */
	private String code;

	/*
	 * 7 宽带账号
	 */
	private String userName;

	/*
	 * 8 NAS IP地址
	 */
	private String nasIpAddress;

	/*
	 * 9 用户终端MAC
	 */
	private String callingStationId;

	/*
	 * 10 BRAS设备名称
	 */
	private String nasIdentifier;

	/*
	 * 11 CVLAN
	 */
	private String vlanid;

	/*
	 * 12 SVLAN
	 */
	private String vlanid2;

	/*
	 * 13 NAS 端口号
	 */
	private int nasPortId;

	/*
	 * 14 上行带宽
	 */
	private long inputBasicRate;

	/*
	 * 15 下行带宽
	 */
	private long outputBasicRate;

	/*
	 * 16 计费报文识别标志
	 */
	private String acctStatusType;

	/*
	 * 17 上行流量
	 */
	private long acctInputOctets;

	/*
	 * 18 下行流量
	 */
	private long acctOutputOctets;

	/*
	 * 19 用户私网IP地址
	 */
	private String framedIpAddressl;

	/*
	 * 20 用户公网IP地址
	 */
	private String natIpAddress;

	/*
	 * 21 公网起始端口号
	 */
	private int natStartPort;

	/*
	 * 22 公网结束端口号
	 */
	private int natEndPort;

	private RadiusType type;

	public RadiusType getType() {
		return type;
	}

	public void setType(RadiusType type) {
		this.type = type;
	}

	public long getDatetime() {
		return datetime;
	}

	public void setDatetime(long datetime) {
		this.datetime = datetime;
	}

	public String getSrcIp() {
		return srcIp;
	}

	public void setSrcIp(String srcIp) {
		this.srcIp = srcIp;
	}

	public int getSrcPort() {
		return srcPort;
	}

	public void setSrcPort(int srcPort) {
		this.srcPort = srcPort;
	}

	public String getFactory() {
		return factory;
	}

	public void setFactory(String factory) {
		this.factory = factory;
	}

	public String getCity() {
		return city;
	}

	public void setCity(String city) {
		this.city = city;
	}

	public String getCode() {
		return code;
	}

	public void setCode(String code) {
		this.code = code;
	}

	public String getUserName() {
		return userName;
	}

	public void setUserName(String userName) {
		this.userName = userName;
	}

	public String getNasIpAddress() {
		return nasIpAddress;
	}

	public void setNasIpAddress(String nasIpAddress) {
		this.nasIpAddress = nasIpAddress;
	}

	public String getCallingStationId() {
		return callingStationId;
	}

	public void setCallingStationId(String callingStationId) {
		this.callingStationId = callingStationId;
	}

	public String getNasIdentifier() {
		return nasIdentifier;
	}

	public void setNasIdentifier(String nasIdentifier) {
		this.nasIdentifier = nasIdentifier;
	}

	public String getVlanid() {
		return vlanid;
	}

	public void setVlanid(String vlanid) {
		this.vlanid = vlanid;
	}

	public String getVlanid2() {
		return vlanid2;
	}

	public void setVlanid2(String vlanid2) {
		this.vlanid2 = vlanid2;
	}

	public int getNasPortId() {
		return nasPortId;
	}

	public void setNasPortId(int nasPortId) {
		this.nasPortId = nasPortId;
	}

	public long getInputBasicRate() {
		return inputBasicRate;
	}

	public void setInputBasicRate(long inputBasicRate) {
		this.inputBasicRate = inputBasicRate;
	}

	public long getOutputBasicRate() {
		return outputBasicRate;
	}

	public void setOutputBasicRate(long outputBasicRate) {
		this.outputBasicRate = outputBasicRate;
	}

	public String getAcctStatusType() {
		return acctStatusType;
	}

	public void setAcctStatusType(String acctStatusType) {
		this.acctStatusType = acctStatusType;
	}

	public long getAcctInputOctets() {
		return acctInputOctets;
	}

	public void setAcctInputOctets(long acctInputOctets) {
		this.acctInputOctets = acctInputOctets;
	}

	public void incrAcctInputOctets(long acctInputOctets) {
		this.acctInputOctets += acctInputOctets;
	}

	public long getAcctOutputOctets() {
		return acctOutputOctets;
	}

	public void setAcctOutputOctets(long acctOutputOctets) {
		this.acctOutputOctets = acctOutputOctets;
	}

	public void incrAcctOutputOctets(long acctOutputOctets) {
		this.acctOutputOctets += acctOutputOctets;
	}

	public String getFramedIpAddressl() {
		return framedIpAddressl;
	}

	public void setFramedIpAddressl(String framedIpAddressl) {
		this.framedIpAddressl = framedIpAddressl;
	}

	public String getNatIpAddress() {
		return natIpAddress;
	}

	public void setNatIpAddress(String natIpAddress) {
		this.natIpAddress = natIpAddress;
	}

	public int getNatStartPort() {
		return natStartPort;
	}

	public void setNatStartPort(int natStartPort) {
		this.natStartPort = natStartPort;
	}

	public int getNatEndPort() {
		return natEndPort;
	}

	public void setNatEndPort(int natEndPort) {
		this.natEndPort = natEndPort;
	}

	/**
	 * redis 中zset 排序的分 时间戳+过程ID
	 * 
	 * @return
	 */
	public double getSortScore() {
		return this.getDatetime() + this.getType().intValue() * 0.1;
	}

	@Override
	public String toString() {
		return "Radius [datetime=" + datetime + ", srcIp=" + srcIp + ", srcPort=" + srcPort + ", factory=" + factory
				+ ", city=" + city + ", code=" + code + ", userName=" + userName + ", nasIpAddress=" + nasIpAddress
				+ ", callingStationId=" + callingStationId + ", nasIdentifier=" + nasIdentifier + ", vlanid=" + vlanid
				+ ", vlanid2=" + vlanid2 + ", nasPortId=" + nasPortId + ", inputBasicRate=" + inputBasicRate
				+ ", outputBasicRate=" + outputBasicRate + ", acctStatusType=" + acctStatusType + ", acctInputOctets="
				+ acctInputOctets + ", acctOutputOctets=" + acctOutputOctets + ", framedIpAddressl=" + framedIpAddressl
				+ ", natIpAddress=" + natIpAddress + ", natStartPort=" + natStartPort + ", natEndPort=" + natEndPort
				+ ", type=" + type + "]";
	}

}


自定义序列化器RadiusSerializer.java

package com.eastcom.second.stream.data;

import com.eastcom.second.stream.enums.RadiusType;
import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.Serializer;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;

/**
 * 
 * 自定义序列化器,没有用到writeClassAndObject(obj) ,不需对 对象进行反射,这样可以提高序列化性能。
 * 
 * @author Administrator
 *
 */
public class RadiusSerializer extends Serializer<Radius> {

	public RadiusSerializer() {

		super(false, true);
	}

	@Override
	public void write(Kryo kryo, Output output, Radius object) {
		output.writeLong(object.getDatetime(), true);
		output.writeString(object.getSrcIp());
		output.writeInt(object.getSrcPort(), true);
		output.writeString(object.getFactory());
		output.writeString(object.getCity());
		output.writeString(object.getCode());
		output.writeString(object.getUserName());
		output.writeString(object.getNasIpAddress());
		output.writeString(object.getCallingStationId());
		output.writeString(object.getNasIdentifier());
		output.writeString(object.getVlanid());
		output.writeString(object.getVlanid2());
		output.writeInt(object.getNasPortId(), true);
		output.writeLong(object.getInputBasicRate(), true);
		output.writeLong(object.getOutputBasicRate(), true);
		output.writeString(object.getAcctStatusType());
		output.writeLong(object.getAcctInputOctets(), true);
		output.writeLong(object.getAcctOutputOctets(), true);
		output.writeString(object.getFramedIpAddressl());
		output.writeString(object.getNatIpAddress());
		output.writeInt(object.getNatStartPort(), true);
		output.writeInt(object.getNatEndPort(), true);

		output.writeString(object.getType().name());

		// TODO Auto-generated method stub

	}

	@Override
	public Radius read(Kryo kryo, Input input, Class<Radius> type) {
		Radius radius = new Radius();
		radius.setDatetime(input.readLong(true));
		radius.setSrcIp(input.readString());
		radius.setSrcPort(input.readInt(true));
		radius.setFactory(input.readString());
		radius.setCity(input.readString());
		radius.setCode(input.readString());
		radius.setUserName(input.readString());
		radius.setNasIpAddress(input.readString());
		radius.setCallingStationId(input.readString());
		radius.setNasIdentifier(input.readString());
		radius.setVlanid(input.readString());
		radius.setVlanid2(input.readString());
		radius.setNasPortId(input.readInt(true));
		radius.setInputBasicRate(input.readLong(true));
		radius.setOutputBasicRate(input.readLong(true));
		radius.setAcctStatusType(input.readString());
		radius.setAcctInputOctets(input.readLong(true));
		radius.setAcctOutputOctets(input.readLong(true));
		radius.setFramedIpAddressl(input.readString());
		radius.setNatIpAddress(input.readString());
		radius.setNatStartPort(input.readInt(true));
		radius.setNatEndPort(input.readInt(true));

		radius.setType(RadiusType.valueOf(input.readString()));
		return radius;
	}

}


测试
package com.eastcom.second.stream.data.parser;

import com.eastcom.second.stream.data.Radius;
import com.eastcom.second.stream.util.KryoUtils;

public class ParserTest {

	public static void main(String[] args) {
		// TODO Auto-generated method stub

		String line = "2017-09-25 14:15:00|221.131.204.223|1813||宁波|Accounting-Request|nbyza58239926|221.131.204.223|c8:3a:35:02:83:20|ZJNBO-MC-CMNET-BRAS66-YZZXME6|477|1024|4|12583000|20972000|Interim-Update|46655120|1127558565|10.121.105.238|39.190.249.213|30720|31743||10.212.68.16|1813|108|0";

		// csvParserTest(line);

		topicParserTest(line);

	}

	private static Object csvParserTest(String line) {

		RadiusCSVParser radiusCSVParser = new RadiusCSVParser();

		Radius radius = radiusCSVParser.parse(line);

		System.out.println(radius.toString());

		return radius;

	}

	private static Object topicParserTest(String line) {

		Radius radius2 = (Radius) TopicCSVParsers.parse("radius", line, null);

		System.out.println(radius2.toString());

		byte[] data = KryoUtils.serialize(radius2);

		System.out.println(data.length);

		Radius radius = KryoUtils.deserialize(data);

		System.out.println(radius.toString());
		return radius2;

	}

}


运行效果

Radius [datetime=1506320100000, srcIp=221.131.204.223, srcPort=1813, factory=, city=宁波, code=Accounting-Request, userName=nbyza58239926, nasIpAddress=221.131.204.223, callingStationId=c8:3a:35:02:83:20, nasIdentifier=ZJNBO-MC-CMNET-BRAS66-YZZXME6, vlanid=477, vlanid2=1024, nasPortId=4, inputBasicRate=12583000, outputBasicRate=20972000, acctStatusType=Interim-Update, acctInputOctets=46655120, acctOutputOctets=1127558565, framedIpAddressl=10.121.105.238, natIpAddress=39.190.249.213, natStartPort=30720, natEndPort=31743, type=ACCOUNTING]
208
Radius [datetime=1506320100000, srcIp=221.131.204.223, srcPort=1813, factory=, city=宁波, code=Accounting-Request, userName=nbyza58239926, nasIpAddress=221.131.204.223, callingStationId=c8:3a:35:02:83:20, nasIdentifier=ZJNBO-MC-CMNET-BRAS66-YZZXME6, vlanid=477, vlanid2=1024, nasPortId=4, inputBasicRate=12583000, outputBasicRate=20972000, acctStatusType=Interim-Update, acctInputOctets=46655120, acctOutputOctets=1127558565, framedIpAddressl=10.121.105.238, natIpAddress=39.190.249.213, natStartPort=30720, natEndPort=31743, type=ACCOUNTING]












评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值