MessagePack是最近一个比较热门与Json做比较的序列化工具,它的优点是相比于json,序列化速度更快和序列化之后的字节数组更小,正如它的官网https://msgpack.org/卖的广告所说
It’s like JSON.
but fast and small.
下面我们以三个方面来对MessagePack做一个初步的探险
一.What is MessagePack
1.以官网的阐述来表明:
MessagePack is an efficient binary serialization format. It lets you exchange data among multiple languages like JSON. But it’s faster and smaller. Small integers are encoded into a single byte, and typical short strings require only one extra byte in addition to the strings themselves.
中文翻译:
MessagePack是一种有效的二进制序列化格式。 它允许您在多种语言(如JSON)之间交换数据。 但它更快更小。 小整数被编码为单个字节,典型的短字符串除了字符串本身外只需要一个额外的字节。
可以看出它序列化出来能比json更小的原因是,整数会被编码成一个字节,另外json的字符串的冒号和引号也会被以一种特殊的方式进行压缩,下面的图片可以看出来MessagePack压缩之后的效果:
from:http://indiegamr.com/cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json/
2.验证Message的序列化速度和内容是否更优于JSON
package msgpack;
import com.alibaba.fastjson.JSON;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.msgpack.jackson.dataformat.MessagePackFactory;
import java.io.IOException;
public class MsgPackMain {
public static void main(String[] args) throws IOException {
Person person = new Person("andrew", "24", "man");
//fastjson
long jsonStartTime = System.currentTimeMillis();
String jsonStr = JSON.toJSONString(person);
System.out.println("json花费时间:" + (System.currentTimeMillis() - jsonStartTime));
System.out.println("json byteArr size is" + jsonStr.getBytes("UTF-8").length);
//msgPack use jackson
ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
long msgPackStartTime = System.currentTimeMillis();
byte[] msgPackByteArr = objectMapper.writeValueAsBytes(person);
System.out.println("msgPack花费时间:" + (System.currentTimeMillis() - msgPackStartTime));
System.out.println("msgPack byteArr size is :" + msgPackByteArr.length);
}
}
测试结果:
json花费时间:199
json byteArr size is40
msgPack花费时间:72
msgPack byteArr size is :28
从测试结果可以看出msgPack明显在速度和size方面都远胜于json,而且由于有jackson的依赖库,所以使用和替换jackson-json都非常简单
二.How to use
下面我们来做一些简单的demo:
引入需要依赖的maven库:
<dependency>
<groupId>org.msgpack</groupId>
<artifactId>jackson-dataformat-msgpack</artifactId>
<version>${msgpack.version}</version>
</dependency>
jackson包装的msgPack,使用非常简单,下面来封装简单的MsgPackUtil
package msgpack;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.msgpack.jackson.dataformat.MessagePackFactory;
import java.io.IOException;
import java.util.List;
import java.util.Map;
public class MsgPackUtil {
private static final ObjectMapper objectMapper = new ObjectMapper(new MessagePackFactory());
public static <T> T readObj(byte[] byteArr, Class<T> clazz) throws IOException {
return objectMapper.readValue(byteArr, clazz);
}
public static <T> byte[] writeObj(T t) throws JsonProcessingException {
return objectMapper.writeValueAsBytes(t);
}
public static <T> List<T> readList(byte[] listByteArr, TypeReference<List<T>> typeReference) throws IOException {
return objectMapper.readValue(listByteArr, typeReference);
}
public static <T,V> Map<T, V> readMap(byte[] mapByteArr, TypeReference<Map<T, V>> typeReference) throws IOException {
return objectMapper.readValue(mapByteArr, typeReference);
}
}
demo代码:
private static void msgPackJackSonDemo() throws IOException {
//Object
Person person = new Person("andrew", "24", "man");
byte[] personByteArr = MsgPackUtil.writeObj(person);
System.out.println("ObjByteArr size is :" + personByteArr.length);
//使用readValue时候,需要空的构造方法
System.out.println(MsgPackUtil.readObj(personByteArr, Person.class));
//list
List<String> testList = new ArrayList<>(3);
testList.add("hello");
testList.add("msgPack");
testList.add("java");
byte[] listByteArr = MsgPackUtil.writeObj(testList);
System.out.println("listByteArr size is :" + listByteArr.length);
//readList只支持POJO和基本类型List
System.out.println(JSON.toJSONString(MsgPackUtil.readList(listByteArr, new TypeReference<List<String>>() {})));
//map
Map<String, String> testMap = new HashMap<>(3);
testMap.put("hello", "hello");
testMap.put("msgPack", "msgPack");
testMap.put("java", "java");
byte[] mapByteArr = MsgPackUtil.writeObj(testMap);
System.out.println("mapByteArr size is :" + mapByteArr.length);
System.out.println(JSON.toJSONString(MsgPackUtil.readMap(mapByteArr, new TypeReference<Map<String, String>>() {})));
}
三.使用场景
1.适用于将数据存储在类似于mc和redis这类的nosql数据库中,因为数据可以序列化的更小更快,那么在网络上的传输消耗会减少,同时存在redis里面的内存占用也会减少。
2.适用于分布式框架里面多机之间的网络传输,生产上的实现:fluentd https://github.com/fluent/fluentd
3.适用于前后端数据的交互,当前后端完全分离之后,后端可以使用MsgPack代替json以获取更快的速度
四.引用的相关资料:
http://indiegamr.com/cut-your-data-exchange-traffic-by-up-to-50-with-one-line-of-code-msgpack-vs-json/
https://msgpack.org/
https://github.com/msgpack/msgpack-java/blob/develop/msgpack-jackson/README.md
written by 黄文岳
2018.09.02