原文地址:http://kdpeterson.net/blog/2009/03/easy-json-with-jackson.html
Posted 2009-03-19.
有时候一个随意的选择可能导致一个大问题。在我们开发社交媒体产品的早期,我们决定使用JSON格式将数据存储到
HDFS中。这个想法后来被证实非常棒。它的(JSON)性能非常好,当你需要查看一个文件对象时,读取它非常方便
,而且几乎每一种语言都有对应的JSON库文件,这让你能够在你所选择的语言中轻松进行流的写入操作。
但是,我们所选择的JSON库却令我们感到不如意。net.sf.json库相对于www.json.org上的代码来说,是一个非常大
的改进,但是这个库却有轻微的内存泄露问题。这个库使用一个基于ThreadLocal中HashMap保存对象,这使得循环
检测速度更快。但是,当我们试图通过同一个Mapper对象发送GB级别的数据时,这个库的性能就会变得非常糟糕—
—因为它会保存每个反序列化对象的引用(这个特点就会导致严重的性能问题:内存泄露)。
刚开始项目时,我们对Hadoop并不熟悉,因此,我们起初并没有意识到JSON库造成了内存泄露问题。我们以为这只
是一个暂时的性能问题。因此,当内存耗尽时,我们就增加更多的内存,但是内存泄露问题却仍然存在!最后,一个
堆的转存为我们指明了正确的方向,明确方向后,就很容易找到可疑代码所在了。
发现问题原因后(json-lib库导致的),我们才做起初就应该做的事情:向邮件列表求助(邮件列表指用户反馈清单)
。我们快速预览了Hadoop核心用户所发的邮件列表,发现其中有一个人,在使用json-lib库的时候,已经遇到过和我
们相同的问题(报告这个Bug之后,json-lib迅速进行了修复)。另外还有两个人推荐使用Jaskson JSON Processor(
来做对象的JSON序列化与反序列化)。相比 json-lib,Jackson 库更好用、更简单!
下面就是一个简单的例子,展示了如何利用Jackson JSON库读写Java对象:
public class JsonUtils {
private static final Log log = LogFactory.getLog(JsonUtils.class);
private static final ObjectMapper mapper = new ObjectMapper();
public static String jsonFromObject(Object object) {
StringWriter writer = new StringWriter();
try {
mapper.writeValue(writer, object);
} catch (RuntimeException e) {
throw e;
} catch (Exception e) {
log.error("Unable to serialize to json: " + object, e);
return null;
}
return writer.toString();
}
public static User userFromJson(String json) {
return (User) objectFromJson(json, User.class);
}
static T objectFromJson(String json, Class klass) {
T object;
try {
object = mapper.readValue(json, klass);
} catch (RuntimeException e) {
log.error("Runtime exception during deserializing "
+ klass.getSimpleName() + " from "
+ StringUtils.abbreviate(json, 80));
throw e;
} catch (Exception e) {
log.error("Exception during deserializing " + klass.getSimpleName()
+ " from " + StringUtils.abbreviate(json, 80));
return null;
}
return object;
}
}
如果你发现上面的代码很有用,请自由使用。这些代码是我在家里写的,相对于我工作中所写的代码来说,这是一个
改进(这是我第一次看到Jackson库时写的)。
另附上英文原文:
EasyJSON with Jackson
Posted 2009-03-19.
Sometimes a casual pick can lead to bigproblems. Early on in developing our new social media product we decided to useJSON for storing our data in HDFS. This idea has worked out really well for us.The performance is good, it's easy to read if you want to take a peak at a fileby hand, and there are libraries for every language under the sun, making iteasy to write streaming jobs in your language of choice.
What didn't work out so well for us was thelibrary we chose. The net.sf.json library is a big improvement over the codeit's based on from www.json.org, but it has minor memory leak, apparently it isusing a threadlocal hashmap of objects seen to make cycle detection faster.This fails miserably when we try to send gigabytes of data through the sameMapper object -- it holds a reference to every object deserialized.
Since we were unfamiliar with Hadoop whenwe started, we didn't identify the library as the cause of the problems. Weexpected some performance hiccups, and when we ran out of memory, we just gaveit more. But still the problems didn't go away. Finally a heap dump pointed usin the right direction, and once we knew what to look for, it was easy to spotthe suspect line of code.
At this point we did what we should havedone at the outset: ask the mailing list. A quick note to the core-users Hadoopmailing list gave us one person who had previously had trouble with json-lib(and got it fixed quickly after reporting the bug), and two who recommendedJaskson JSON Processor. Jackson is even easier to use than json-lib.
Here's somesample code that shows how to read and write objects.
public class JsonUtils {
private static final Log log = LogFactory.getLog(JsonUtils.class);
private static final ObjectMapper mapper = new ObjectMapper();
public static String jsonFromObject(Object object) {
StringWriter writer = new StringWriter();
try {
mapper.writeValue(writer, object);
} catch (RuntimeException e) {
throw e;
} catch (Exception e) {
log.error("Unable to serialize to json: " + object, e);
return null;
}
return writer.toString();
}
public static User userFromJson(String json) {
return (User) objectFromJson(json, User.class);
}
static T objectFromJson(String json, Class klass) {
T object;
try {
object = mapper.readValue(json, klass);
} catch (RuntimeException e) {
log.error("Runtime exception during deserializing "
+ klass.getSimpleName() + " from "
+ StringUtils.abbreviate(json, 80));
throw e;
} catch (Exception e) {
log.error("Exception during deserializing " + klass.getSimpleName()
+ " from " + StringUtils.abbreviate(json, 80));
return null;
}
return object;
}
}
Feelfree to use the above if you find it useful. It was written at home and I thinkis actually an improvement over what I wrote at work (the first time seeing thelibrary).