错误代码:
org.apache.storm.zookeeper.server.NIOServerCnxnFactory - Thread Thread[main,5,main] died
java.lang.RuntimeException: java.io.NotSerializableException: java.io.BufferedWriter
at backtype.storm.serialization.DefaultSerializationDelegate.serialize(DefaultSerializationDelegate.java:43) ~[storm-core-0.9.3.jar:0.9.3]
at backtype.storm.utils.Utils.serialize(Utils.java:85) ~[storm-core-0.9.3.jar:0.9.3
storm NotSerializableException统一解决方法:
不能在构造方法中初始化OutputStreamWriter(类似的对象),因为storm先初始化supervisor中的bolt对象,然后发送给worker,在调用prepare()方法
解决的办法就是,从supervisor发送到bolt这一步并不是初始化OutputStreamWriter类,可以先声明相关的对象,然后等发送到worker了以后,在bolt的prepare()里进行实例化,就可以避免此问题。下面是google来的一段话:
Thesupervisor instantiates the bolts, sends them to the workers, and then callsprepare() on all of them. Therefore, anything that isn’t serializable that isinstantiated before prepare() causes this process to fail
**这主要是由于bolt和spout里面自己定义了私人的成员变量,并且实例化了,导致构造方法中没法初始化
private person p = new person()
诸如这种现象
解决办法:
- 在spout和bolt的执行方法里new对象,但是不停调用NEW对象,会导致内存溢出,速度变慢,会变得超级慢,十几秒才2000次
- 解决办法二,先private成null,在prepare中初始化,速度提升N倍不止
- 解决办法二,先private成null,在prepare中初始化,速度提升N倍不止