Exception in thread "main" java.lang.RuntimeException: org.apache.storm.thrift.transport.TTransportException
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:256)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:311)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
at com.yiguan.topology.RtAppTopology.run(RtAppTopology.java:93)
at com.yiguan.topology.RtStart.control(RtStart.java:34)
at com.yiguan.topology.RtStart.main(RtStart.java:18)
Caused by: org.apache.storm.thrift.transport.TTransportException
at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at org.apache.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:294)
at org.apache.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:278)
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:244)
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:256)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:311)
at org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:157)
at com.yiguan.topology.RtAppTopology.run(RtAppTopology.java:93)
at com.yiguan.topology.RtStart.control(RtStart.java:34)
at com.yiguan.topology.RtStart.main(RtStart.java:18)
Caused by: org.apache.storm.thrift.transport.TTransportException
at org.apache.storm.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.storm.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
at org.apache.storm.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
at org.apache.storm.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at org.apache.storm.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:77)
at org.apache.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:294)
at org.apache.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:278)
at org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:244)
... 5 more
或者:
org.apache.thrift.transport.TTransportException: Frame size (17302738) larger than max length (16384000)
这个问题的原因是序列化后的topology对象过大导致的,通常可能是你在spout/bolt中创建了一个大对象(比如bitmap, 大数组等),导致序列化后对象的大小超过了thrift的max frame size(thrift中16384000这个值是写死的,只能调小不能调大)。在JStorm中,如果需要在spout/bolt中创建大对象,建议是在open/prepare方法中来做,延迟对象的创建时间。参见:
https://github.com/alibaba/jstorm/issues/230
在提交topology到集群之前是不能做大量数据加载。