(5)关于Flume内存溢出的问题,此时会报各种莫名奇妙的异常,异常如下
< span style = "font-size:18px;" > org.apache.avro.AvroRuntimeException: Unknown datum type: java.lang.Exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 14/10/17 09:56:06 WARN ipc.NettyServer: Unexpected exception from downstream. java.lang.OutOfMemoryError: Java heap space 14/10/17 09:50:31 ERROR flume.SinkRunner: Unable to deliver event. Exception follows. java.lang.IllegalStateException: close() called when transaction is OPEN - you must either commit or rollback first Exception in thread "pool-17-thread-6" java.lang.NoClassDefFoundError: Could not initialize class java.text.MessageFormat EXCEPTION: java.lang.OutOfMemoryError: GC overhead limit exceeded) java.lang.OutOfMemoryError: GC overhead limit exceeded Error while writing to required channel: org.apache.flume.channel.MemoryChannel{name: AvroRuntimeException: Excessively large list allocation request detected: 1398022191 items! Connection closed. </ span >
这个问题害得我同事通宵到第二天10点了还没搞定,网上查了很多资料都是说设置JVM参数,我们同事还一起修改flume-en.sh里的
# Give Flume more memory and pre-allocate, enable remote monitoring via JMX #JAVA_OPTS="-Xms1024m -Xmx2048m -Dcom.sun.management.jmxremote" 这个参数,可是怎么改都不管用。都是没有明显的说出原因,还说修改什么limits.conf文件,什么解释都有很乱。
第二天我来上班接着排查:
解决办法:
<1>ps -aux|grep flume 查看刚才启动的flume进程,发现如下信息
< span style = "font-size:18px;" > /usr/jdk/bin/java -Xmx20m -Dflume.root.logger = INFO -cp conf </ span >
<2>于是看一个flume-ng的python脚本,发现
< span style = "font-size:18px;" > JAVA_OPTS = "-Xmx20m" </ span >
问题就是出在这里了,然后我把flume-ng脚本里的这个值调大后,一切运行正常了。
< span style = "font-size:18px;" > JAVA_OPTS = "-Xmx2048m" </ span >