java.util.ConcurrentModificationException异常分析

异常日志

2016-07-20 15:02:41,278 (New I/O  worker #197) [ERROR - org.apache.flume.source.AvroSource.appendBatch(AvroSource.java:388)] Avro source r1: Unable to process event batch. Exception follows.
java.util.ConcurrentModificationException
    at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
    at java.util.ArrayList$Itr.next(ArrayList.java:851)
    at org.apache.flume.channel.ChannelProcessor.processEventBatch(ChannelProcessor.java:157)
    at org.apache.flume.source.AvroSource.appendBatch(AvroSource.java:386)
    at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.apache.avro.ipc.specific.SpecificResponder.respond(SpecificResponder.java:91)
    at org.apache.avro.ipc.Responder.respond(Responder.java:151)
    at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.messageReceived(NettyServer.java:188)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.handleUpstream(NettyServer.java:173)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
    at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:786)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:458)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:439)
    at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:311)
    at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:558)
    at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:553)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
    at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
    at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:84)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:471)
    at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:332)
    at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:35)
    at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:102)
    at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

异常原因

首先我们可以看到异常是由ArrayList的checkForComodification抛出的。这个方法在遍历List调用next()方法的时候会被调用，用于检查List是否已经被修改了。
也就是说问题出在遍历的时候，List被修改了。
进一步检查代码，发现List<Channel> requiredChannels = new ArrayList<Channel>();作为成员变量被定义，而在运行的过程中，这个变量会被修改。

解决办法

• 将共享变量改为线程私有变量。这次遇到的情况，刚好是不需要定义为成员变量的情况，所以直接将成员变量改为方法局部变量，问题解决。根据实际情况来定，如果能用这种方案，当然是优先选择这种方案。
• 使用线程安全的类替换。由于ArrayList是一个非线程安全的类，可以采用线程安全的类进行替换，如CopyOnWriteArrayList。不能使用Vector，Vector在遍历的时候不是线程安全的。
• 使用加锁等同步方案。