org.apache.flume.ChannelException: Take list for MemoryTransaction, capacity 100 full, consider comm...

flume在抽取MySQL数据到kafka时报错，如下

[SinkRunner-PollingRunner-DefaultSinkProcessor] ERROR org.apache.flume.sink.kafka.KafkaSink - Failed to publish events
org.apache.flume.ChannelException: Take list for MemoryTransaction, capacity 100 full, consider committing more frequently, increasing capacity, or increasing thread count
    at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doTake(MemoryChannel.java:96)
    at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
    at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
    at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:161)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
    at java.lang.Thread.run(Thread.java:745)
[SinkRunner-PollingRunner-DefaultSinkProcessor] ERROR org.apache.flume.SinkRunner - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to publish events
    at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:264)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.flume.ChannelException: Take list for MemoryTransaction, capacity 100 full, consider committing more frequently, increasing capacity, or increasing thread count
    at org.apache.flume.channel.MemoryChannel$MemoryTransaction.doTake(MemoryChannel.java:96)
    at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
    at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
    at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:161)
    ... 3 more
    

MemoryChannel的事务的Put队列满
解决:将channel的capacity的值调大即可

a1.channels.ch-1.capacity = 10000

注意：Flume配置每行后边不能加注释，否则会导致配置不生效

转载于:https://www.cnblogs.com/EnzoDin/p/10156485.html