问题一:
ERROR [PollableSourceRunner-KafkaSource-r2] (org.apache.flume.source.kafka.KafkaSource.doProcess:314) - KafkaSource EXCEPTION, {}
org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed due to group rebalance
这个错误提示比较直白,意思是消费者消费了数据,但在规定时间内没有commit,所以kafka认为这个consumer挂掉了,这时对consumer的group进行再平衡。
问题二:
ERROR [PollableSourceRunner-KafkaSource-r2] (org.apache.flume.source.kafka.KafkaSource.doProcess:314) - KafkaSource EXCEPTION, {}
org.apache.flume.ChannelFullException: The channel has reached it's capacity. This might be the result of a sink on the channel having too low of batch size, a downstream system running slower than normal, or t
hat the channel capacity is just too low. [channel=c3]
问题三:
ERROR [hdfs-k3-call-runner-1] (org.apache.flume.sink.hdfs.AbstractHDFSWriter.hflushOrSync:268