Kafka报错：org.apache.flume.ChannelException: Take failed due to IO error [channel=c1]

2018-08-09 11:32:06,186 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.channel.file.LogFile$RandomReader.checkOut(LogFile.java:560)] Opening /home/bigdata/flume/data/log-2 for read, remaining number of file handles available for reads of this file is 50
2018-08-09 11:32:06,187 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:240)] Failed to publish events
org.apache.flume.ChannelException: Take failed due to IO error [channel=c1]
    at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:534)
    at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
    at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
    at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:158)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.EOFException
    at java.io.RandomAccessFile.readByte(RandomAccessFile.java:669)
    at org.apache.flume.channel.file.LogFile$RandomReader.get(LogFile.java:491)
    at org.apache.flume.channel.file.Log.get(Log.java:612)
    at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:531)
    ... 6 more
2018-08-09 11:32:06,187 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:158)] Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to publish events
    at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:252)
    at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
    at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
    at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flume.ChannelException: Take failed due to IO error [channel=c1]
    at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:534)
    at org.apache.flume.channel.BasicTransactionSemantics.take(BasicTransactionSemantics.java:113)
    at org.apache.flume.channel.BasicChannelSemantics.take(BasicChannelSemantics.java:95)
    at org.apache.flume.sink.kafka.KafkaSink.process(KafkaSink.java:158)
    ... 3 more
Caused by: java.io.EOFException
    at java.io.RandomAccessFile.readByte(RandomAccessFile.java:669)
    at org.apache.flume.channel.file.LogFile$RandomReader.get(LogFile.java:491)
    at org.apache.flume.channel.file.Log.get(Log.java:612)
    at org.apache.flume.channel.file.FileChannel$FileBackedTransaction.doTake(FileChannel.java:531)
    ... 6 more

我是多级flume采集日志信息，第一层flume的conf文件，我改了一个地址参数（dataDirs），就是第一层flume采集完日志需要保存数据的路径，然后就爆以上的错误，咋调都不行

（解决方法在conf文件下边）

以下是我flume的conf文件

**********************************************************************************

#部署到hadoopF

# /home/bigdata/flume/bin/flume-ng agent -n a1 -c /home/bigdataflume/conf -f /home/bigdata/flume/config/tailDir-avro.conf  -Dflume.root.logger=INFO,console
# nohup /opt/apps/flume/bin/flume-ng agent -n a1 -c /opt/apps/flume/conf -f /opt/apps/flume/config/tailDir-avro.conf 1>/dev/null 2>/dev/null &

a1.sources = r1
a1.channels = c1
a1.sinkgroups = g1
a1.sinks = k1 k2

#source
a1.sources.r1.type = TAILDIR
a1.sources.r1.channels = c1
##记录每个文件的采集位置（断点位置） #这个需要修改
a1.sources.r1.positionFile = /home/bigdata/flume/checkpoint/source/tail_position.json
a1.sources.r1.filegroups = f1
#这个需要修改(这里既可以是具体的文件，也可以是目录)
a1.sources.r1.filegroups.f1 = /home/bigdata/nginx/logs/access.log
#a1.sources.r1.filegroups.f2 = /opt/apps/nginx/logs2/*.log
#a1.sources.r1.fileHeader = true

#channel
a1.channels.c1.type = file
#这个需要修改（防止数据丢失）[更改路径报错解决不了，可以试着把这个路径下的文件删掉：慎用]
a1.channels.c1.checkpointDir = /home/bigdata/flume/checkpoint/channel
#这个需要修改(可以定义多个目录，目录和目录之间用逗号“,”隔开)（监控数据存储位置）
a1.channels.c1.dataDirs = /home/bigdata/flume/data
#规定管道只能存多少byte数据
a1.channels.c1.maxFileSize = 104857600
#规定管道只能存多少条数据
a1.channels.c1.capacity = 90000000
a1.channels.c1.keep-alive = 60

#sink 负载均衡器
a1.sinkgroups.g1.sinks = k1 k2
a1.sinkgroups.g1.processor.type = load_balance
a1.sinkgroups.g1.processor.backoff = true
#轮询
a1.sinkgroups.g1.processor.selector = round_robin
a1.sinkgroups.g1.processor.selector.maxTimeOut = 10000

#sink 1
a1.sinks.k1.type = avro
a1.sinks.k1.channel = c1
#每次发送多少条数据
a1.sinks.k1.batchSize = 1
#这个需要修改（绝对不能使用主机名）
a1.sinks.k1.hostname = 192.168.6.82
a1.sinks.k1.port = 9999

#sink 2
a1.sinks.k2.type = avro
a1.sinks.k2.channel = c1
a1.sinks.k2.batchSize = 1
#这个需要修改
a1.sinks.k2.hostname = 192.168.6.83
a1.sinks.k2.port = 9999
 

解决方法：

checkpointDir参数会保存flume的agent 进程channel每次储存数据的地址：

（a1.channels.c1.checkpointDir = /home/bigdata/flume/checkpoint/channel）

找到/home/bigdata/flume/checkpoint/channel

把channel目录下的文件删掉就可以了(这个是我自己设置的路径，每个人根据自己的checkpointDir配置的路径删)