java执行spool_flume agent 收集SogouQ.reduced数据的时候报错

配置如下：

1.在SYH-2 conf/flume_sogoulog.conf

a1.sources = r1

a1.channels = c1

a1.sinks = k1

#配置source

a1.sources.r1.type = spooldir

a1.sources.r1.spoolDir=/root/data/sogoulog

a1.sources.r1.channels = c1

#配置channels

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

#配置sinks

a1.sinks.k1.type = avro

a1.sinks.k1.hostname = SYH-1

a1.sinks.k1.port = 5554

a1.sinks.k1.channel = c1

2.在SYH-1创建flume_sogouhdfs配置文件，conf/flume_sogouhdfs.conf

a1.sources = r1

a1.channels = c1

a1.sinks = k1

# 配置sources

a1.sources.r1.type = avro

a1.sources.r1.bind = SYH-1

a1.sources.r1.port = 5554

a1.sources.r1.channels = c1

#配置channels

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

#配置sinks

a1.sinks.k1.type=hdfs

a1.sinks.k1.hdfs.path = hdfs://SYH-1:8020/user/root/logs/%y-%m-%d/%H%M%S

a1.sinks.k1.hdfs.fileType=DataStream

a1.sinks.k1.hdfs.writeFormat=TEXT

a1.sinks.k1.hdfs.round=true

a1.sinks.k1.hdfs.roundValue=5

a1.sinks.k1.hdfs.roundUnit=second

a1.sinks.k1.hdfs.useLocalTimeStamp=true

a1.sinks.k1.hdfs.filePrefix=events-

a1.sinks.k1.channel = c1

SYH-1启动flume_sogouhdfs

cd /usr/lib/flume-ng/

bin/flume-ng agent -c conf -f conf/flume_sogouhdfs.conf -n a1 -Dflume.root.logger=INFO,console

SYH-2启动flume_sogoulog

cd /usr/lib/flume-ng/

bin/flume-ng agent -c conf -f conf/flume_sogoulog.conf -n a1 -Dflume.root.logger=INFO,console

在SYH-2

cd /root/data/sogoulog

echo "111" > 111 没有任何问题 并且正常传入hdfs上

但把 SogouQ.reduced 文件放在 /root/data/sogoulog就会报错，是不是和字符编码有关，如何解决

2015-07-27 21:08:23,783 (pool-4-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r1: { spoolDir: /root/data/sogoulog }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.

java.nio.charset.MalformedInputException: Input length = 1

at java.nio.charset.CoderResult.throwException(CoderResult.java:277)

at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)

at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)

at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)

at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:90)

at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:252)

at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:744)