java.nio.charset 包,java.nio.charset.MalformedInputException: Input length = 1

java.nio.charset.MalformedInputException: Input length = 1

2017-08-22 17:11:11   作者：MangoCool   来源：MangoCool

flume流化文件报错：java.nio.charset.MalformedInputException: Input length = 1，直接挂掉。

详细信息：

2017-08-22 16:49:42,819 (pool-3-thread-1) [ERROR - org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:256)] FATAL: Spool Directory source r_ggsn: { spoolDir: /data1/kafka/data2/ggsndata_flume }: Uncaught exception in SpoolDirectorySource thread. Restart or reconfigure Flume to continue processing.

java.nio.charset.MalformedInputException: Input length = 1

at java.nio.charset.CoderResult.throwException(CoderResult.java:281)

at org.apache.flume.serialization.ResettableFileInputStream.readChar(ResettableFileInputStream.java:195)

at org.apache.flume.serialization.LineDeserializer.readLine(LineDeserializer.java:133)

at org.apache.flume.serialization.LineDeserializer.readEvent(LineDeserializer.java:71)

at org.apache.flume.serialization.LineDeserializer.readEvents(LineDeserializer.java:90)

at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:252)

at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:228)

at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)

at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

原因：flume默认读取utf-8编码的文件，但是文件本身不是utf-8编码。

解决办法：

iconv filename.txt -f ISO-8859-1 -t UTF-8 -o filename_2.txt --verbose

这里假设文件编码为ISO-8859-1。

分享：