在Flume写入HDFS的时候会自动换行问题, 项目要求写入HDFS中数据不需要换行, 查看源码及配置如下:
BodyTextEventSerializer.java
@Override
public void write(Event e) throws IOException {
out.write(e.getBody());
if (appendNewline) {
out.write('\n');
}
}
// for legacy reasons, by default, append a newline to each event written out
private final String APPEND_NEWLINE = "appendNewline";
private final boolean APPEND_NEWLINE_DFLT = false;
private final OutputStream out;
private final boolean appendNewline;
private BodyTextEventSerializer(OutputStream out, Context ctx) {
this.appendNewline = ctx.getBoolean(APPEND_NEWLINE, APPEND_NEWLINE_DFLT);
this.out = out;
}
http://flume.apache.org/FlumeUserGuide.html中查看此配置项为:
Body Text Serializer
Alias: text. This interceptor writes the body of the event to an output stream without any transformation or modification. The event headers are ignored. Configuration options are as follows:
Property Name Default Description
appendNewline true Whether a newline will be appended to each event at write time. The default of true assumes that events do not contain newlines, for legacy reasons.
Example for agent named a1:
a1.sinks = k1
a1.sinks.k1.type = file_roll
a1.sinks.k1.channel = c1
a1.sinks.k1.sink.directory = /var/log/flume
a1.sinks.k1.sink.serializer = text
a1.sinks.k1.sink.serializer.appendNewline = false