- 先要进行写入的配置:
private static Configuration conf = new Configuration();
static {
conf.setBoolean("dfs.support.append", true);
conf.set("dfs.client.block.write.replace-datanode-on-failure.policy", "NEVER");
conf.setBoolean("dfs.client.block.write.replace-datanode-on-failure.enable", true);
}
这些配置事固定的,第一个很容易理解,第二个和第三个,顾名思义替换数据节点datanode,如果不这样设置写入文件会报错。具体错误不在此罗列,有兴趣的可以不设置看一下
- 创建追加写入的方法
/** * json 存储 * * @param * @param * @throws IOException */ public static void write2hdfs(Object listContent, String filePath) throws IOException { ObjectMapper objectMapper = new ObjectMapper(); FileSystem fs = null; Path path = new Path(filePath); FSD