flume hdfs sink报错: java.lang.NoSuchFieldError: IGNORE_CLIENT_LOCALITY

集群环境:CDH 6.2.0  

Hadoop 3.0.0

在使用flume hdfs sink时 报错,java.lang.NoSuchFieldError: IGNORE_CLIENT_LOCALITY

flume sink在相应目录下创建了.tmp文件,但无法写入数据

2020-03-16 18:32:44,757 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:246)] Creating /user/spark/maidian/behavior_output/interceptor_test/20200316/03/interceptor-memory-channel-.1584354764737.tmp
2020-03-16 18:32:46,130 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)] process failed
java.lang.NoSuchFieldError: IGNORE_CLIENT_LOCALITY
	at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:204)
	at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:247)
	at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:313)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1176)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1155)
	at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1093)
	at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:463)
	at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:460)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:474)
	at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:401)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1103)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1083)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:972)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:960)
	at org.apache.flume.sink.hdfs.HDFSDataStream.doOpen(HDFSDataStream.java:81)
	at org.apache.flume.sink.hdfs.HDFSDataStream.open(HDFSDataStream.java:108)
	at org.apache.flume.sink.hdfs.BucketWriter$1.call(BucketWriter.java:257)
	at org.apache.flume.sink.hdfs.BucketWriter$1.call(BucketWriter.java:247)
	at org.apache.flume.sink.hdfs.BucketWriter$8$1.run(BucketWriter.java:727)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
	at org.apache.flume.auth.UGIExecutor.execute(UGIExecutor.java:46)
	at org.apache.flume.auth.KerberosAuthenticator.execute(KerberosAuthenticator.java:64)
	at org.apache.flume.sink.hdfs.BucketWriter$8.call(BucketWriter.java:724)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchFieldError: IGNORE_CLIENT_LOCALITY

百度各种无果,凭感觉是jar包问题

将flume /lib下的hadoop-common-3.0.0.jar 替换成 hadoop-common-3.0.0-cdh6.2.0.jar 问题解决

  • 0
    点赞
  • 0
    收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
©️2022 CSDN 皮肤主题:创作都市 设计师:CSDN官方博客 返回首页
评论

打赏作者

me凡

你的鼓励将是我创作的最大动力

¥2 ¥4 ¥6 ¥10 ¥20
输入1-500的整数
余额支付 (余额:-- )
扫码支付
扫码支付:¥2
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值