flume 运行时报NoSuchMethodError: com.google.common.base.Preconditions.checkArgument

最新推荐文章于 2024-10-09 20:37:38 发布

原创最新推荐文章于 2024-10-09 20:37:38 发布 · 2.5k 阅读

4 ·

CC 4.0 BY-SA版权

本文记录了在Hadoop与Flume环境中，因Guava版本不一致导致的NoSuchMethodError异常问题。通过对比flume和hadoop的lib目录下Guava版本，删除旧版本并替换为新版本，成功解决了问题。

1、遇到问题

今天在一台新服务器上搭建 hadoop、flume 常规操作，配完jdk，配 avro-hdfs.conf ，在开一个终端调用 avro-client 时，报NoSuchMethodError 异常。

2020-01-07 16:39:56,359 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:459)] process failed
java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
 at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
 at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
 at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
 at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
 at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
 at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
 at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
 at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
 at java.lang.Thread.run(Thread.java:748)
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
 at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
 at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
 at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:1679)
 at org.apache.flume.sink.hdfs.BucketWriter.open(BucketWriter.java:221)
 at org.apache.flume.sink.hdfs.BucketWriter.append(BucketWriter.java:572)
 at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:412)
 at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)
 at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)
 at java.lang.Thread.run(Thread.java:748)

2、解决问题

遇到这种问题，无非两种情况，一是丢失jar包，二是 jar包冲突。由于对hadoop以及flume版本选择比较随意。猜测是jar包存在冲突。找到报错的类 com.google.common.base.Preconditions ，通过查询该类存在 guava jar包中。flume的lib中 /opt/flume/apache-flume-1.9.0-bin/lib （guava-11.0.2.jar）而 hadoop 的 lib 中 /opt/hadoop/hadoop-3.2.1/share/hadoop/common/lib/ （guava-27.0-jre.jar），删除版本低的，并拷贝高版本的。问题解决。

3、意外发现

Google Guava 学习使用Google Guava可以让你快乐编程，写出优雅的JAVA代码！

附上原文

https://www.jianshu.com/p/ea44224b9cba