【问题描述】
今天在CDH集群中使用Flume的时候报了如下的错误:
2020-04-10 11:02:33,627 (conf-file-poller-0) [ERROR - org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:154)] Unhandled error
java.lang.NoSuchMethodError: org.apache.flume.Context.getSubProperties(Ljava/lang/String;)Lcom/google/common/collect/ImmutableMap;
at org.keedio.flume.source.HibernateHelper.<init>(HibernateHelper.java:46)
at org.keedio.flume.source.SQLSource.configure(SQLSource.java:72)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:325)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:105)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
【问题原因】
根据报错内容,可以发现原因是因为Flume中getSubProperties的返回类型不匹配。org.apache.flume.context类在flume-ng-configuration-1.9.0-cdh6.3.1.jar、flume-ng-configuration.jar包里面,查看这个两个jar包,里面都是有方法:public Map<String, String> getSubProperties(String prefix):
public Map<String, String> getSubProperties(String prefix)
{
Preconditions.checkArgument(prefix.endsWith("."), "The given prefix does not end with a period (" + prefix + ")");
Map<String, String> result = Maps.newHashMap();
synchronized (this.parameters)
{
for (Map.Entry<String, String> entry : this.parameters.entrySet())
{
String key = (String)entry.getKey();
if (key.startsWith(prefix))
{
String name = key.substring(prefix.length());
result.put(name, entry.getValue());
}
}
}
return ImmutableMap.copyOf(result);
}
而添加的flume-ng-sql-source jar包虽然也有这个方法,但是返回值类型不一样,这个jar包里面的方法返回值类型是ImmutableMap。
【解决办法】
更换flume-ng-sql-source jar包,jar包可以点这里(链接:https://pan.baidu.com/s/13TcC4uyLtNLeFKAheA0Jqw 提取码:xnij)下载,此处的jar包返回值类型为Map。