org.apache.flume.conf.ConfigurationException: brokerList must contain at least one Kafka brokerList

楼主在使用flume采集数据,sink到kafka中是,启动flume后报错,楼主在网上搜了一圈,说的原因是对的,就是flume1.8版本之后,要对应kafka0.9版本以上,另外配置文件的配置项也不同,楼主这里不再说明,更改完成后,依然,报错:

WARN kafka.KafkaSink: The Property 'topic' is not set. Using the default topic name: default-flume-topic
19/04/20 13:11:44 INFO kafka.KafkaSinkUtil: context={ parameters:{kafka.flumeBatchSize=20, kafka.bootstrap.servers=hadoop001:9092 hadoop002:9092 hadoop003:9092, kafka.producer.acks=1, channel=c1, kafka.topic=calllog, type=org.apache.flume.sink.kafka.KafkaSink} }
19/04/20 13:11:44 ERROR node.AbstractConfigurationProvider: Sink k1 has been removed due to an error during configuration
org.apache.flume.conf.ConfigurationException: brokerList must contain at least one Kafka broker
	at org.apache.flume.sink.kafka.KafkaSinkUtil.addDocumentedKafkaProps(KafkaSinkUtil.java:55)
	at org.apache.flume.sink.kafka.KafkaSinkUtil.getKafkaProperties(KafkaSinkUtil.java:37)
	at org.apache.flume.sink.kafka.KafkaSink.configure(KafkaSink.java:211)
	at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
	at org.apache.flume.node.AbstractConfigurationProvider.loadSinks(AbstractConfigurationProvider.java:413)
	at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:98)
	at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)

折腾了一上午,楼主发现,环境变量未生效,source /etc/profile .之后再启动,就好了。首先需要配置环境变量,楼主之前,已经配好的,但是用的是虚拟机,重启之后,一些命令的执行,都需要刷新一下环境变量,比如hadoop fs  命令等

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值