storm出现log4j-over-slf4j.jar和slf4j-log4j12.jar冲突

最新推荐文章于 2024-10-08 18:43:03 发布

gongmf

最新推荐文章于 2024-10-08 18:43:03 发布

阅读量1.5w

点赞数 2

分类专栏： storm 文章标签： storm slf4j-log4j12.jar冲突

本文链接：https://blog.csdn.net/gongmf/article/details/40379547

版权

storm 专栏收录该内容

2 篇文章 0 订阅

订阅专栏

提交topology后，通过ui界面看到Emitted和Transferred都为0

storm jar xxxx.jar  包名.类名

到$STORM_HOME/logs目录下，查看storm的日志看到报错信息，因为log4j-over-slf4j.jar和slf4j-log4j12.jar冲突。

2014-10-22 17:03:38 STDIO [ERROR] SLF4J: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError.
2014-10-22 17:03:38 STDIO [ERROR] SLF4J: See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
2014-10-22 17:03:38 b.s.util [ERROR] Async loop died!
java.lang.ExceptionInInitializerError: null
	at org.apache.log4j.Logger.getLogger(Logger.java:39) ~[log4j-over-slf4j-1.6.6.jar:1.6.6]
	at kafka.utils.Logging$class.logger(Logging.scala:24) ~[stormjar.jar:na]
	at kafka.network.BlockingChannel.logger(BlockingChannel.scala:35) ~[stormjar.jar:na]
	at kafka.utils.Logging$class.debug(Logging.scala:51) ~[stormjar.jar:na]
	at kafka.network.BlockingChannel.debug(BlockingChannel.scala:35) ~[stormjar.jar:na]
	at kafka.network.BlockingChannel.connect(BlockingChannel.scala:64) ~[stormjar.jar:na]
	at kafka.consumer.SimpleConsumer.connect(SimpleConsumer.scala:44) ~[stormjar.jar:na]
	at kafka.consumer.SimpleConsumer.getOrMakeConnection(SimpleConsumer.scala:142) ~[stormjar.jar:na]
	at kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:69) ~[stormjar.jar:na]
	at kafka.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:124) ~[stormjar.jar:na]
	at kafka.javaapi.consumer.SimpleConsumer.getOffsetsBefore(SimpleConsumer.scala:79) ~[stormjar.jar:na]
	at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:77) ~[stormjar.jar:na]
	at storm.kafka.KafkaUtils.getOffset(KafkaUtils.java:67) ~[stormjar.jar:na]
	at storm.kafka.trident.TridentKafkaEmitter.doEmitNewPartitionBatch(TridentKafkaEmitter.java:111) ~[stormjar.jar:na]
	at storm.kafka.trident.TridentKafkaEmitter.failFastEmitNewPartitionBatch(TridentKafkaEmitter.java:72) ~[stormjar.jar:na]
	at storm.kafka.trident.TridentKafkaEmitter.emitNewPartitionBatch(TridentKafkaEmitter.java:79) ~[stormjar.jar:na]
	at storm.kafka.trident.TridentKafkaEmitter.access$000(TridentKafkaEmitter.java:46) ~[stormjar.jar:na]
	at storm.kafka.trident.TridentKafkaEmitter$1.emitPartitionBatch(TridentKafkaEmitter.java:204) ~[stormjar.jar:na]
	at storm.kafka.trident.TridentKafkaEmitter$1.emitPartitionBatch(TridentKafkaEmitter.java:194) ~[stormjar.jar:na]
	at storm.trident.spout.OpaquePartitionedTridentSpoutExecutor$Emitter.emitBatch(OpaquePartitionedTridentSpoutExecutor.java:127) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at storm.trident.spout.TridentSpoutExecutor.execute(TridentSpoutExecutor.java:82) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at storm.trident.topology.TridentBoltExecutor.execute(TridentBoltExecutor.java:369) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.daemon.executor$fn__5641$tuple_action_fn__5643.invoke(executor.clj:631) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:399) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj:746) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating]
	at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na]
	at java.lang.Thread.run(Thread.java:722) [na:1.7.0_09]
Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError. See also http://www.slf4j.org/codes.html#log4jDelegationLoop for more details.
	at org.apache.log4j.Log4jLoggerFactory.<clinit>(Log4jLoggerFactory.java:49) ~[log4j-over-slf4j-1.6.6.jar:1.6.6]
	... 32 common frames omitted

解决方法：在工程的pom文件exclude掉slf4j-log4j12，我的是hbase--client中exclude，要根据实际情况选择哪个dependency下。

参考：http://stackoverflow.com/questions/21630860/storm-topology-not-submit

		<dependency>
			<groupId>org.apache.hbase</groupId>
			<artifactId>hbase-client</artifactId>
			<version>0.98.4-hadoop2</version>
																
			<!-- add by gongmingfeng 2014-10-22  解决问题：
			Caused by: java.lang.IllegalStateException: Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflowError.
			-->
			<exclusions>
				  <exclusion> 
				    <groupId>org.slf4j</groupId>
				    <artifactId>slf4j-log4j12</artifactId>
				  </exclusion>
				  <exclusion> 
				    <groupId>log4j</groupId>
				    <artifactId>log4j</artifactId>
				  </exclusion>
			</exclusions>
		</dependency>