spark streaming报错
WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel!
WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel!
WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel!
WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel!
WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel!
WARN sink.TransactionProcessor: Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
at org.apache.spark.streaming.flume.sink.TransactionProcessorKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲populateEvents$…anonfun$populateEvents
1.
a
p
p
l
y
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
113
)
a
t
s
c
a
l
a
.
O
p
t
i
o
n
.
f
o
r
e
a
c
h
(
O
p
t
i
o
n
.
s
c
a
l
a
:
236
)
a
t
o
r
g
.
a
p
a
c
h
e
.
s
p
a
r
k
.
s
t
r
e
a
m
i
n
g
.
f
l
u
m
e
.
s
i
n
k
.
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
p
o
p
u
l
a
t
e
E
v
e
n
t
s
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
113
)
a
t
o
r
g
.
a
p
a
c
h
e
.
s
p
a
r
k
.
s
t
r
e
a
m
i
n
g
.
f
l
u
m
e
.
s
i
n
k
.
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
c
a
l
l
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
243
)
a
t
o
r
g
.
a
p
a
c
h
e
.
s
p
a
r
k
.
s
t
r
e
a
m
i
n
g
.
f
l
u
m
e
.
s
i
n
k
.
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
c
a
l
l
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
43
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
F
u
t
u
r
e
T
a
s
k
.
r
u
n
(
F
u
t
u
r
e
T
a
s
k
.
j
a
v
a
:
266
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
.
r
u
n
W
o
r
k
e
r
(
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
.
j
a
v
a
:
1149
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
1.apply(TransactionProcessor.scala:113) at scala.Option.foreach(Option.scala:236) at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor
1.apply(TransactionProcessor.scala:113)atscala.Option.foreach(Option.scala:236)atorg.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)atorg.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)atorg.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)atjava.util.concurrent.FutureTask.run(FutureTask.java:266)atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)atjava.util.concurrent.ThreadPoolExecutorWorker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
WARN sink.TransactionProcessor: Spark was unable to successfully process the events. Transaction is being rolled back.
WARN sink.SparkAvroCallbackHandler: Received an error batch - no events were received from channel!
WARN sink.TransactionProcessor: Error while processing transaction.
java.lang.IllegalStateException: begin() called when transaction is OPEN!
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.channel.BasicTransactionSemantics.begin(BasicTransactionSemantics.java:131)
at org.apache.spark.streaming.flume.sink.TransactionProcessorKaTeX parse error: Can't use function '$' in math mode at position 8: anonfun$̲populateEvents$…anonfun$populateEvents
1.
a
p
p
l
y
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
113
)
a
t
s
c
a
l
a
.
O
p
t
i
o
n
.
f
o
r
e
a
c
h
(
O
p
t
i
o
n
.
s
c
a
l
a
:
236
)
a
t
o
r
g
.
a
p
a
c
h
e
.
s
p
a
r
k
.
s
t
r
e
a
m
i
n
g
.
f
l
u
m
e
.
s
i
n
k
.
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
p
o
p
u
l
a
t
e
E
v
e
n
t
s
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
113
)
a
t
o
r
g
.
a
p
a
c
h
e
.
s
p
a
r
k
.
s
t
r
e
a
m
i
n
g
.
f
l
u
m
e
.
s
i
n
k
.
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
c
a
l
l
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
243
)
a
t
o
r
g
.
a
p
a
c
h
e
.
s
p
a
r
k
.
s
t
r
e
a
m
i
n
g
.
f
l
u
m
e
.
s
i
n
k
.
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
c
a
l
l
(
T
r
a
n
s
a
c
t
i
o
n
P
r
o
c
e
s
s
o
r
.
s
c
a
l
a
:
43
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
F
u
t
u
r
e
T
a
s
k
.
r
u
n
(
F
u
t
u
r
e
T
a
s
k
.
j
a
v
a
:
266
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
.
r
u
n
W
o
r
k
e
r
(
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
.
j
a
v
a
:
1149
)
a
t
j
a
v
a
.
u
t
i
l
.
c
o
n
c
u
r
r
e
n
t
.
T
h
r
e
a
d
P
o
o
l
E
x
e
c
u
t
o
r
1.apply(TransactionProcessor.scala:113) at scala.Option.foreach(Option.scala:236) at org.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243) at org.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor
1.apply(TransactionProcessor.scala:113)atscala.Option.foreach(Option.scala:236)atorg.apache.spark.streaming.flume.sink.TransactionProcessor.populateEvents(TransactionProcessor.scala:113)atorg.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:243)atorg.apache.spark.streaming.flume.sink.TransactionProcessor.call(TransactionProcessor.scala:43)atjava.util.concurrent.FutureTask.run(FutureTask.java:266)atjava.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)atjava.util.concurrent.ThreadPoolExecutorWorker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
解决方法：
在启动Flume之前会拷贝三个jar(commons-lang3-3.5.jar、scala-library-2.11.8.jar、spark-streaming-flume-sink_2.11-2.2.0.jar)到lib目录下，查看lib目录下的这三个jar是否有冲突，删掉冲突的jar即可。
扫一扫
1756
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
