【Flink异常】java.lang.NullPointerException:at org.apache.flink.streaming.runtime.tasks.OperatorChain$Co

该系列博文为笔者安装使用Flink时遇到的问题,将其整理以供日后查阅。如发现问题,欢迎批评指正。

序号
(1)【Flink异常】java.lang.NullPointerException:at org.apache.flink.streaming.runtime.tasks.OperatorChain$Co
(2)【Flink异常】Caused by: java.lang.RuntimeException: Row arity of from does not match serializers
(3)【Flink异常】No operators defined in streaming topology. Cannot generate StreamGraph
(4)【Flink异常】Flink使用kafka报错(flink与kafka版本匹配):NetworkClient$DefaultMetadataUpdater.handleServerDisconnect
(5)【Flink异常】Caused by: org.apache.flink.table.api.ValidationException: Could not find any factories
(6)【Flink异常】Unable to instantiate java compiler
(7)【Flink异常】Could not find any factory for identifier ‘jdbc‘

1. 异常详情

java.lang.NullPointerException
    at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:577)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
	at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:579)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
	at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:579)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
	at org.apache.flink.streaming.api.operators.StreamMap.processElement(StreamMap.java:41)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.pushToOperator(OperatorChain.java:579)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:554)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:534)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
	at org.apache.flink.streaming.api.operators.TimestampedCollector.collect(TimestampedCollector.java:51)

2. 原因及解决办法

flink报空指针异常,虽然日志中报错的位置为数据读取的时候,如kafka抽取,但真正发生异常的位置多半不是这里。是由于没有捕捉到异常,导致异常抛到了pushToOperator中。以下提供可能导致此种现象的几种情况:

数据异常

如keyBy中key为null时,这里参考flink的空指针异常-阿里云

数据类型转换异常

笔者使用了Flink sql中的函数CASE WHEN转换表达式结果为“0”,“1”,"-1"。该函数转换后会自动给结果"0"和"1"后补空格,使它们都与"-1"一致,占两位。导致后续操作将这三个字符串转为数值型时出现异常,报错也报的是该文中的空指针问题。

数据类型为空
笔者调用flink DataStream的transform方法时,传TypeInfomation时意外传了空值。且由于transform内部方法未捕获异常(包括open()方法),导致真正的异常无法抛出,最终抛出该文空指针异常。如果在open中捕获异常,则又抛出异常如下:

java.lang.NullPointerException: null
	at java.util.Objects.requireNonNull(Objects.java:203)
	at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.<init>(StreamElementSerializer.java:68)
	at org.apache.flink.streaming.runtime.io.RecordWriterOutput.<init>(RecordWriterOutput.java:72)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createStreamOutput(OperatorChain.java:409)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:121)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:267)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
	at java.lang.Thread.run(Thread.java:745)

描述的就是TypeInformation序列化对象为空。因此修改DataStream的TypeInformation即可。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值