flink开发过程中遇到的问题

类转换异常

cannot be cast to com.google.protobuf.Message

Caused by: java.lang.ClassCastException: org.apache.hadoop.yarn.proto.YarnServiceProtos$RegisterApplicationMasterRequestProto cannot be cast to com.google.protobuf.Message
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
	at com.sun.proxy.$Proxy14.registerApplicationMaster(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
	at com.sun.proxy.$Proxy15.registerApplicationMaster(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:222)
	at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:214)
	at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138)
	at org.apache.flink.yarn.YarnResourceManager.createAndStartResourceManagerClient(YarnResourceManager.java:205)
	at org.apache.flink.yarn.YarnResourceManager.initialize(YarnResourceManager.java:234)
	... 11 common frames omitted

这种问题一般发生在将flink程序部署到yarn集群的时候，由于flink启动的时候会加载hadoop下面的jar包，加到自己的classpath下面，如果hadoop的jar包和自己flink工程的jar包版本不一致就会导致这个问题，解决办法：排除自己工程中的hadoop相关的jar,打包的时候不要打进来.

 <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>${hadoop.version}</version>
            <scope>provided</scope>
        </dependency>

提示，有可能你没有显式的引入hadoop-common、hadoop-hdfs这样的jar包，但是你引入了hbase的jar包，因为hbase会引入hdfs相关的jar包，也会导致类似的错误。

local class incompatible

org.apache.flink.table.api.StreamQueryConfig; local class incompatible: stream classdesc serialVersionUID = XXX, local class serialVersionUID = -XXX

在提交flink程序的时候如果出现这种情况一般是由于jar包冲突导致的，可能是某一个jar的冲突、也可能是scala的版本不一致导致的，比如你的flink集群的scala是2.11，你的程序的scala版本是2.12

ClassCastException

java.lang.ClassCastException: org.codehaus.janino.CompilerFactory cannot be cast to org.codehaus.commons.compiler.ICompilerFactory

这种情况是由于你的程序中flink-table-planner和flink libs下面的包冲突造成的，pom添加 provided打包的时候不要将flink-table-planner打进去

 <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-table-planner_${scala.binary.version}</artifactId>
            <version>${flink.version}</version>
            <scope>provided</scope>
        </dependency>

使用kafka异常

java.lang.ClassCastException: cannot assign instance of org.apache.commons.collections.map.LinkedMap to field org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.pendingOffsetsToCommit of type org.apache.commons.collections.map.LinkedMap in instance of org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer010

这种情况一般是由于kafka和flink的jar不兼容造成的，flink1.10之后，默认是child-first，
解决方法：
修改flink配置 classloader.resolve-order: parent-first

水印异常

java.lang.RuntimeException: Record has Long.MIN_VALUE timestamp (= no timestamp marker). Is the time characteristic set to 'ProcessingTime', or did you forget to call 'DataStream.assignTimestampsAndWatermarks(...)'?

原因：你所使用的datastream没设置水印

No ExecutorFactory found to execute the application

flink 升级1.11之后报这个错误，添加以下依赖就行了

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-clients_${scala.binary.version}</artifactId>
    <version>${flink.version}</version>
</dependency>

Deployment took more than 60 seconds

org.apache.flink.yarn.AbstractYarnClusterDescriptor           - Deployment took more than 60 seconds. Please check if the requested resources are available in the YARN cluster
我这里遇到的问题是所在机器8081被占用了，换一个端口就好了

更多精彩内容，欢迎关注我的公众号，谢谢