ubuntu下使用IDEA开发大数据遇到的坑记录

概述

我开始使用ubuntu来进行开发活动.此贴持续更新记录下发生遇到的问题.ubuntu开发环境构建有空我写个帖子.

环境信息

软件版本号安装位置
osubuntu 20.04-
javajdk-8u333/usr/local
maven3.8.5/usr/local
ideideaIU-2021.3.3/opt/
redis6.2.7/usr/local/soft
nodejs16.14.2/usr/local/node16.14.2
yarn1.22.18/usr/local/node16.14.2/bin/
npm8.10.0/usr/local/node16.14.2/bin/
mysql5.7.37数据库目录:/var/lib/mysql/ 配置文件:/usr/share/mysql(命令及配置文件) ,/etc/mysql(如:my.cnf)相关命令:/usr/bin(mysqladmin mysqldump等命令) 和/usr/sbin启动脚本:/etc/init.d/mysql(启动脚本文件mysql的目录)
vscode1.67.2-1652812855_amd64-
wpswps-office_11.1.0.10976_amd64-

IDEA报错

Error1. ‘{’ expected but ‘import’ found.

背景:之前在win10上正常运行的程序,迁移到ubuntu上就变这样了.开始莫名其妙.
在这里插入图片描述
百度完全找不到答案,感觉最近的答案都是谷歌找到的.看到有个stackoverflow的帖子说怀疑是换行符的问题导致.Play exception: ‘{’ expected but ‘import’ found
在这里插入图片描述
于是我找了下换行符的相关问题,( IntelliJ换行CRLF, LF, CR的解释和默认设置)透过这篇帖子我发现我得换行符在IDEA右下角显示是CR,于是我开始批量替换成linux的LF型换行符(参考: IDEA批量替换文件换行符、分隔符CRLF、LF、CR),再试一次跑成功了.

Error2. flink1.13 checkpoint序列化反序列化问题.log如下

2022-05-23 15:21:21
java.lang.Exception: Exception while creating StreamOperatorStateContext.
	at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:254)
	at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:272)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:436)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:582)
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.executeRestore(StreamTask.java:562)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:647)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:537)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:759)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.FlinkException: Could not restore operator state backend for StreamSource_cbc357ccb763df2852fee8c4fc7d55f2_(3/3) from any of the 1 provided restore options.
	at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:160)
	at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.operatorStateBackend(StreamTaskStateInitializerImpl.java:285)
	at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:173)
	... 10 more
Caused by: org.apache.flink.runtime.state.BackendBuildingException: Failed when trying to restore operator state backend
	at org.apache.flink.runtime.state.DefaultOperatorStateBackendBuilder.build(DefaultOperatorStateBackendBuilder.java:83)
	at org.apache.flink.runtime.state.hashmap.HashMapStateBackend.createOperatorStateBackend(HashMapStateBackend.java:148)
	at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$operatorStateBackend$0(StreamTaskStateInitializerImpl.java:276)
	at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:168)
	at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
	... 12 more
Caused by: com.esotericsoftware.kryo.KryoException: Encountered unregistered class ID: -1794653143
Serialization trace:
$outer (scala.collection.convert.Wrappers$SeqWrapper)
	at com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:119)
	at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:641)
	at com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:99)
	at com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
	at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:761)
	at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.deserialize(KryoSerializer.java:354)
	at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:151)
	at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.deserialize(TupleSerializer.java:37)
	at org.apache.flink.runtime.state.OperatorStateRestoreOperation.deserializeOperatorStateValues(OperatorStateRestoreOperation.java:217)
	at org.apache.flink.runtime.state.OperatorStateRestoreOperation.restore(OperatorStateRestoreOperation.java:188)
	at org.apache.flink.runtime.state.DefaultOperatorStateBackendBuilder.build(DefaultOperatorStateBackendBuilder.java:80)
	... 16 more

这个没看到咋解决,我重启了任务.估计是哪个类加了序列化,flink的序列化没用退化为kryo的序列化.结果报错了

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值