spark算子中用到scalal类，由于未序列化报错

最新推荐文章于 2024-04-26 22:04:53 发布

H_annuus

最新推荐文章于 2024-04-26 22:04:53 发布

阅读量6.7k

点赞数 3

分类专栏： linux hadoop 文章标签： scala spark Serializable 序列化 linux

本文链接：https://blog.csdn.net/u014487509/article/details/49994965

版权

由于spark算子用到的class没有实现序列化，报错如下所示15/11/23 14:43:47 ERROR Executor: Exception in task 0.0 in stage 4.0 (TID 4)java.io.NotSerializableException: EntityMentionSerialization stack: - object not ser

摘要由CSDN通过智能技术生成

<span style="font-family: Arial, Helvetica, sans-serif; font-size: 12px; background-color: rgb(255, 255, 255);">由于spark算子用到的class没有实现序列化，报错如下所示</span>

15/11/23 14:43:47 ERROR Executor: Exception in task 0.0 in stage 4.0 (TID 4)
java.io.NotSerializableException: EntityMention
Serialization stack:
	- object not serializable (class: EntityMention, value: EntityMention@5cdadff6)
	at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:40)
	at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47)
	at org.apache.spark.serializer.SerializationStream.writeValue(Serializer.scala:147)
	at org.apache.spark.storage.DiskBlockObjectWriter.write(DiskBlockObjectWriter.scala:181)
	at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.insertAll(BypassMergeSortSh

最低0.47元/天解锁文章

H_annuus

关注

3
点赞
踩
0

收藏

觉得还不错? 一键收藏
3
评论
spark算子中用到scalal类，由于未序列化报错

由于spark算子用到的class没有实现序列化，报错如下所示15/11/23 14:43:47 ERROR Executor: Exception in task 0.0 in stage 4.0 (TID 4)java.io.NotSerializableException: EntityMentionSerialization stack: - object not ser
复制链接

扫一扫

专栏目录