错误：Caused by:org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow.Available: ...

这个是写入Redis时用的序列化器，然后错误提示是超过了大小限制，把配置调大即可。

.set("spark.kryoserializer.buffer.max","128");

如果没有配置，那么找一下看下有没有硬编码写了大小的范围导致的。

 

参考：

http://blog.csdn.net/keyuquan/article/details/73379955

https://www.jianshu.com/p/758f147c63b4

https://www.2cto.com/net/201701/589554.html

https://stackoverflow.com/questions/27403732/kryoexception-buffer-overflow-with-very-small-input