Hive与Supersql参数

1.supersql设置内存

当报错时

Caused by: org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 0, required: 2006. To avoid this, increase spark.kryoserializer.buffer.max value.

可通过一下参数设定

set `livy.session.conf.spark.driver.memory` = `8g`;
set `livy.session.conf.spark.excutor.memory` = `8g`;
set `livy.session.conf.spark.kryoserializer.buffer.max`=`512m`;

2.Hive与Presto的函数对比

1.在Hive中 having(distinct id) 会报错, 而在presto中正常运行