![](https://img-blog.csdnimg.cn/9d19500ae54d4a22883afd606e5f00d1.jpeg?x-oss-process=image/resize,m_fixed,h_224,w_224)
项目问题
文章平均质量分 81
项目问题
遥遥晚风点点
大数据,java
展开
-
记一次Spark内存溢出与性能优化
场景执行一个spark任务后,发现控制台一直报错如下:21/04/21 10:32:29 ERROR cluster.YarnScheduler: Lost executor 3 on cdh-slave1.test.com: Container killed by YARN for exceeding memory limits. 5.5 GB of 5.5 GB physical memory used. Consider boosting spark.yarn.executor.memoryO原创 2021-04-21 16:06:41 · 745 阅读 · 0 评论 -
项目问题:hive中读取parquet文件的问题
问题:hive中查询一段语句:select guid, sessionid, properties, first_value(properties['pageid']) over(partition by guid,sessionid order by ts) as in_page, last_value(properties['pageid']) over(partition by guid,sessionid order by ts rows betwe原创 2020-09-08 12:44:15 · 2174 阅读 · 0 评论 -
项目问题:布隆过滤器与kyro序列化
https://zhuanlan.zhihu.com/p/72378274原创 2020-09-06 08:58:30 · 379 阅读 · 0 评论