zeppline 报大量的 Failed to allocate a page 错误

最新推荐文章于 2023-04-20 11:46:08 发布

愤怒的天空

最新推荐文章于 2023-04-20 11:46:08 发布

阅读量1.1k

点赞数

分类专栏：踩坑日记

本文链接：https://blog.csdn.net/weixin_42660215/article/details/86651836

版权

在使用Zepplin运行SQL时遇到'Failed to allocate a page'错误，同时发现Spark性能下降。通过调整Zepplin配置，增加driver内存至5GB解决了此问题。尽管executor内存充足，但driver内存不足可能是关键原因。

摘要由CSDN通过智能技术生成

跑SQL就报这个问题, spark 很卡

WARN [2019-01-25 07:23:33,827] ({broadcast-exchange-0} TaskMemoryManager.java[allocatePage]:300) - Failed to allocate a page (2097152 bytes), try again.
 WARN [2019-01-25 07:23:34,703] ({broadcast-exchange-0} TaskMemoryManager.java[allocatePage]:300) - Failed to allocate a page (2097152 bytes), try again.
 WARN [2019-01-25 07:23:35,597] ({broadcast-exchange-0} TaskMemoryManager.java[allocatePage]:300) - Failed to allocate a page (2097152 bytes), try again.
 WARN [2019-01-25 07:23:36,495] ({broadcast-exchange-0} TaskMemoryManager.java[allocatePage]:300) - Failed to allocate a page (2097152 bytes), try again.
 WARN [2019-01-25 07:23:37,371] ({broadcast-exchange-0} TaskMemoryManager.java[allocatePage]:300) - Failed to allocate a page (2097152 bytes), try again.
 WARN [2019-01-25 07:23:38,244] ({broadcast-exchange-0} TaskMemoryManager.java[allocatePage]:300) - Failed to allocate a page (2097152 bytes), try again.
 WARN [2019-01-25 07:23:39,130] ({bro