Flink 运行错误 java.lang.OutOfMemoryError: Direct buffer memory

最新推荐文章于 2023-09-21 19:14:43 发布

原创

最新推荐文章于 2023-09-21 19:14:43 发布 · 2.8k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#java #flink #jvm

本文介绍了解决 Apache Flink 运行过程中遇到的 Direct Buffer Memory 不足的问题，通常表现为 OutOfMemoryError 错误。针对此问题，文章建议调整 taskmanager.memory.framework.off-heap.size 的配置选项。

如遇到如下错误，表示需要调大配置项 taskmanager.memory.framework.off-heap.size 的值，taskmanager.memory.framework.off-heap.size 的默认值为 128MB，错误显示不够用需要调大。

2022-12-16 09:09:21,633 INFO [464321] [org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:366)] - Hadoop UGI authentication : TAUTH
2022-12-16 09:09:21,735 ERROR [464355] [org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager$1.accept(SplitFetcherManager.java:119)] - Received uncaught exception.
java.lang.OutOfMemoryError: Direct buffer memory
   at java.nio.Bits.reserveMemory(Bits.java:695)
   at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
   at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
   at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:241)
   at sun.nio.ch.IOUtil.read(IOUtil.java:195)
   at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
   at org.apache.kafka.common.network.PlaintextTransportLayer.read(PlaintextTransportLayer.java:103)
   at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:118)
   at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452)
   at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402)
   at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674)
   at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576)
   at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
   at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:561)
   at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.transmitSends(ConsumerNetworkClient.java:324)
   at org.

最低0.47元/天解锁文章