问题:
测试环境异常,经过查看server.log日志,发现:java.lang.OutOfMemoryError: Java heap space,具体如下:
server.log:
[2021-03-03 09:25:17,744] ERROR [ReplicaManager broker=1] Error processing append operation on partition ss-liantong-20210303-2 (kafka.server.ReplicaManager)
java.lang.OutOfMemoryError: Java heap space
[2021-03-03 09:25:24,683] ERROR [ReplicaManager broker=1] Error processing append operation on partition sz-l2-level-20210303-0 (kafka.server.ReplicaManager)
java.lang.OutOfMemoryError: Java heap space
java.io.IOException: Connection to 1 was disconnected before the response was read
at org.apache.kafka.clients.NetworkClientUtils.sendAndReceive(NetworkClientUtils.java:95)
at kafka.server.ReplicaFetcherBlockingSend.sendRequest(ReplicaFetcherBlockingSend.scala:96)
at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:205)
at kafka.server.ReplicaFetcherThread.fetch(ReplicaFetcherThread.scala:41)
at kafka.server.AbstractFetcherThread.processFetchRequest(AbstractFetcherThread.scala:149)
at kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:113)
at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:64)
[2021-03-03 09:31:13,002] INFO [GroupMetadataManager brokerId=0] Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)@
[2021-03-03 10:00:03,353] ERROR Processor got uncaught exception. (kafka.network.Processor)
java.lang.OutOfMemoryError: Java heap space
[2021-03-03 10:00:09,669] INFO Opening socket connection to server 172.16.2.179/172.16.2.179:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2021-03-03 10:00:07,487] INFO Updated PartitionLeaderEpoch. New: {epoch:1067, offset:172043}, Current: {epoch:1066, offset172027} for Partition: __consumer_offsets-24. Cache now contains 23 entries. (kafka.server.epoch.LeaderEpochFileCache)
[2021-03-03 10:00:07,484] ERROR Processor got uncaught exception. (kafka.network.Processor)
java.lang.OutOfMemoryError: Java heap space
以下是解决方法:
在kafka的启动配置文件中去设置,在/bin目录下找到kafka-run-class文件并编辑
#vim /bin/kafka-run-class
找到 Memory options处,默认设置是256M,将其修改为如下值
# Memory options
if [ -z "$KAFKA_HEAP_OPTS" ]; then
KAFKA_HEAP_OPTS="-Xmx1024M -Xms512M"
fi
保存退出,再运行连接器,不再出现Java heap space错误,解决该问题。