安装zookeeper和kafaka

最新推荐文章于 2022-05-30 12:02:45 发布

weixin_33937913

最新推荐文章于 2022-05-30 12:02:45 发布

阅读量429

点赞数

文章标签：大数据 python java

原文链接：https://my.oschina.net/yonghan/blog/1929039

版权

2019独角兽企业重金招聘Python工程师标准>>>

安装zookeeper和kafaka

http://mirrors.hust.edu.cn/apache/zookeeper/

http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.10/

(python2dev) root@alex-virtual-machine:/usr/src# mv zookeeper-3.4.10 /usr/local/zookeeper
(python2dev) root@alex-virtual-machine:/usr/src# mkdir -p /var/lib/zookeeper
(python2dev) root@alex-virtual-machine:/usr/src# cat > /usr/local/zookeeper/conf/zoo.cfg << EOF
> tickTime=2000
> dataDir=/var/lib/zookeeper
> clientPort=2181
> EOF
(python2dev) root@alex-virtual-machine:/usr/src# echo $JAVA_HOME
/usr/local/jdk/jdk1.8.0_131

(python2dev) root@alex-virtual-machine:/usr/local/zookeeper# /usr/local/zookeeper/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

安装kafaka

(python2dev) root@alex-virtual-machine:/usr/local/zookeeper# mv kafka_2.11-2.0.0 /usr/local/kafaka
(python2dev) root@alex-virtual-machine:/usr/local/zookeeper# mkdir /tmp/kafaka-logs
(python2dev) root@alex-virtual-machine:/usr/local/zookeeper# /usr/local/kafaka/bin/kafka-server-start.sh -daemon /usr/local/kafaka/config/server.properties

kafaka 启动报错

Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c0000000, 1073741824, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 1073741824 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /usr/local/zookeeper/hs_err_pid49324.log

解决方案

然后进入 kafka_2.12-0.11.0.0 的 bin 目录，找到 kafka-server-start.sh 文件，vi kafka-server-start.sh

找到这一行 export KAFKA_HEAP_OPTS=”-Xmx1G -Xms1G”

改为 export KAFKA_HEAP_OPTS=”-Xmx256M -Xms128M”

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    # export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"                                                                                                                                                
    export KAFKA_HEAP_OPTS="-Xmx256M -Xms128M" 
 fi

修改后重新启动

/usr/local/kafaka/bin/kafka-server-start.sh /usr/local/kafaka/config/server.properties