关闭

zookeeper和kafka的介绍

标签: zookeeperhadoop
60人阅读 评论(0) 收藏 举报

zookeeper部署


[hadoop@centos-1 zookeeper-3.4.5-cdh4.3.0]$ bin/./zkServer.sh start
JMX enabled by default
Using config: /export/home/tools/zookeeper-3.4.5-cdh4.3.0/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

[hadoop@centos-2 zookeeper-3.4.5-cdh4.3.0]$ bin/./zkServer.sh status
JMX enabled by default
Using config: /export/home/tools/zookeeper-3.4.5-cdh4.3.0/bin/../conf/zoo.cfg
Mode: standalone
```Step




<div class="se-preview-section-delimiter"></div>

1:以t账号登陆,创建stream用户
[root@iZ25fnur5jkZ home]# useradd storm
[root@iZ25fnur5jkZ home]# passwd storm
Step2:切换为shaka账号
[shaka@iZ25fnur5jkZ ~]sustormStep3zookeeper[shaka@iZ25fnur5jkZ ] mkdir zookeeper
Step4:进入zookeeper目录
[shaka@iZ25fnur5jkZ ~]cdzookeeperStep5zookeeperStep6[shaka@iZ25fnur5jkZzookeeper] tar -xvf zookeeper-3.3.6.deploy.tar.gz
Step7:获取本地IP,方法有多种,下面为常用的命令:
[shaka@iZ25fnur5jkZ conf]ifconfigIPStep8zookeeperppt[shaka@iZ25fnur5jkZzookeeper] vim conf/zoo.cfg
Step10:修改配置文件中的dataDir、dataLogDir、server.1

The number of milliseconds of each tick tickTime=2000
# The number of ticks that the initial # synchronization phase can take initLimit=10

sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. dataDir=/home/shaka/dep/zookeeper-3.3.6/data (改成自己的目录)
the port at which the clients will connect clientPort=2181
# set logs dataLogDir=/home/shaka/dep/zookeeper-3.3.6/logs (改成自己的目录) # set server server.1=hostname:4887:5887 (改成自己的hostname或IP)
#server.2=10.162.219.52:4887:5887 #server.3=10.163.15.119:4887:5887 # add by shaka # set max client connects maxClientCnxns=30Step11:启动zookeeper
修改bin下执行权限: chmod +x *
[shaka@iZ25fnur5jkZ zookeeper]$ bin/zkServer.sh start

Step12:启动后查看状态:
[shaka@iZ25fnur5jkZ zookeeper]$ bin/zkServer.sh status

停止ZK服务: [shaka@iZ25fnur5jkZ zookeeper]$ bin/zkServer.sh stop

重启ZK服务: [shaka@iZ25fnur5jkZ zookeeper]$ bin/zkServer.sh restart

zk的初始化目录为:get /zookeeper/quota

Step13:启动zookeeper的客户端

安装zkui ##首先安装编译工具mvn:
vim ~/.bash_profile export mvn3  export MVN_HOME=/usr/local/streamprocessing/dependence/apache-maven-3.2.1 export PATH=$PATH:$MVN_HOME/bin
进入zkui目录
mvn clean install
ln -s target/zkui-2.0-SNAPSHOT-jar-with-dependencies.jar  zkui-2.0-SNAPSHOT-jar-with-dependencies.jar
vim config.cfg

zkServer=cluster-node-01:2181

启动:nohup java -jar zkui-2.0-SNAPSHOT-jar-with-dependencies.jar
 &

查看:http://123.56.76.213:9090  或者    jps -l
登录账号
"username":"admin" , "password":"manager"
日志:zkui-out.log   nohup.out

创建一个node: 以自己名字命名





<div class="se-preview-section-delimiter"></div>

伪分布式安装

tar -xzf kafka_2.10-0.8.1.1.tgz

启动ZK bin/zookeeper-server-start.sh config/zookeeper.properties

启动服务 bin/kafka-server-start.sh -daemon config/server.properties

创建主题 bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

查看主题 bin/kafka-topics.sh --list --zookeeper localhost:2181
bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic test
创建生产者
 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test 

创建消费者
 bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

伪分布式安装
==

tar -xzf kafka_2.10-0.8.1.1.tgz

启动ZK bin/zookeeper-server-start.sh config/zookeeper.properties

启动服务 bin/kafka-server-start.sh -daemon config/server.properties

创建主题 bin/kafka-topics.sh –create –zookeeper localhost:2181 –replication-factor 1 –partitions 1 –topic test

查看主题 bin/kafka-topics.sh –list –zookeeper localhost:2181
bin/kafka-topics.sh –describe –zookeeper localhost:2181 –topic test
创建生产者
bin/kafka-console-producer.sh –broker-list localhost:9092 –topic test

创建消费者
bin/kafka-console-consumer.sh –zookeeper localhost:2181 –topic test –from-beginning

“`

kafka部署

step1:拷贝软件包

step2:解压
tar -xvf kafka_2.9.2-0.8.1.1.tgz 或者用 unzip kafka_2.9.2-0.8.1.1.zip
,修改配置文件conf/server.properties:
broker.id=0;
host.name=10.100.5.9;
zookeeper.connect=10.100.5.9:2181可逗号分隔配置多个

step3: 确保有执行权限
chmod +x sbin/*

step 4修改配置文件vim log4j.properties
log4j.appender.D.File = /data1/home/shaka/kafka/kafka_2.9.2-0.8.1.1/logs/debug.log
log4j.appender.E.File = /data1/home/shaka/kafka/kafka_2.9.2-0.8.1.1/logs/error.log

Step5:启动kafka服务
sbin/start-kafka.sh
查看是否启动: jsp -l
step6.创建topic主题:
bin/kafka-create-topic.sh –zookeeper 10.100.5.9:2181 –replica 1 –partition 1 –topic mykafka

0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:513次
    • 积分:102
    • 等级:
    • 排名:千里之外
    • 原创:10篇
    • 转载:1篇
    • 译文:0篇
    • 评论:0条
    文章分类
    文章存档