一、机器规划
主机名 | IP | Zookeeper | Nimbus | Supbervisor |
---|---|---|---|---|
liuyazhuang161 | 192.168.209.161 | 是 | 是 | 否 |
liuyazhuang162 | 192.168.209.162 | 是 | 否 | 是 |
liuyazhuang163 | 192.168.209.163 | 是 | 否 | 是 |
二、搭建Java环境
1、下载JDK
可以到Oracle官网下载Linux版本的JDK,链接为:http://www.oracle.com/technetwork/java/javase/downloads/index.html 我下载的是jdk1.7.0_72
2、解压JDK
在命令行输入tar -zxvf jdk-7u72-linux-x64.tar.gz 进行解压
3、配置环境变量
在命令行输入vim /etc/profile打开profile文件,在文件末尾添加如下代码:
JAVA_HOME=/usr/local/jdk1.7.0_72
CLASS_PATH=.:$JAVA_HOME/lib
PATH=$JAVA_HOME/bin:$PATH
export JAVA_HOME CLASS_PATH PATH
输入命令source /etc/profile使环境变量生效
4、拷贝文件
在命令行输入如下命令将JDK环境和/etc/profile文件拷贝到liuyazhuang1162和liuyazhuang163主机上。
scp -r /usr/local/jdk1.7.0_72 liuyazhuang162:/usr/local
scp -r /usr/local/jdk1.7.0_72 liuyazhuang163:/usr/local
scp /etc/profile liuyazhuang162:/etc/
scp /etc/profile liuyazhuang163:/etc/
同时,我们分别在liuyazhuang162和liuyazhuang163主机上执行命令source /etc/profile使环境变量生效
至此,JDK环境搭建完成。
三、搭建Zookeeper集群
1、下载Zookeeper
在Apache官网下载Zookeeper,链接为:http://www.apache.org/dyn/closer.cgi/zookeeper/我下载的是zookeeper-3.4.9
2、解压Zookeeper
在命令行输入tar -zxvf zookeeper-3.4.9.tar.gz对zookeeper进行解压。
3、Zookeeper集群搭建
切换到Zookeeper的conf目录下执行以下命令
cp zoo_sample.cfg zoo.cfg
然后输入vim zoo.cfg命令 配置如下:
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/usr/local/zookeeper-3.4.9/data
dataLogDir=/usr/local/zookeeper-3.4.9/datalog
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1
server.1=liuyazhuang161:2888:3888
server.2=liuyazhuang162:2888:3888
server.3=liuyazhuang163:2888:3888
同时在Zookeeper目录下新建data和dataLog两个目录用来存放Zookeeper的myid文件和日志文件,
将目录切换到zookeeper的data目录下
执行命令
vim myid
按下键盘i输入1后,按下esc键,再按下shift+; (也就是输入冒号:),再输入wq退出,此时就在zookeeper的data目录下创建了一个myid文件,这个myid的文件中的内容为1
4、配置Zookeeper环境变量
为操作方便,我们也将Zookeeper配置到环境变量中,加上之前配置的JDK,我们在profile的配置如下:
JAVA_HOME=/usr/local/jdk1.7.0_72
CLASS_PATH=.:$JAVA_HOME/lib
ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.9
PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH
export JAVA_HOME HADOOP_HOME CLASS_PATH PATH
5、拷贝文件
将Zookeeper环境和profile文件分别拷贝到liuyazhuang162和liuyazhuang163主机上。如下命令:
scp -r /usr/local/zookeeper-3.4.9 liuyazhuang162:/usr/local
scp -r /usr/local/zookeeper-3.4.9 liuyazhuang163:/usr/local
scp /ect/profile liuyazhuang162:/etc/
scp /ect/profile liuyazhuang163:/etc/
同时,我们分别在liuyazhuang162和liuyazhuang163主机上执行命令source /etc/profile使环境变量生效。
6、修改其他主机的myid文件
注:别忘了将liuyazhuang162主机上Zookeeper中myid文件内容修改为2 将liuyazhuang163主机上Zookeeper中myid文件内容修改为3
至此,Zookeeper集群环境搭建完毕
四、Storm集群搭建
1、下载Storm
Apache官方下载Storm链接为:http://storm.apache.org/downloads.html ,我这里下载的版本为apache-storm-1.1.1.tar.gz
2、解压Storm
tar -zxvf apache-storm-1.1.1.tar.gz
3、修改storm.yaml
修改storm.yaml,主要是修改如下配置
storm.zookeeper.servers:
- "192.168.209.161"
- "192.168.209.162"
- "192.168.209.163"
storm.zookeeper.port: 2181
storm.local.dir: "/usr/local/apache-storm-1.1.1/data"
#
nimbus.seeds: ["192.168.209.161"]
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
其中storm.zookeeper.servers,表示Zookeeper集群地址。如果Zookeeper集群使用的不是默认端口,那么还需要配置,storm.zookeeper.port。storm.local.dir用于配置Storm存储少量文件的路径。nimbus.seeds用于配置主控节点的地址,可以配置多个。
4、拷贝Storm
scp -r /usr/local/apache-storm-1.1.1/ liuyazhuang162:/usr/local/
scp -r /usr/local/apache-storm-1.1.1/ liuyazhuang163:/usr/local/
5、配置环境变量并拷贝
vim /etc/profile
JAVA_HOME=/usr/local/jdk1.7.0_72
CLASS_PATH=.:$JAVA_HOME/lib
STORM_HOME=/usr/local/apache-storm-1.1.1
ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.9
PATH=$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$STORM_HOME/bin:$PATH
export JAVA_HOME HADOOP_HOME CLASS_PATH PATH STORM_HOME
scp /ect/profile liuyazhuang162:/etc/
scp /ect/profile liuyazhuang163:/etc/
同时,我们分别在liuyazhuang162和liuyazhuang163主机上执行命令source /etc/profile使环境变量生效
6、启动Storm
192.168.209.161 后台运行 nimbus;
192.168.209.162 192.168.209.163后台运行 supervisor;
192.168.209.161 后台运行 storm ui,storm ui 得要在 nimbus 机子上运行,不能够在 supervisor 机子上运行。
五、访问Storm UI
在浏览器中输入链接http://192.168.209.161:8080