JStorm的搭建文档

1、下载jstorm的jar包
　　https://github.com/alibaba/jstorm/releases

2、解压jstorm的包
　　tar -xvf jstorm-2.4.0.tgz
　　mv jstorm-2.4.0 jstorm
修改环境变量：
　　vi /etc/profile
　　　　export JSTORM_HOME=/usr/local/jstorm
　　　　export PATH=$PATH:$JSTORM_HOME/bin
　　生效 source /etc/profile

2、上传并修改storm.yaml
　　mv storm.yaml storm.yaml.bak

　　vi storm.yaml
　　nimbus.host: 表示nimbus的地址， 填写ip

storm.zookeeper.servers:
     - "zk-datanode-01"
     - "zk-datanode-02"
     - "zk-datanode-03"
nimbus.host: "192.168.46.160"     
storm.zookeeper.root: "/jstorm"
storm.local.dir: "/usr/local/jstorm/data"
supervisor.slots.ports:
    - 6800
    - 6801
    - 6802
    - 6803

3、部署到其他节点
　　scp -r jstorm zk-datanode-02:/usr/local/
　　scp -r jstorm zk-datanode-03:/usr/local/

4、启动storm
　　在nimbus 节点上执行
　　nohup jstorm nimbus & 查看$JSTORM_HOME/logs/nimbus.log检查有无错误

　　在supervisor节点上执行
　　nohup jstorm supervisor & 查看$JSTORM_HOME/logs/supervisor.log检查有无错误
　　若没有配置环境变量
　　nohup bin/jstorm supervisor &

5、webUI的部署
　　必须使用tomcat 7.x版本， 注意不要忘记拷贝 ~/.jstorm/storm.yaml
　　Web UI 可以和Nimbus不在同一个节点

　　

        mkdir ~/.jstorm
        cp -f $JSTORM_HOME/conf/storm.yaml ~/.jstorm
        cd /usr/local/tomcat
        cd webapps
        cp $JSTORM_HOME/jstorm-ui-2.4.0.war ./            
        mv ROOT ROOT.old
        ln -s jstorm-ui-2.4.0 ROOT
        cd ../bin
        ./startup.sh            

访问http://192.168.46.160:8080/ 即可查看到jstorm的界面

 

6、WebUI 增加新集群
　　注意： web ui使用的版本必须和集群中JStorm最高的版本一致

　　在运行Web UI的机器上， 修改~/.jstorm/storm.yaml
　　把默认的ui.clusters注释给去掉， 补充

# UI MultiCluster
        # Following is an example of multicluster UI configuration
         ui.clusters:
             - {
                 name: "jstorm.share",
                 zkRoot: "/jstorm",
                 zkServers:
                     [ "zk-datanode-01", "zk-datanode-02", "zk-datanode-03"],
                 zkPort: 2181,
               }
             - {
                 name: "jstorm.bu1",
                 zkRoot: "/jstorm.dw",
                 zkServers:
                     [ "10.125.100.101", "10.125.100.101", "10.125.100.101"],
                 zkPort: 2181,
               }
        解释一下
        
        - {
                 name: "jstorm.bu1",    --- 这个集群的名字， 每个集群的名字必须不一样
                 zkRoot: "/jstorm.dw",  --- 这个集群 zk的根节点，可以参考$JSTORM_HOME/con/storm.yaml 中“storm.zookeeper.root” 字段
                 zkServers:
                     [ "10.125.100.101", "10.125.100.101", "10.125.100.101"],   -- zk 机器列表
                 zkPort: 2181,                                                  -- zk 的客户端端口
          }

 

提交jstorm的任务
　　jstorm jar /usr/local/jstormpack/JstormTest-0.0.1-SNAPSHOT.jar com.dinpay.bdp.rcp.mywordcount.WordCountTopologMain WordCountTopolog

停止jstorm的任务
　　jstorm kill KTTAndNormalTopology

nimbus.childopts: "-Xms1g -Xmx1g -Xmn512m -XX:PermSize=256m  -XX:SurvivorRatio=4 -XX:+UseConcMarkSweepGC  -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000"

worker.childopts: "-Xms512m -Xmx512m -Xmn256m -XX:SurvivorRatio=2 -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=65"

ui.childopts: "-Xms768m -Xmx768m -Xmn256m -XX:PermSize=96m -XX:+UseConcMarkSweepGC  -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError  -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000"

supervisor.childopts: "-Xms2g -Xmx2g -Xmn1g -XX:PermSize=256m -XX:+UseConcMarkSweepGC  -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:CMSFullGCsBeforeCompaction=5 -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=100M -XX:+UseCMSCompactAtFullCollection -XX:CMSMaxAbortablePrecleanTime=5000"

 

转载于:https://www.cnblogs.com/atomicbomb/p/8214477.html