经过一个暑假的实习,项目已经完成,在此期间,对storm, jstorm, storm-yarn都有了进一步的了解,在此对jstorm on yarn项目作一个整体的总结。
1. jstorm-yarn项目是依照storm-yarn项目进行修改,主体架构保持不变,只在jstorm与storm不同的地方进行修改,使之能够运行。
2. jstorm-yarn架构:
用户启动jstorm-yarn后,会首先向yarn申请一个container运行MasterServer,MasterServer会启动jstorm Nimbus Server。同时MasterServer也继承了ThriftServer,用于接收客户端命令。当yarn集群资源充足时,MasterServer也会申请启动默认数量的jstorm Supervisor Server。supervisor数量可通过命令addSupervisor进行增加。
3. jstorm-yarn命令: []内为可选参数
- jstorm-yarn help
- jstorm-yarn launch <master.yaml>
-appname <arg> Application Name. Default value - JStorm-on-Yarn
-jstormConfOutput <arg> storm.yaml file
-jstormHome <arg> JStorm Home Directory
-jstormZip <arg> file path of jstorm.zip
-output <arg> Output file
-queue <arg> RM Queue in which this application is to be submitted
- jstorm-yarn addSupervisors -appId <arg> [-output <arg>] -supervisors <arg>
- jstorm-yarn stopSupervisors -appId <arg> [-output <arg>]
- jstorm-yarn startSupervisors -appId <arg> [-output <arg>]
- jstorm-yarn startNimbus -appId <arg> [-output <arg>]
- jstorm-yarn stopNimbus -appId <arg> [-output <arg>]
- jstorm-yarn getStormConfig -appId <arg> [-output <arg>]
- jstorm-yarn setStormConfig -appId <arg> [-output <arg>]
- jstorm-yarn version
- jstorm-yarn shutdown
详见项目说明