Flume搭建与Kafka分布式搭建

1. Flume搭建：解压，重命名，环境变量

   1.	tar -zxvf
   2.	mv 
   3.	vi ~/.bash_profile
       export FLUME_HOME=/usr/local/src/flume
       export PATH=$FLUME_HOME/bin:$PATH
   4.	刷新环境变量	source ~/.bash_profile

2. 修改flume-env.sh文件

   1.	cd hive/conf 
   2.	cp flume-env.sh.template flume-env.sh
   3.	vi    flume-env.sh
       修 export JAVA_HOME=/usr/local/src/jdk

3. 测试

    flume-ng

4. Kafka分布式搭建：解压，重命名，环境变量

   1.	tar -zxvf
   2.	mv 
   3.	vi ~/.bash_profile
       export KAFKA_HOME=/usr/local/src/kafka
       export PATH=$KAFKA_HOME/bin:$PATH
   4.	刷新环境变量	source ~/.bash_profile

5. 修改server.properties文件

   1.	cd kafka/conf 
   3.	vi   server.properties
       修 broker.id=0(slave1和slave2分别改为1,2)
          listeners=PLAINTEXT://master:9092(slave1和slave2分别改为slave1,slave2)
          advertised.listeners=PLAINTEXT://master:9092(slave1和slave2分别改为slave1,slave2)
          log.dirs=/usr/local/src/kafka/kafka-logs  
          zookeeper.connect=master:2181,slave1:2181,slave2:2181

6. 分发文件

   1.	scp -r kafka slave1:/usr/local/src
   2.	scp -r kafka slave2:/usr/local/src
   3.	scp ~/.bash_profile slave1:~
   4.	scp ~/.bash_profile slave2:~

7. 守护进程方式启动kafka

   nohup kafka-server-start.sh config/server.properties > console.log 2>&1 &(slave1,slave2)