kafka安装启动

1、安装jdk（依赖）

root@/data#tar -zxvf jdk-8u141-linux-x64.tar.gz
root@/data#mv jdk1.8.0_141/ /usr/lib/jdk
root@/data#vim /etc/profile #后面加上
export JAVA_HOME=/usr/lib/jdk
export JRE_HOME=${JAVA_HOME}/jre  
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib  
export  PATH=${JAVA_HOME}/bin:$PATH
export PATH=/usr/local/bin:$PATH
root@/data#source /etc/profile
root@/data# java -version	#有这个输出表示jdk安装成功
java version "1.8.0_141"
Java(TM) SE Runtime Environment (build 1.8.0_141-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.141-b15, mixed mode)

2、安装kafka（不需要安装，解包即可）

root@/data#tar -xzf kafka_2.12-1.1.1.tgz
root@/data#cd kafka_2.12-1.1.0
root@/data#mv kafka_2.12-1.1.0/ /usr/local/kafka/
root@/data#cd /usr/local/kafka/config/
root@/data#vim server.properties 
#个人建议默认的1，如果改成2以上，创建的topic会默认有num个分区，
#到时候处理代码的时候忘记了以为会是bug
num.partitions=1				
log.dirs=/usr/local/kafka/logs	#修改日志目录 其他不变
	
root@/data#vim zookeeper.properties	#其他不变
dataDir=/usr/local/kafka/zookeeper/data
dataLogDir=/usr/local/kafka/zookeeper/logs

添加启动文件：

root@/data#vim /usr/local/kafka/bin/kafkaStart.sh 
##文件内容如下
#!/bin/bash
#启动zookeeper
/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties &
sleep 3  #默默等3秒后执行 
#启动kafka
/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties &	

启动kafka：

root@/data#/usr/local/kafka/bin/kafkaStart.sh 

3、命令行查看kafka相关信息

消费组相关：

消费组列表
/usr/local/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list
myConsumerGroup1
myConsumerGroup

某个消费组的详细信息
root@~#/usr/local/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group myConsumerGroup
GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
myConsumerGroup shop            0          202             202             0               -               -               -

topic主题相关：

/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
创建主题类别和分区
/usr/local/kafka/bin/kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 4 --topic student
将分区数有4修改为8
/usr/local/kafka/bin/kafka-topics.sh --alter --zookeeper 127.0.0.1:2181 --topic student --partitions 8

//分区数只能升不能降（从2个分区修改到1个分区）
root@/media/wwwroot/mynode/kafka#/usr/local/kafka/bin/kafka-topics.sh --alter --zookeeper 127.0.0.1:2181 --topic LIST-microOrder --partitions 1
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Error while executing topic command : The number of partitions for a topic can only be increased. Topic LIST-microOrder currently has 2 partitions, 1 would not be an increase.
[2020-04-15 13:47:37,716] ERROR org.apache.kafka.common.errors.InvalidPartitionsException: The number of partitions for a topic can only be increased. Topic LIST-microOrder currently has 2 partitions, 1 would not be an increase.

查看主题列表
/usr/local/kafka/bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --list
high
high-part
high-part-test
high-part-test3
shop
student

查看某个主题的详细信息
/usr/local/kafka/bin/kafka-topics.sh --describe --zookeeper 127.0.0.1:2181 --topic student
Topic:student   PartitionCount:4        ReplicationFactor:1     Configs:
   Topic: student  Partition: 0    Leader: 0       Replicas: 0     Isr: 0
   Topic: student  Partition: 1    Leader: 0       Replicas: 0     Isr: 0
   Topic: student  Partition: 2    Leader: 0       Replicas: 0     Isr: 0
   Topic: student  Partition: 3    Leader: 0       Replicas: 0     Isr: 0