一、环境安装配置
1、去官网下载kafka
我下载的版本是 kafka_2.11-0.10.0.1.tgz,下面的实例也是基于该版本。
2、解压安装
tar -xzf kafka_2.11-0.10.0.1.tgz
mv kafka_2.11-0.10.0.1 /root
3、修改配置文件
cd /root/kafka_2.11-0.10.0.1/config
cp server.properties server1.properties
cp server.properties server2.properties
cp server.properties server3.properties
修改配置中的三个参数如下:
server1.properties
broker.id=1
listeners=PLAINTEXT://:9092
log.dirs=/tmp/kafka-logs-1
server2.properties
broker.id=2<pre name="code" class="plain"><span style="font-family: Arial, Helvetica, sans-serif;"></span><pre name="code" class="plain">listeners=PLAINTEXT://:9094
log.dirs=/tmp/kafka-logs-2
server3.properties
broker.id=3
listeners=PLAINTEXT://:9094
log.dirs=/tmp/kafka-logs-3
4、启动脚本kafkacluster编写
bin/zookeeper-server-start.sh config/zookeeper.properties &
sleep 3s
bin/kafka-server-start.sh config/server1.properties &
bin/kafka-server-start.sh config/server2.properties &
bin/kafka-server-start.sh config/server3.properties &
启动kafka集群只需要执行./kafkacluster
另外附上一个kill kafka脚本
#!/bin/sh
PIDS=$(ps ax | grep -i 'kafka\.Kafka' | grep java | grep -v grep | awk '{print $1}')
if [ -z "$PIDS" ]; then
echo "No kafka server to stop"
else
for pid in $PIDS
do
echo "$pid"
kill -s 9 $pid
done
fi
sleep 3s
PIDS=$(ps ax | grep java | grep -i QuorumPeerMain | grep -v grep | awk '{print $1}')
if [ -z "$PIDS" ]; then
echo "No zookeeper server to stop"
exit 1
else
kill -s TERM $PIDS
fi
5、创建topic
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 3 --topic test
二、代码编写
pom.xml
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<spring.version>3.2.14.RELEASE</spring.version>
<kafka.version>0.10.0.1</kafka.version>
</properties>
<dependencies>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.12</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.12</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${spring.version}</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.11</artifactId>
<version>${kafka.version}</version>
<scope>compile</scope>
<exclusions>
<exclusion>
<artifactId>jmxri</artifactId>
<groupId>com.sun.jmx</groupId>
</exclusion>
<exclusion>
<artifactId>jms</artifactId>
<groupId>javax.jms</groupId>
</exclusion