Ubuntu下kafka使用总结

安装运行环境

apt-get install openjdk-8-jre
pip3 install kafka

官网下载kafka 略，直接下从官网下就好了，不需要安装
官网下载地址
启动kafka

bin/zookeeper-server-start.sh config/zookeeper.properties
bin/kafka-server-start.sh config/server.properties

启动生产者：
1.从终端启动

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
#开启了一个名为test的topic

2.从python3启动

>>> from kafka import KafkaProducer
>>> producer = KafkaProducer(bootstrap_servers='localhost:9092')
>>> for _ in range(100):
...     producer.send('test',b'some_message_bytes')  #指定发送给名为test的topic

启动消费者：
1.从终端启动

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

2.从python3启动

>>> from kafka import KafkaConsumer
>>> consumer=KafkaConsumer('test',bootstrap_servers=['localhost:9092'])
>>> for msg in consumer:
...   recv = "%s:%d:%d: key=%s value=%s" %(msg.topic,msg.partition,msg.offset,msg.key,msg.value)
...   print(recv)
...
test:0:222: key=None value=b'some_message_bytes'
test:0:223: key=None value=b'some_message_bytes'
test:0:224: key=None value=b'some_message_bytes'
test:0:225: key=None value=b'some_message_bytes'
。。。。

测试可用，要先启动消费者再启动生产者，topic一定要一致
部分参考来源
kafka与zookeeper简介
Kafka中bootstrap-server、broker-list和zookeeper的区别
有空可以看一下极客学院的课程