Kafka作为消息中间件是非常受欢迎的,对应Python库名为kafka-python;
import json
import traceback
from kafka.errors import kafka_errors
from kafka import KafkaProducer, KafkaConsumer
producer = KafkaProducer(bootstrap_servers=['localhost:9092'],
key_serializer=lambda key: json.dumps(key).encode(),
value_serializer=lambda value: json.dumps(value).encode())
for i in range(3):
msg = 'message ' + str(i)
future = producer.send('kafka_topic', key='count', value=msg)
print(f"send {msg}")
try:
future.get(timeout=10) # 看是否发送成功
except kafka_errors:
traceback.print_exc()
上段为Kafka生产者,下段对应Kafka消费者,KafkaConsumer会进入持续接收状态;
consumer = KafkaConsumer('kafka_topic', bootstrap_servers=['localhost:9092'], group_id='test')
for msg in consumer:
print(f"receive, key: {json.loads(msg.key.decode())}, value: {json.loads(msg.value.decode())}")