python安装rabbitmq
1.安装:rabbitmq sudo apt install rabbitmq-server
2.添加用户:rabbitmqctl add_user username pwd rabbitmqctl set_user_tags jason administrator
3.设置权限:Rabbitmqctl set_permission -p / username“.” “.” “.*”
4.重启rabbitmq: /etc/init.d/rabbitmq-server restart
其他操作
1.查看队列: rabbitmqctl list_queues
2.重置rabbitmq:rabbitmqctl stop_app rabbitmqctl reset rabbitmqctl start_app
相关重要知识点整理
队列消息持久化:
①队列持久化很简单,只需要在服务端(produce)声明queue的时候添加一个参数:
channel.queue_declare(queue=‘shuaigaogao’, durable=True) # durable=True 持久化
②仅仅持久化队列是没有意义的,还需要多消息进行持久化
channel.basic_publish(exchange="",
routing_key=“shuaigaogao”, #queue的名字
body=“hello world”, #body是要发送的内容
properties=pika.BasicProperties(delivery_mode=2,) # make message persistent=>使消息持久化的特性)
③最后一步,在服务端队列消息都持久化了之后需要在客户端声明queue的时候也持久化
channel.queue_declare(queue=‘shuaigaogao’, durable=True)
这样就算再传递消息过程中,服务端的发生宕机,消息和队列也不会丢失
小结:RabbitMQ在服务端没有声明队列和消息持久化时,队列和消息是存在内存中的,服务端宕机了,队列和消息也不会保留。
服务端声明持久化,客户端想接受消息的话,必须也要声明queue时,也要声明持久化,不然的话,客户端执行会报错。
消息公平分发
消息公平分发就是消费者每次取一个消息,执行完了再取下一个。
其实现只需要在消费者端加上channel.basic_qos(prefetch_count=1)
消息确认机制
Productor Confirm模式
生产者将channel设置成confirm模式,一旦channel进入confirm模式,所有在该channel上面发布的消息都会被指派一个唯一的ID(Correlation Id从1开始)。一旦消息被投递到所匹配的队列之后,broker就会发送一个确认给生产者(包含消息的唯一ID)。这就使得生产者知道消息已经正确到达目的队列了,如果消息和队列是可持久化的,那么确认消息会将消息写入磁盘之后发出。
Consumer Confirm模式
队列中的消息分成了两个部分:一部分是等待投递给消费者的消息;一部分是已经投递给消费者,但是还没有收到消费者ack的消息。
如果处理一条消息需要几秒钟的时间,你可能会想,如果在处理消息的过程中,消费者服务器、网络、网卡出现故障挂了,那可能这条正在处理的消息或者任务就没有完成,就会失去这个消息和任务。为了确保消息或者任务不会丢失,RabbitMQ支持消息确认ACK。ACK机制是消费端从RabbitMQ收到消息并处理完成后,反馈给RabbitMQ,RabbitMQ收到反馈后才将此消息从队列中删除。如果一个消费者在处理消息时挂掉(网络不稳定、服务器异常、网站故障等原因导致频道、连接关闭或者TCP连接丢失等),那么他就不会有ACK反馈,RabbitMQ会认为这个消息没有正常消费,会将此消息重新放入队列中。
消息的ACK确认机制默认是开启的。ACK分为自动(auto)和手动(manul)2种模式:
自动确认模式,消息被认为是在发送后立即成功发送的,这种模式可以实现更高的吞吐量,但却是不安全的。如果在成功发送之前,消费者的TCP连接或通道关闭,服务器发送的消息将丢失。在使用自动确认模式时,需要考虑的另一件事是消费者过载。
实现代码
一对一
productor端:
import pika,time,sys
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel() #建立通道
channel.queue_declare(queue='task_queue', durable=True) #声明queue durable=True队列持久化
message = ' '.join(sys.argv[1:]) or "Hello World! %s" % time.time()
channel.basic_publish(exchange='', routing_key='task_queue', body=message,
properties=pika.BasicProperties( delivery_mode=2, #消息持久化 ))
print(" [x] Sent %r" % message)
connection.close()
consumer端:
import pika, time
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel()
channel.queue_declare(queue='task_queue', durable=True) #声明队列durable=True队列持久化
def callback(ch, method, properties, body):
print("method.delivery_tag", method.delivery_tag)
ch.basic_ack(delivery_tag=method.delivery_tag) #收到后向发送端确认执行完毕
channel.basic_consume('task_queue', callback) #True参数相当于不用确认,默认是False需要确认。
print(' [*] Waiting for messages. To exit press CTRL+C')
channel.start_consuming()
一对多广播
全员广播:fanout
productor端:
import pika,sys
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel()
channel.exchange_declare(exchange='logs', exchange_type='fanout') #绑定交换机logs
message = ' '.join(sys.argv[1:]) or "info: Hello World!"
channel.basic_publish(exchange='logs',
routing_key='',
body=message)
print(" [x] Sent %r" % message)
connection.close()
consumer端:
import pika
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel()
channel.exchange_declare(exchange='logs', exchange_type='fanout')
result = channel.queue_declare('',exclusive=True) # 不指定queue名字,rabbit会随机分配一个名字,exclusive=True会在使用此queue的消费者断开后,自动将queue删除
queue_name = result.method.queue
channel.queue_bind(exchange='logs', queue=queue_name) #绑定组,声明队列
def callback(ch, method, properties, body):
print(" [x] %r" % body)
channel.basic_consume(queue_name, callback, True)
channel.start_consuming()
分组广播:direct
productor端:
import pika, sys
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel()
channel.exchange_declare(exchange='direct_logs', exchange_type='direct')
severity = sys.argv[1] if len(sys.argv) > 1 else 'info' #发送到哪个组
message = ' '.join(sys.argv[2:]) or 'Hello World!'
channel.basic_publish(exchange='direct_logs',
routing_key=severity, #routing_key是发送的目标
body=message )
print(" [x] Sent %r:%r" % (severity, message))
connection.close()
comsumer端:
import pika
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel()
channel.exchange_declare(exchange='direct_logs', exchange_type='direct')
result = channel.queue_declare('', exclusive=True)
queue_name = result.method.queue
severities = ['info'] # 接受哪个组的信息,是一个列表
for severity in severities:
channel.queue_bind(exchange='direct_logs', #绑定组,声明队列
queue=queue_name,
routing_key=severity)
def callback(ch, method, properties, body):
print(" [x] %r:%r" % (method.routing_key, body))
channel.basic_consume(queue_name, callback,True)
channel.start_consuming()
话题广播:topic
productor端:
import pika, sys
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel()
channel.exchange_declare(exchange='topic_logs', exchange_type='topic')
routing_key = sys.argv[1] if len(sys.argv) > 1 else 'anonymous.info' #发送的目标,str类型
message = ' '.join(sys.argv[2:]) or 'Hello World!' #发送的信息
channel.basic_publish(exchange='topic_logs',
routing_key=routing_key,
body=message)
print(" [x] Sent %r:%r" % (routing_key, message))
connection.close()
comsumer端:
import pika
import sys
credentials = pika.PlainCredentials('jason', 'jason123')
connection = pika.BlockingConnection(pika.ConnectionParameters('127.0.0.1', credentials=
credentials))
channel = connection.channel()
channel.exchange_declare(exchange='topic_logs', exchange_type='topic')
result = channel.queue_declare('',exclusive=True)
queue_name = result.method.queue
binding_keys = ['#',mysql.*, *.django] #接受所有信息, 细分组以点为区分,值为列表
for binding_key in binding_keys:
channel.queue_bind(exchange='topic_logs',
queue=queue_name,
routing_key=binding_key)
def callback(ch, method, properties, body):
print(" [x] %r:%r" % (method.routing_key, body))
channel.basic_consume(queue_name, callback,True)
channel.start_consuming()