python redis队列_python实现简单的redis 队列 SIMPLE PYTHON QUEUE WITH REDIS

The following article shows how to useredisto build a simple multi-producer, multi-consumer Queue with an interface similar to thepython

standardlib queue. With this queue you can easily share data between multiple processes or offload time consumig calculations to multiple worker processes.

To store the data we use theredis list data type. Redis Lists stores simple strings sorted by insertion order.

The following redis commands are used:

The implementation uses theredis-pylibrary to talk to the server.

import redis

class RedisQueue(object):

"""Simple Queue with Redis Backend"""

def __init__(self, name, namespace='queue', **redis_kwargs):

"""The default connection parameters are: host='localhost', port=6379, db=0"""

self.__db= redis.Redis(**redis_kwargs)

self.key = '%s:%s' %(namespace, name)

def qsize(self):

"""Return the approximate size of the queue."""

return self.__db.llen(self.key)

def empty(self):

"""Return True if the queue is empty, False otherwise."""

return self.qsize() == 0

def put(self, item):

"""Put item into the queue."""

self.__db.rpush(self.key, item)

def get(self, block=True, timeout=None):

"""Remove and return an item from the queue.

If optional args block is true and timeout is None (the default), block

if necessary until an item is available."""

if block:

item = self.__db.blpop(self.key, timeout=timeout)

else:

item = self.__db.lpop(self.key)

if item:

item = item[1]

return item

def get_nowait(self):

"""Equivalent to get(False)."""

return self.get(False)

使用方法：

>>> from RedisQueue import RedisQueue

>>> q = RedisQueue('test')

>>> q.put('hello world')

Now if we have a look at the redis database with theredis-cliclient

it shows the expected results:

redis 127.0.0.1:6379> keys *

1) "queue:test"

redis 127.0.0.1:6379> type queue:test

list

redis 127.0.0.1:6379> llen queue:test

(integer) 1

redis 127.0.0.1:6379> lrange queue:test 0 1

1) "hello world"

We can get the item from a different script with:

>>> from RedisQueue import RedisQueue

>>> q = RedisQueue('test')

>>> q.get()

'hello world'

A subsequent call ofq.get()will block until anotherone puts a new item into the Queue.

The next step would be to an endoder/decoder (e.gpython-json) to the Queue so that you are not limited to send strings.

There alredy exists the nice and simplehotqueuelibrary which has the same interface as the above example and provides encoding/decoding.

Other mentionable queue implementations with a redis backend are:

FRom：http://peter-hoffmann.com/2012/python-simple-queue-redis-queue.html

分享到：

2012-09-18 23:44

浏览 1137

评论