python callbacks_Python扭曲：迭代器和yield/inlineCallbacks

你是对的，你不能表达你想要在缓存中表达什么。 inlineCallbacks装饰器不会让你有一个返回迭代器的函数。如果您使用它来装饰一个函数，那么结果就是一个总是返回Deferred的函数。这就是它的意思。

使这种困难的一部分是迭代器对于异步代码不能很好地工作。如果有一个Deferred涉及到生成迭代器的元素，那么迭代器中出现的元素将是Deferreds。

你可能会这样做：

@inlineCallbacks

def process_work():

for element_deferred in some_jobs:

element = yield element_deferred

work_on(element)

这可以工作，但它看起来特别奇怪。由于生成器只能向其调用者发放(而不是例如对其调用者的调用者)，所以some_jobs迭代器不能对此做任何事情;只有在process_work中的词法代码才能产生一个延迟到inlineCallbacks提供的蹦床来等待。

如果您不介意这种模式，那么我们可以将您的代码映射成像：

from twisted.internet.task import deferLater

from twisted.internet.defer import inlineCallbacks, returnValue

from twisted.internet import reactor

class cacheiter(object):

def __init__(self, cached):

self._cached = iter(cached.items())

self._remaining = []

def __iter__(self):

return self

@inlineCallbacks

def next(self):

# First re-fill the list of synchronously-producable values if it is empty

if not self._remaining:

for name, value in self._cached:

# Wait on this Deferred to determine if this cache item should be included

if (yield check_condition(name, value)):

# If so, put all of its values into the value cache so the next one

# can be returned immediately next time this method is called.

self._remaining.extend([(name, k, v) for (k, v) in value.items()])

# Now actually give out a value, if there is one.

if self._remaining:

returnValue(self._remaining.pop())

# Otherwise the entire cache has been visited and the iterator is complete.

# Sadly we cannot signal completion with StopIteration, because the iterator

# protocol isn't going to add an errback to this Deferred and check for

# StopIteration. So signal completion with a simple None value.

returnValue(None)

@inlineCallbacks

def process_chunk(myiter, num):

for i in xrange(num):

nextval = yield myiter.next()

if nextval is None:

# The iterator signaled completion via the special None value.

# Processing is complete.

returnValue(True)

# Otherwise process the value.

yield some_processing(nextval)

# Indicate there is more processing to be done.

returnValue(False)

def sleep(sec):

# Simple helper to delay asynchronously for some number of seconds.

return deferLater(reactor, sec, lambda: None)

@inlineCallbacks

def process_loop(cached):

myiter = cacheiter(cached)

while True:

# Loop processing 10 items from myiter at a time, until process_chunk signals

# there are no values left.

result = yield process_chunk(myiter, 10)

if result:

print 'All done'

break

print 'More left'

# Insert the 5 second delay before starting on the next chunk.

yield sleep(5)

d = process_loop(cached)

然而，您可能需要采取的另一种方法是使用twisted.internet.task.cooperate。合作采取迭代器并消耗它，假设消耗它可能是昂贵的，并且通过多个反应堆迭代分解工作。从上面来看cacheiter的定义：

from twisted.internet.task import cooperate

def process_loop(cached):

finished = []

def process_one(value):

if value is None:

finished.append(True)

else:

return some_processing(value)

myiter = cacheiter(cached)

while not finished:

value_deferred = myiter.next()

value_deferred.addCallback(process_one)

yield value_deferred

task = cooperate(process_loop(cached))

d = task.whenDone()