Tornado gen简介

[size=small]
[b]Contains:
[/b]
[list]
[*]1、iterator
[*]2、generator
[*]3、yield expressions
[*]4、tornado.gen
[/list]
[b]一、迭代器
[/b]
[quote]
Python supports a concept of iteration over containers. This is implemented using two distinct methods; these are used to allow user-defined classes to support iteration. Sequences, described below in more detail, always support the iteration methods.
Iteration is a process implying iterables (implementing the __iter__() method) and iterators (implementing the __next__() method). Iterables are any objects you can get an iterator from. Iterators are objects that let you iterate on iterables.[/quote]
The iterator objects themselves are required to support the following two methods, which together form the iterator protocol:

1、iterator.__iter__()
Return the iterator object itself. This is required to allow both containers and iterators to be used with the for and in statements. This method corresponds to the tp_iter slot of the type structure for Python objects in the Python/C API.
2、iterator.next()
Return the next item from the container. If there are no further items, raise the StopIteration exception. This method corresponds to the tp_iternext slot of the type structure for Python objects in the Python/C API.

[b]二、生成器
[/b]
[quote] Python’s generators provide a convenient way to implement the iterator protocol. If a container object’s __iter__() method is implemented as a generator, it will automatically return an iterator object (technically, a generator object) supplying the __iter__() and next() methods. More information about generators can be found in the documentation for the yield expression.
Generators are iterables, but you can only read them once. It's because they do not store all the values in memory[/quote]
简单的讲，Generator在需要时返回中间值，能够保存当前的状态，等待下一次的返回要求。
迭代到下一次的调用时，所使用的参数都是第一次所保留下的，即是说，在整个所有函数调用的参数都是第一次所调用时保留的，而不是新创建的
Generators functions allow you to declare a function that behaves like an iterator, i.e. it can be used in a for loop.
[quote] The performance improvement from the use of generators is the result of the lazy (on demand) generation of values, which translates to lower memory usage. Furthermore, we do not need to wait until all the elements have been generated before we start to use them. This is similar to the benefits provided by iterators, but the generator makes building iterators easy.[/quote]
This can be illustrated by comparing the range and xrange built-ins of Python 2.x.
Both range and xrange represent a range of numbers, and have the same function signature, but range returns a list while xrange returns a generator (at least in concept; the implementation may differ).
sum_of_numbers = sum(range(1000000))
sum_of_numbers = sum(xrange(1000000))
When we use range we build a 1,000,000 element list in memory and then find its sum. This is a waste, considering that we use these 1,000,000 elements just to compute the sum.
This waste becomes more pronounced as the number of elements (our n) becomes larger, the size of our elements become larger, or both.
On the other hand, when we use xrange, we do not incur the cost of building a 1,000,000 element list in memory. The generator created by xrange will generate each number, which sum will consume to accumulate the sum.
In the case of the "range" function, using it as an iterable is the dominant use-case, and this is reflected in Python 3.x, which makes the range built-in return a generator instead of a list.
[quote] Note: a generator will provide performance benefits only if we do not intend to use that set of generated values more than once.
[/quote]
在实现上，generator并不是生成一个列表，然后再由for对元素一个一个进行处理，而是一次只返回一个元素（用yield语句）并保存generator的状态，等下次被调用时再从当前状态往下执行，这样可以省却保存整个大列表的存储代价。
[b]什么时候适合使用生成器：
[/b][list]
[*]1、You don't need to read the values twice.
[*]2、You can have a lot of children and you don't want them all stored in memory.
[/list]
[b]三、yield expressions
[/b]
[quote] The yield expression is only used when defining a generator function, and can only be used in the body of a function definition. Using a yield expression in a function definition is sufficient to cause that definition to create a generator function instead of a normal function.
When a generator function is called, it returns an iterator known as a generator iterator, or more commonly, a generator. The body of the generator function is executed by calling the generator’s next() method repeatedly until it raises an exception.
When a yield statement is executed, the state of the generator is frozen and the value of expression_list is returned to next()‘s caller. By “frozen” we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack: enough information is saved so that the next time next() is invoked, the function can proceed exactly as if the yield statement were just another external call.
[/quote]
Yield is a keyword that is used like return, except the function will return a generator.
The first time your function will run, it will run from the beginning until it hits yield, then it'll return the first value of the loop. Then, each other call will run the loop you have written in the function one more time, and return the next value, until there is no value to return.
yield简单说来就是一个生成器，生成器是这样一个函数，它记住上一次返回时咋函数体中的位置。对生成器函数的第二次(或第n次)调用跳转至该函数中间，而上次调用的所有局部变量都保持不变。
可以说带有yield的函数在python中被称之为generator.
一个带有 yield 的函数就是一个 generator，它和普通函数不同，生成一个 generator 看起来像函数调用，但不会执行任何函数代码，直到对其调用 next()（在 for 循环中会自动调用 next()）才开始执行。虽然执行流程仍按函数的流程执行，但每执行到一个 yield 语句就会中断，并返回一个迭代值，下次执行时从 yield 的下一个语句继续执行。看起来就好像一个函数在正常执行的过程中被 yield 中断了数次，每次中断都会通过 yield 返回当前的迭代值。

[b]四、tornado gen
[/b]
[quote] tornado.gen is a generator-based interface to make it easier to work in an asynchronous environment. Code using the gen module is technically asynchronous, but it is written as a single generator instead of a collection of separate functions.[/quote]
decorator:
[b]tornado.gen.engine(func)
[/b]
Decorator for asynchronous generators.
Any generator that yields objects from this module must be wrapped in this decorator. The decorator only works on functions that are already asynchronous. For RequestHandler get/post/etc methods, this means that both the tornado.web.asynchronous and tornado.gen.engine decorators must be used (for proper exception handling, asynchronous should come before gen.engine). In most other cases, it means that it doesn’t make sense to use gen.engine on functions that don’t already take a callback argument.
[b]Yield points:
[/b]Instances of the following classes may be used in yield expressions in the generator.
[b]class tornado.gen.Task(func, *args, **kwargs)
[/b]
Runs a single asynchronous operation.
Takes a function (and optional additional arguments) and runs it with those arguments plus a callback keyword argument. The argument passed to the callback is returned as the result of the yield expression.
A Task is equivalent to a Callback/Wait pair (with a unique key generated automatically)

[b]class tornado.gen.Callback(key)
[/b]
[b]class tornado.gen.Wait(key)
[/b]
[b]class tornado.gen.WaitAll(keys)
[/b]示例：

class GenAsyncHandler(RequestHandler):
    @asynchronous
    @gen.engine
    def get(self):
        http_client = AsyncHTTPClient()
        response = yield gen.Task(http_client.fetch, "http://example.com")
        do_something_with_response(response) 
        self.render("template.html")

参考资料：
http://wiki.python.org/moin/Generators
http://linuxgazette.net/100/pramode.html
http://www.python.org/dev/peps/pep-0234/
http://www.python.org/dev/peps/pep-0255/
http://www.pythonclub.org/python-basic/yield
http://docs.python.org/2/library/stdtypes.html
http://zouyesheng.com/generator-for-async.html
http://www.tornadoweb.org/documentation/gen.html
http://blog.xiaogaozi.org/2012/09/21/understanding-tornado-dot-gen/
http://stackoverflow.com/questions/7883962/where-to-use-yield-in-python-best
http://stackoverflow.com/questions/231767/the-python-yield-keyword-explained
[/size]