Python yield expression (generator)

1. When a function has a "yield" expression, it is a generator.
2. A generator doesn't run until it is called with next(g) or g.next(). 
3. The first time to call a function which has "yield expression", it is returning a generator object, say gen = echo([11,...]) below,  gen is an generator object.
4. Every time, gen.next (or next(gen)) is called, 
   • If it is the first time to call "next", the generator executes until it reaches "yield expression". When the generator hits "yield expression", the value just behind the keyword "yield" is returned to caller. For eg, return "val" in the "yield val" expression in the following code. Please notice that the "yield expression" itself doesn't return (UNTIL "next/send" etc is called again against this generator)
   • If it is not the first time to call "next", the generator resumes to execute just at the point where the "yield expression" paused at the very last time. At this very time, the "yield expression" returns (None by default). The generator executes until it reaches the next "yield" expression.
5. "yield" expression itself usually returns "None", unless we send a explicit value to the "yield" expression by calling "send(val)". For, eg, val = (yield val), val on the left side usually have None value, but if we call this generator with send(20), then val on the left side will have value with 20.
6. When a generator is closed (call its "close()" method), we can't call next on it anymore, it will throw StopIteration exception. When "close()" is called against a generator, an "GeneratorExit" exception will be thrown inside the generator, which will terminate the iteration. So there is one last chance to do post processing by catching this exception. Take the following code for eg, before the "echo" generator is exhausted, if "close()" is called, "GeneratorExit" exception will be thrown automatically, and "GeneratorExit exception thrown..." will be printed.
   
7. When a generator is garbage collected, "close()" is called automatically.
8. We can explicitly feed an exception to a generator (to terminate the iteration maybe) by calling "throw()" method. The exception is raised by the "yield expression" where the generator’s execution is paused.
9. Always remember to put parentheses around a "yield expression" when you’re doing something with the returned value. (Can you tell the difference between "val = yield val + 10" and "val = (yield val) + 10" ?)
10. Generator has great flexibility and memory efficiency if we just loop through some iterable and process one item at a time.
11. Generator expression has more memory efficiency than list comprehension 
    for x, y in ((x, y) for x in a for y in b): print x, y  # generator expression
    for x, y in [(x, y) for x in a for y in b]: print x, y  # list comprehension

def echo(arr):

    print('Execution start when next() is called for the first time.');
    try:
        siz = len(arr)
        i = 0
        val = None
        while True:
            try:
                if i < siz:
                    print ('before yielding...{0}'.format(val))
                    val = arr[i]
                    # When hitting the following yield expr, "val" is returned to the caller, 
                    # but the expr itself is not returned until "next/again" is called again
                    val = (yield val)
                    print ('after yielding: {0}'.format(val))
                    i += 1
                else:
                    raise StopIteration()
            except Exception as e:
                print('Other exception thrown %s' %str(e))
                raise e
            except GeneratorExit as e:
                # Please rethrow this exception or rethrow StopIteration exception
                # Otherwise "runtime error" will be triggered.
                # Note GeneratorExit is not an error
                print('GeneratorExit exception thrown %s' %str(e))
                raise e
    finally:
        # When "close()" is called or the generator garbage-collected, 
        # this section of code is executed 
        print("Don't forget to clean up when close() is called.")


gen = echo([11, 12, 13, 14])
print type(gen)
print(next(gen))
#print(next(gen))
print(gen.send(20))
#gen.throw(TypeError, 'Spam')
gen.close()
print(next(gen))
#for i in gen:
#    print(i)


def counter(start_at=0):
    count = start_at
    while True:
        val = (yield count)
        if val is not None:
            count = val
        else:
            count += 1


#count = counter(5)
#print count.next()
#print count.send(9)
#print count.next()
#print count.next()

An other example:
---
def paragraphs(lines, is_separator=str.isspace, joiner=''.join):
    paragraph = [ ]
    for line in lines:
        if is_separator(line):
            if paragraph:
                yield joiner(paragraph)
                paragraph = del [:]
        else:
            paragraph.append(line)
    if paragraph:
        yield joiner(paragraph)

if __name__ == '__main__':
    text = 'a first\nparagraph\n\nand a\nsecond one\n\n'
    for p in paragraphs(text.splitlines(True)): 
        print repr(p)

Yet another example:
---
def fetchsome(cursor, arraysize=1000):
    ''' A generator that simplifies the use of fetchmany '''
    while True:
        results = cursor.fetchmany(arraysize)

    if not results: break
        for result in results:
            yield result