yield in Python

To understand what yield does, you must understand what generators are. And before generators come iterables.

1. Iterable

>>> mylist = [x*x for x in range(3)]
>>> for i in mylist:
...    print(i)
0
1
4

Everything you can use "for... in..." on is an iterable: lists, strings, files... These iterables are handy 
because you can read them as much as you wish, but you store all the values in memory and it's not always what you want when you have a lot of values.

Iteration is a process implying iterables (implementing the __iter__() method) and iterators (implementing the __next__()  method). Iterables are any objects you can get an iterator from. Iterators are objects that let you iterate on iterables.

2. Generator:

Generators are iterators, but you can only iterate over them once. 
It's because they do not store all the values in memory, they generate the values on the fly:

>>> mygenerator = (x*x for x in range(3))
>>> for i in mygenerator:
...    print(i)
0
1
4

3. Yield

Yield is a keyword that is used like return, except the function will return a generator.

>>> def createGenerator():
...    mylist = range(3)
...    for i in mylist:
...        yield i*i
...
>>> mygenerator = createGenerator() # create a generator
>>> print(mygenerator) # mygenerator is an object!
<generator object createGenerator at 0xb7555c34>
>>> for i in mygenerator:
...     print(i)
0
1
4

To master yield, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, 

The first time the for calls the generator object created from your function, it will run the code in your function from the beginning until it hits yield, 
then it'll return the first value of the loop. Then, each other call will run the loop you have written in the function one more time, and return the next value, until there is no value to return.

>>> class Bank(): # let's create a bank, building ATMs
...    crisis = False
...    def create_atm(self):
...        while not self.crisis:
...            yield "$100"
>>> hsbc = Bank() # when everything's ok the ATM gives you as much as you want
>>> corner_street_atm = hsbc.create_atm()
>>> print(corner_street_atm.next())
$100
>>> print(corner_street_atm.next())
$100
>>> print([corner_street_atm.next() for cash in range(5)])
['$100', '$100', '$100', '$100', '$100']
>>> hsbc.crisis = True # crisis is coming, no more money!
>>> print(corner_street_atm.next())
<type 'exceptions.StopIteration'>
>>> wall_street_atm = hsbc.create_atm() # it's even true for new ATMs
>>> print(wall_street_atm.next())
<type 'exceptions.StopIteration'>
>>> hsbc.crisis = False # trouble is, even post-crisis the ATM remains empty
>>> print(corner_street_atm.next())
<type 'exceptions.StopIteration'>
>>> brand_new_atm = hsbc.create_atm() # build a new one to get back in business
>>> for cash in brand_new_atm:
...    print cash
$100
$100
$100

There are a few key ideas I hope you take away from this discussion:

• generators are used to generate a series of values
• yield is like the return of generator functions
• The only other thing yield does is save the "state" of a generator function
• A generator is just a special type of iterator
• Like iterators, we can get the next value from a generator using next()
  • for gets values by calling next() implicitly

4. example

'''
 Reservoir Sampling implementation
'''
from random import Random

def RandomSelect(knum, rand=None):
    ''' (int, func) -> list

    Reservoir Sampling implementation
    >>> myList = [1,2,3,4,5,6,7,8] 
    >>> ge = RandomSelect(3)
    >>> ge.next()
    >>> for v in myList:
            cr.send(v)
    '''
    selection = None
    k_elems_list = []
    count = 0
    if rand is None:
        rand = Random()
    while True:
	try:
	    item = yield selection
	except 
	    break
        if len(k_elems_list) < knum:
            k_elems_list.append(item)
        elif rand.randint(0, count) == 0:
            index = rand.randint(0, knum-1)
            k_elems_list.pop(index)
            k_elems_list.insert(index, item)
        count += 1
    print k_elems_list

if __name__ == '__main__':
    myList = [1,2,3,4,5,6,7,8]
    cr = RandomSelect(3);
    cr.next() # advance to the yield statement, otherwise I can't call send
	for val in myList:
		cr.send(val)
	cr.close()

import random

def get_data():
    """Return 3 random integers between 0 and 9"""
    return random.sample(range(10), 3)

def consume():
    """Displays a running average across lists of integers sent to it"""
    running_sum = 0
    data_items_seen = 0

    while True:
        data = yield
        data_items_seen += len(data)
        running_sum += sum(data)
        print('The running average is {}'.format(running_sum / float(data_items_seen)))

def produce(consumer):
    """Produces a set of values and forwards them to the pre-defined consumer
    function"""
    while True:
        data = get_data()
        print('Produced {}'.format(data))
        consumer.send(data)
        yield

if __name__ == '__main__':
    consumer = consume()
    consumer.send(None)
    producer = produce(consumer)

    for _ in range(10):
        print('Producing...')
        next(producer)

To solve the N-queens question. 

import random
"""
N-queens problem

solve 8 queens problem with generator method - brute force
"""

def conflict(state, nextX):
    """(tuple, int) -> bool

    Return whether the current position can be put one queen or not

    >>> conflict((1,3,0), 1)
    False
    >>> conflict((1,3,0), 2)
    True
    """
    nextY = len(state)
    for i in xrange(nextY):
        if abs(state[i] - nextX) in (0, nextY - i):
            return True
    return False

def queens(num=8, state=()):
    """(int, tuple) -> tuple

    Return the list of the N queens position

    >>> list(queen(3))
    []
    >>> list(queen(4))
    [(1, 3, 0, 2), (2, 0, 3, 1)]
    """
    for pos in xrange(num):
        if not conflict(state, pos):
            if len(state) == num - 1:
                yield (pos,)
            else:
                for result in queens(num, state+(pos,)):
                    yield (pos,)+result


def pretty_print(solution):
    """(function) -> NoneType
    
    """
    def line(pos, length=len(solution)):
        return '- ' * pos + 'X ' + '- ' * (length - pos - 1)
    for pos in solution:
        print line(pos)

if __name__ == '__main__':
    pretty_print(random.choice(list(queens(8))))

Ref:

http://www.jeffknupp.com/blog/2013/04/07/improve-your-python-yield-and-generators-explained/