第二十~章 Functions and Generators

第二十章 Comprehensions and Generations

1.List Comprehensions and Functional Tools

1.1 List Comprehensions Versus map

>>> res = list(map(ord, 'spam')) # Apply function to sequence (or other)
>>> res
[115, 112, 97, 109]



>>> res = [ord(x) for x in 'spam'] # Apply expression to sequence (or other)
>>> res
[115, 112, 97, 109]

>>> [x for x in range(5) if x % 2 == 0]
[0, 2, 4]



 >>> list(filter((lambda x: x % 2 == 0), range(5)))
[0, 2, 4]

>>> [x ** 2 for x in range(10) if x % 2 == 0]
[0, 4, 16, 36, 64]


>>> list( map((lambda x: x**2), filter((lambda x: x % 2 == 0), range(10))) )
[0, 4, 16, 36, 64]

[ expression for target1 in iterable1 if condition1
for target2 in iterable2 if condition2 ...
for targetN in iterableN if conditionN ]

This same syntax is inherited by set and dictionary comprehensions as well as the generator expressions

>>> M = [[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]

>>> [col + 10 for row in M for col in row] # Assign to M to retain new value
[11, 12, 13, 14, 15, 16, 17, 18, 19]
>>> [[col + 10 for col in row] for row in M]
[[11, 12, 13], [14, 15, 16], [17, 18, 19]]

2.Generator Functions and Expressions

• Generator functions (available since 2.3) are coded as normal def statements, but
  use yield statements to return results one at a time, suspending and resuming their
  state between each.
• Generator expressions (available since 2.4) are similar to the list comprehensions
  of the prior section, but they return an object that produces results on demand
  instead of building a result list.
  

2.1 Generator Functions: yield Versus return

Generator functions are compiled specially into
an object that supports the iteration protocol. And when called, they don’t return a
result: they return a result generator that can appear in any iteration context.

>>> def gensquares(N):
	for i in range(N):
		yield i ** 2

		
>>> x = gensquares(4)
>>> x
<generator object gensquares at 0x0000000003284480>
>>> next(x)
0
>>> next(x)
1
>>> next(x)
4
>>> x.__next__()
9
>>> x.__next__()
Traceback (most recent call last):
  File "<pyshell#88>", line 1, in <module>
    x.__next__()
StopIteration

>>> y = gensquares(5) # Returns a generator which is its own iterator
>>> iter(y) is y # iter() is not required: a no-op here
True

generators can be better in terms of both memory use and performance in larger programs

generators can provide a simpler alternative to manually saving the state between iterations in class objects

Extended generator function protocol: send versus next

>>> def gen():
        for i in range(10):
            X = yield i
            print(X)
>>> G = gen()
>>> next(G) # Must call next() first, to start generator
0
>>> G.send(77) # Advance, and send value to yield expression
77
1
>>> G.send(88)
88
2
>>> next(G) # next() and X.__next__() send None
None
3

Generator Expressions: Iterables Meet Comprehensions

>>> [x ** 2 for x in range(4)] # List comprehension: build a list
[0, 1, 4, 9]
>>> (x ** 2 for x in range(4)) # Generator expression: make an iterable
<generator object <genexpr> at 0x00000000029A8288>


>>> list(x ** 2 for x in range(4)) # List comprehension equivalence
[0, 1, 4, 9]

>>> G = (x ** 2 for x in range(4))
>>> iter(G) is G # iter(G) optional: __iter__ returns self
True
>>> next(G) # Generator objects: automatic methods
0
>>> next(G)
1
>>> next(G)
4
>>> next(G)
9
>>> next(G)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
>>> G
<generator object <genexpr> at 0x00000000029A8318>

>>> for num in (x ** 2 for x in range(4)): # Calls next() automatically
print('%s, %s' % (num, num / 2.0))
0, 0.0
1, 0.5
4, 2.0
9, 4.5

>>> ''.join(x.upper() for x in 'aaa,bbb,ccc'.split(','))  #doesn’t require extra parentheses around the generator.
'AAABBBCCC'                                       #parentheses are not required around a generator expression  
                                                  #that is the sole item already enclosed in parentheses used for other purposes
                                                  #like those of a function call.
>>> sum(x ** 2 for x in range(4)) # Parens optional
14
>>> sorted(x ** 2 for x in range(4)) # Parens optional
[0, 1, 4, 9]
>>> sorted((x ** 2 for x in range(4)), reverse=True) # Parens required
[9, 4, 1, 0]

>>> line = 'aaa,bbb,ccc'
>>> ''.join([x.upper() for x in line.split(',')]) # Makes a pointless list
'AAABBBCCC'
>>> ''.join(x.upper() for x in line.split(',')) # Generates results
'AAABBBCCC'
>>> ''.join(map(str.upper, line.split(','))) # Generates results
'AAABBBCCC'

>>> [x * 2 for x in [abs(x) for x in (−1, −2, 3, 4)]] # Nested comprehensions
[2, 4, 6, 8]
>>> list(map(lambda x: x * 2, map(abs, (−1, −2, 3, 4)))) # Nested maps
[2, 4, 6, 8]
>>> list(x * 2 for x in (abs(x) for x in (−1, −2, 3, 4))) # Nested generators
[2, 4, 6, 8]

Generator expressions versus filter

>>> line = 'aa bbb c'
>>> ''.join(x for x in line.split() if len(x) > 1) # Generator with 'if'
'aabbb'
>>> ''.join(filter(lambda x: len(x) > 1, line.split())) # Similar to filter
'aabbb'

Generator Functions Versus Generator Expressions

• Generator functions
  A function def statement that contains a yield statement is turned into a generator
  function. When called, it returns a new generator object with automatic retention
  of local scope and code position; an automatically created __iter__ method that
  simply returns itself; and an automatically created __next__ method (next in 2.X)
  that starts the function or resumes it where it last left off, and raises StopItera
  tion when finished producing results.
• Generator expressions
  A comprehension expression enclosed in parentheses is known as a generator expression.
  When run, it returns a new generator object with the same automatically
  created method interface and state retentionas a generator function call’s results
  —with an __iter__ method that simply returns itself; and a _next__ method
  (next in 2.X) that starts the implied loop or resumes it where it last left off, and
  raises StopIteration when finished producing results.
  

Generators Are Single-Iteration Objects

>>> G = (c * 4 for c in 'SPAM')
>>> iter(G) is G # My iterator is myself: G has __next__
True

如果重头开始，重新生成  (c * 4 for c in 'SPAM')对象

>>> def both(N):
        for i in range(N): yield i
        for i in (x ** 2 for x in range(N)): yield i
>>> list(both(5))
[0, 1, 2, 3, 4, 0, 1, 4, 9, 16]


>>> def both(N):
        yield from range(N)           #3.3 syntax
        yield from (x ** 2 for x in range(N))
>>> list(both(5))
[0, 1, 2, 3, 4, 0, 1, 4, 9, 16]

>>> import os
>>> for (root, subs, files) in os.walk('.'): # Directory walk generator
        for name in files: # A Python 'find' operation
            if name.startswith('call'):
                print(root, name)

                
. callables.py
.\dualpkg callables.py

Scopes and Comprehension Variables

3.X variables assigned in a comprehension are really a

further nested special-case scope; other names referenced within these expressions follow
the usual LEGB rules.

c:\code> py −3
>>> (X for X in range(5))
<generator object <genexpr> at 0x00000000028E4798>
>>> X
NameError: name 'X' is not defined
>>> X = 99
>>> [X for X in range(5)] # 3.X: generator, set, dict, and list localize
[0, 1, 2, 3, 4]
>>> X
99


 >>> Y = 99
>>> for Y in range(5): pass # But loop statements do not localize names
>>> Y
4

>>> {x: y for x in [1, 2, 3] for y in [4, 5, 6]} # Neither do dict keys 此处更新了值
{1: 6, 2: 6, 3: 6}

第二十一章 The Benchmarking Interlude

timeit

Function Gotchas：

• Local Names Are Detected Statically
• Defaults and Mutable Objects
• Functions Without returns：Technically, all functions return a value; if you don’t provide a return
  statement, your function returns the None object automatically