[读书笔记]流畅的python-函数装饰器和闭包

装饰器基础知识

装饰器是可调用的对象，其参数是另一个函数（被装饰的函数）。 装饰器可能会处理被 装饰的函数，然后把它返回，或者将其替换成另一个函数或可调用对象。严格来说，装饰器只是语法糖。如前所示，装饰器可以像常规的可调用对象那样调用，其 参数是另一个函数。有时，这样做更方便，尤其是做元编程（在运行时改变程序的行 为）时。
装饰器有两大特性：

• 能把被装饰的函数替换成其他函数。
• 装饰器在加载模块时立即执行

Python何时执行装饰器

装饰器函数在被装饰的函数定义之后立即运行，通常在加载被装饰函数的代码后，装饰器函数就立即运行。
例如：被装饰的f1和f2会在main之前运行，示例 主要想强调，函数装饰器在导入模块时立即执行，而被装饰的函数只在明确调用 时运行。这突出了 Python 程序员所说的导入时和运行时之间的区别。

registry = []  ➊

def register(func):  ➋
    print('running register(%s)' % func)  ➌
    registry.append(func)  ➍
    return func  ➎

@register  ➏
def f1():
    print('running f1()')

@register
def f2():
    print('running f2()')

def f3():  ➐
    print('running f3()')

def main():  ➑
    print('running main()')
    print('registry ->', registry)
    f1()
    f2()
    f3()

if __name__=='__main__':
    main()  ➒


结果：
$ python3 registration.py
running register(<function f1 at 0x100631bf8>) # 
running register(<function f2 at 0x100631c80>) #
running main()
registry -> [<function f1 at 0x100631bf8>, <function f2 at 0x100631c80>]
running f1()
running f2()
running f3()

变量作用域规则

Python 不要求声明变量，但是假定在函数定义体中赋值的变量是局部变量。
例子：

>>> b = 6
>>> def f2(a):
...     print(a)
...     print(b)
...     b = 9  # 如果没有赋值语句，不报错
...
>>> f2(3)
3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in f2
UnboundLocalError: local variable 'b' referenced before assignment

出错原因：Python 编译函数的定义体时，它判断 b 是局部变量，因为在函数中给它赋值了。生成的字节码证实了这种判断，Python 会尝试从本地环境获取 b 。后面调用 f2(3) 时， f2 的定义体会获取并打印局部变量 a 的值，但是尝试获取局部变量 b 的值时，发现 b 没有绑定值，所以报错。如果在函数中赋值时想让解释器把 b 当成全局变量，需要将b声明为global

闭包

定义：闭包指延伸了作用域的函数，其中包含函数定义体中引用、但是不在定义体中定义的非全局变量。函数是不是匿名的没有关系，关键是它能访问定义体之外定义的非全局变量。
闭包的例子：

def make_averager():
    series = []

    def averager(new_value):
        series.append(new_value)
        total = sum(series)
        return total/len(series)

    return averager

>>> avg.__code__.co_varnames
 ('new_value', 'total') 
>>>> avg.__code__.co_freevars
('series',)
>>> avg.__closure__
(<cell at 0x107a44f78: list object at 0x107a91a48>,)
>>> avg.__closure__[0].cell_contents
[10, 11, 12]

nonlocal

计算移动平均值的另一种实现，只记录总数和数量，但对数字、字符串、元组等不可变类型来说，只能读取，不能更新。如果尝试重新绑 定，例如 count = count + 1，其实会隐式创建局部变量 count。这样，count 就不是 自由变量了，因此不会保存在闭包中。

def make_averager():
    count = 0
    total = 0

    def averager(new_value):
        count += 1
        total += new_value
        return total / count

    return averager

>>> avg = make_averager()
>>> avg(10)
Traceback (most recent call last):
  ...
UnboundLocalError: local variable 'count' referenced before assignment
>>>

正确的做法是把count和tot声明为nonlcoal

def make_averager():
    count = 0
    total = 0

    def averager(new_value):
        nonlocal count,total
        count += 1
        total += new_value
        return total / count

    return averager

实现简单的装饰器

典型的装饰器：把被装饰的函数替换成新函数，二者接受相同的参数，而且（通 常）返回被装饰的函数本该返回的值，同时还会做些额外操作。

import time

def clock(func):
    def clocked(*args):  # ➊
        t0 = time.perf_counter()
        result = func(*args)  # ➋
        elapsed = time.perf_counter() - t0
        name = func.__name__
        arg_str = ', '.join(repr(arg) for arg in args)
        print('[%0.8fs] %s(%s) -> %r' % (elapsed, name, arg_str, result))
        return result
    return clocked #  ➌

但上述装饰器存在几个缺点：不支持关键字参数，而且遮盖了被装饰函 数的 name 和 doc 属性。使用functools.wraps 改进。

import time
import functools

def clock(func):
    @functools.wraps(func)
    def clocked(*args, **kwargs):
        t0 = time.time()
        result = func(*args, **kwargs)
        elapsed = time.time() - t0
        name = func.__name__
        arg_lst = []
        if args:
            arg_lst.append(', '.join(repr(arg) for arg in args))
        if kwargs:
            pairs = ['%s=%r' % (k, w) for k, w in sorted(kwargs.items())]
            arg_lst.append(', '.join(pairs))
        arg_str = ', '.join(arg_lst)
        print('[%0.8fs] %s(%s) -> %r ' % (elapsed, name, arg_str, result))
        return result
    return clocked

标准库中的装饰器

Python 内置了三个用于装饰方法的函数：property、classmethod 和 staticmethod。
另外functtools也有些常见的装饰器有：functools.wraps，functools.lru_cache，functools.singledispatch

functools.lru_cache

lru_cache实现了备忘功能，可以把函数结果保存起来。
lru_wrapper的实现：

def _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo):
    # Constants shared by all lru cache instances:
    sentinel = object()          # unique object used to signal cache misses
    make_key = _make_key         # build a key from the function arguments
    PREV, NEXT, KEY, RESULT = 0, 1, 2, 3   # names for the link fields

    cache = {}
    hits = misses = 0
    full = False
    cache_get = cache.get    # bound method to lookup a key or return None
    cache_len = cache.__len__  # get cache size without calling len()
    lock = RLock()           # because linkedlist updates aren't threadsafe
    root = []                # root of the circular doubly linked list
    root[:] = [root, root, None, None]     # initialize by pointing to self

    if maxsize == 0:

        def wrapper(*args, **kwds):
            # No caching -- just a statistics update after a successful call
            nonlocal misses
            result = user_function(*args, **kwds)
            misses += 1
            return result

    elif maxsize is None:

        def wrapper(*args, **kwds):
            # Simple caching without ordering or size limit
            nonlocal hits, misses
            key = make_key(args, kwds, typed)
            result = cache_get(key, sentinel)
            if result is not sentinel:
                hits += 1
                return result
            result = user_function(*args, **kwds)
            cache[key] = result
            misses += 1
            return result

    else:

        def wrapper(*args, **kwds):
            # Size limited caching that tracks accesses by recency
            nonlocal root, hits, misses, full
            key = make_key(args, kwds, typed)
            with lock:
                link = cache_get(key)
                if link is not None:
                    # Move the link to the front of the circular queue
                    link_prev, link_next, _key, result = link
                    link_prev[NEXT] = link_next
                    link_next[PREV] = link_prev
                    last = root[PREV]
                    last[NEXT] = root[PREV] = link
                    link[PREV] = last
                    link[NEXT] = root
                    hits += 1
                    return result
            result = user_function(*args, **kwds)
            with lock:
                if key in cache:
                    # Getting here means that this same key was added to the
                    # cache while the lock was released.  Since the link
                    # update is already done, we need only return the
                    # computed result and update the count of misses.
                    pass
                elif full:
                    # Use the old root to store the new key and result.
                    oldroot = root
                    oldroot[KEY] = key
                    oldroot[RESULT] = result
                    # Empty the oldest link and make it the new root.
                    # Keep a reference to the old key and old result to
                    # prevent their ref counts from going to zero during the
                    # update. That will prevent potentially arbitrary object
                    # clean-up code (i.e. __del__) from running while we're
                    # still adjusting the links.
                    root = oldroot[NEXT]
                    oldkey = root[KEY]
                    oldresult = root[RESULT]
                    root[KEY] = root[RESULT] = None
                    # Now update the cache dictionary.
                    del cache[oldkey]
                    # Save the potentially reentrant cache[key] assignment
                    # for last, after the root and links have been put in
                    # a consistent state.
                    cache[key] = oldroot
                else:
                    # Put result in a new link at the front of the queue.
                    last = root[PREV]
                    link = [last, root, key, result]
                    last[NEXT] = root[PREV] = cache[key] = link
                    # Use the cache_len bound method instead of the len() function
                    # which could potentially be wrapped in an lru_cache itself.
                    full = (cache_len() >= maxsize)
                misses += 1
            return result

    def cache_info():
        """Report cache statistics"""
        with lock:
            return _CacheInfo(hits, misses, maxsize, cache_len())

    def cache_clear():
        """Clear the cache and cache statistics"""
        nonlocal hits, misses, full
        with lock:
            cache.clear()
            root[:] = [root, root, None, None]
            hits = misses = 0
            full = False

    wrapper.cache_info = cache_info
    wrapper.cache_clear = cache_clear
    return wrapper

functools.singledispatch

因为 Python 不支持重载方法或函数，所以我们不能使用不同的签名定义 htmlize 的变 体，也无法使用不同的方式处理不同的数据类型。在 Python 中，一种常见的做法是把 htmlize 变成一个分派函数，使用一串 if/elif/elif，调用专门的函数，如 htmlize_str、htmlize_int，等等。这样不便于模块的用户扩展，还显得笨拙：时间 一长，分派函数 htmlize 会变得很大，而且它与各个专门函数之间的耦合也很紧密。
functools.singledispatch实现了单分派功能(类似C++，Java中的重载功能),Python 3.4 新增的 functools.singledispatch装饰器可以把整体方案拆分成多个模 块。使用 @singledispatch 装饰的普通函 数会变成泛函数（generic function）：根据第一个参数的类型，以不同方式执行相同操作 的一组函数.

from functools import singledispatch
from collections import abc
import numbers
import html

@singledispatch  ➊
def htmlize(obj):
    content = html.escape(repr(obj))
    return '<pre>{}</pre>'.format(content)

@htmlize.register(str)  ➋
def _(text):            ➌
    content = html.escape(text).replace('\n', '<br>\n')
    return '<p>{0}</p>'.format(content)

@htmlize.register(numbers.Integral)  ➍
def _(n):
    return '<pre>{0} (0x{0:x})</pre>'.format(n)

@htmlize.register(tuple)  ➎
@htmlize.register(abc.MutableSequence)
def _(seq):
    inner = '</li>\n<li>'.join(htmlize(item) for item in seq)
    return '<ul>\n<li>' + inner + '</li>\n</ul>'

参数化装饰器

解析源码中的装饰器时，Python 把被装饰的函数作为第一个参数传给装饰器函数。那怎么 让装饰器接受其他参数呢？答案是：创建一个装饰器工厂函数，把参数传给它，返回一个 装饰器，然后再把它应用到要装饰的函数上

不带参数的装饰器
registry = []

def register(func):
    print('running register(%s)' % func)
    registry.append(func)
    return func

@register
def f1():
    print('running f1()')

print('running main()')
print('registry ->', registry)
f1()

带参数的装饰器，多加一层工厂函数
registry = set()  ➊
def register(active=True):  ➋
    def decorate(func):  ➌
        print('running register(active=%s)->decorate(%s)'
              % (active, func))
        if active:   ➍
            registry.add(func)
        else:
            registry.discard(func)  ➎

        return func  ➏
    return decorate  ➐

@register(active=False)  ➑
def f1():
    print('running f1()')

@register()  ➒
def f2():
    print('running f2()')

def f3():
    print('running f3()')

如上节提到的lru_cache是带参数的，外层工厂函数实现如下：

def lru_cache(maxsize=128, typed=False):
    """Least-recently-used cache decorator.

    If *maxsize* is set to None, the LRU features are disabled and the cache
    can grow without bound.

    If *typed* is True, arguments of different types will be cached separately.
    For example, f(3.0) and f(3) will be treated as distinct calls with
    distinct results.

    Arguments to the cached function must be hashable.

    View the cache statistics named tuple (hits, misses, maxsize, currsize)
    with f.cache_info().  Clear the cache and statistics with f.cache_clear().
    Access the underlying function with f.__wrapped__.

    See:  http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used

    """

    # Users should only access the lru_cache through its public API:
    #       cache_info, cache_clear, and f.__wrapped__
    # The internals of the lru_cache are encapsulated for thread safety and
    # to allow the implementation to change (including a possible C version).

    # Early detection of an erroneous call to @lru_cache without any arguments
    # resulting in the inner function being passed to maxsize instead of an
    # integer or None.
    if maxsize is not None and not isinstance(maxsize, int):
        raise TypeError('Expected maxsize to be an integer or None')

    def decorating_function(user_function):
        wrapper = _lru_cache_wrapper(user_function, maxsize, typed, _CacheInfo)
        return update_wrapper(wrapper, user_function)

    return decorating_function