使用decorator在本模块中向系统函数插入自定义代码

什么是decorator

decorator是python这类动态语言独有的语法糖，其使用闭包和factory函数实现,hook了被装饰函数的调用过程但又不影响被装饰函数的输入输出接口（即不用修改被装饰函数的调用代码），同时又不影响其它模块中对原始被装饰函数的调用，使得一些重复性的任务变得简洁又简单，例如，插入代码使得一个模块中的所有函数运行完毕输出运行时间，或者在自定义函数或者系统函数运行前运行后进行日志/调试输出或者profiling、将一个函数转变成异步调用。 关于decorator的实际用途，这里有上百个例子。在本项目中，decorator主要用来对SQL查询的cursor.execute()函数调用统计运行时间。使用decorator的优点：
1. 无需修改被装饰函数的调用代码，这一点在被装饰函数是其他人编写的第三方函数修改其调用代码很困难时显得尤其重要
2. 降低了出错几率和输入量，定义一个嵌套的工厂函数就可以在所有需要插入代码的地方使用，无论是编译时在自定义函数定义处插入’@MyDecorator’代码还是调用前对第三方函数使用setattr函数插入代码都只需要几行代码就可以完成。
3. 不污染名字空间。由于Python在在一个文件中定义的全局变量作用域仅限于本文件，因此在一个文件中定义的修改版的被调用函数不会影响原函数在其他文件中被调用，如果在自己的*.py文件中对系统库函数插入了新的代码，其他开发者调用该系统函数时不受影响，被调用的仍然是原始系统库函数而不是修改版的函数。

需要了解的概念

• closure闭包， 所有函数式编程语言的标配，即一个封闭的带状态的函数环境 ，可以理解为定义在函数A内的函数B， 一般使用一个工厂函数A返回一个绑定了某些参数（状态）的函数B，可以实现与类函数这种带状态变量的函数相同的功能但无需额外专门定义一个类。闭包的用途包括实现有限状态自动机DFA，实现装饰器等等。Python中闭包的实际运用例子可以参考这里。
• 函数也是变量。函数也是变量意味着函数对象可以作为参数传递给另一个函数，也可以作为其它函数的返回值。 python中的一切变量都是可以在运行时改变的，函数是变量(函数变量一般是个类，保存了函数的入口地址等，在C++中称为functor)意味着调用该函数前可以将其替换为另一个函数对象/函数变量，无论该函数是用户自定义函数还是系统函数。其中系统函数可以用setattr替换模块内的指定函数。只要保证在decorator中调用原函数并且不改变原函数的签名(参数列表，返回值不变)，这种替换就是安全的。

用到的工具库

• inspect 用于动态查看当前的函数名，调用堆栈所在帧(frame)等运行情况
• functools 使得decorator function与decorated function在pythonde的object体系中有一样的attribute,例如name, docstring

装饰普通函数的decorator:

最简单的例子：decorator 无参数，decorated function是一个普通函数而非类成员函数，即不需要self指针
示例代码

import inspect
from functools import wraps
print
print "decorator begin..."
def Decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
                func_name=inspect.currentframe().f_code.co_name
                print "inside innermost func '%s'" % func_name
                ret=func(args, kwargs)
                return ret 
        return wrapper
@Decorator
def SimpleFunc(*args, **kwargs):
        func_name=inspect.currentframe().f_code.co_name
        print "Inside func '%s'" % func_name
        return "\t'%s' test return value" % func_name
print "SimpleFunc() return:\t%s" % SimpleFunc()

输出：

decorator begin...
inside innermost func 'wrapper'
Inside func 'SimpleFunc'
SimpleFunc() return:        'SimpleFunc' test return value

装饰普通函数的带参数decorator:

decorator构造时需要若干参数, 此时不能直接将函数变量传递至decorator的构造函数，而是需要增加一个间接层,见下例

@DecoratorWithParam("test_arg1", "test_arg2") 
def SomeFunc(*args, **kwargs): pass

相当于

decorator_with_param_func=DecoratorWithParam("test_arg1", "test_arg2")
SomeFunc = decorator_with_param_func(SomeFunc)

示例代码

print
print "decorator with arguments begin..."
def DecoratorWithParam(*deco_args):
        def real_wrapper(func):
                @wraps(func)
                def wrapper(*args, **kwargs):
                        func_name=inspect.currentframe().f_code.co_name
                        print "decorator parameters:"
                        for para in deco_args:
                                print "\t%s" % para
                        print "inside innermost func '%s'" % func_name
                        ret=func(args, kwargs)
                        return ret 
                return wrapper
        return real_wrapper

@DecoratorWithParam("test_arg1", "test_arg2")
def SomeFunc(*args, **kwargs):
        func_name=inspect.currentframe().f_code.co_name
        print "inside func '%s'" % func_name
        return "\t'%s' test return value" % func_name
print "SomeFunc() return:\t%s" % SomeFunc()

输出：

decorator with arguments begin...
decorator parameters:
    test_arg1
    test_arg2
inside innermost func 'wrapper'
inside func 'SomeFunc'
SomeFunc() return:      'SomeFunc' test return value

装饰类函数的decorator

本例中使用了class实现decorator, 用函数也可以实现同样功能, 本例使用稍复杂版本的decorator class演示，函数版的decorator实现大同小异，上面两个例子已经演示过了. 注意事项：
1. 下例中的decorator class是一个callable object(函数对象), 与函数式的decorator不同，其中定义的函数call不能命名为别的名字。
2. 使用class实现的decorator的语法是

    @ClassMethodDecorator()
    @ClassMethodDecoratorWithArguments("these", "are", "decorator", "arguments")

注意其中的括号

class A:
    @ClassMethodDecorator() 
    def SomeFunc(self, *args, **kwargs): pass

相当于

class A:
    class_method_decorator_instance=ClassMethodDecorator()
    def SomeFunc(self, *args, **kwargs):pass
    SomeFunc = class_method_decorator_instance(SomeFunc)

带参数的decorator class

class A:
    @ClassMethodDecoratorWithArguments("these", "are", "decorator", "arguments") 
    def SomeFunc(self, *args, **kwargs): pass

相当于

class A:
    class_method_decorator_with_arguments_instance=ClassMethodDecoratorWithArguments("these", "are", "decorator", "arguments")
    def SomeFunc(self, *args, **kwargs):pass
    SomeFunc = class_method_decorator_with_arguments_instance(SomeFunc)

示例代码

class ClassMethodDecorator(object):
        def __call__(decorator_self, func): # this func name cannot change
                def wrapper(decorated_self, *args, **kwargs):
                        ## you can access the "self" of class member func here through the "decorated_self" parameter, first argument is self ref by protocol, hence you could naming it as you like
                        ## therefor do whatever you want        
                        print "\tinside innermost funct '%s', argument decorated_self:'%s', outer decorator_self:'%s'" % ( inspect.currentframe().f_code.co_name, \
                                                                                                      decorated_self, decorator_self )
                        return func(decorated_self, *args, **kwargs)
                return wrapper

class ClassMethodDecoratorWithArguments(object):
        ''' decorator with arguments: http://www.artima.com/weblogs/viewpost.jsp?thread=240845 and access an decorated class instance from inside a class decorator:https://stackoverflow.com/questions/2181275/python-how-do-i-access-an-decorated-classs-instance-from-inside-a-class-decora'''
        def __init__(self, *deco_args, **deco_kwargs):
                self.deco_args=deco_args
                self.deco_kwargs=deco_kwargs
        def __call__(decorator_self, func): # this func name cannot change
                def wrapper(decorated_self, *args, **kwargs):
                        print "\tinside innermost funct '%s'\n" % inspect.currentframe().f_code.co_name,
                        print "\tdecorator arguments\targs:%s\tkwrags:%s" %( " ".join(decorator_self.deco_args), \
                                                                            " ".join(decorator_self.deco_kwargs) )
                        ## you can access the "self" of class member func here through the "decorated_self" parameter, first argument is self ref by protocol, hence you could naming it as you like
                        ## therefor do whatever you want        
                        print "\targument decorated_self:'%s', outer decorator_self:'%s'" % (decorated_self, decorator_self )                                                     
                        return func(decorated_self, *args, **kwargs)
                return wrapper

class SomeClass(object):
        ##self.name = 'John' # error here, self unavailable at class definition
        name="John"
        alt_name="Lennon"
        def __init__(self):
                print "\tclass %s initing, self:\t'%s'" % ( type(self).__name__, self )
        @ClassMethodDecorator()
        def nameprinter(self):
                func_name=inspect.currentframe().f_code.co_name
                print  "\tinside func '%s.%s'" % ( type(self).__name__,func_name )
                print(self.name)
                return "test return value"

        @ClassMethodDecoratorWithArguments("these", "are", "decorator", "arguments")
        def nameprinter2(self):
                func_name=inspect.currentframe().f_code.co_name
                print  "\tinside func '%s.%s'" % ( type(self).__name__,func_name )
                print(self.alt_name)
                return "test return value 2"

print
print "decorator with user-defined class member function begin..."
myinstance = SomeClass()
print "\t%s.nameprinter() return:\t%s" % ( myinstance, myinstance.nameprinter() )
print "\t%s.nameprinter2() return:\t%s" % ( myinstance, myinstance.nameprinter2() )

输出

decorator with user-defined class member function begin...
    class SomeClass initing, self:  '<__main__.SomeClass object at 0xc46e10>'
    inside innermost funct 'wrapper', argument decorated_self:'<__main__.SomeClass object at 0xc46e10>', outer decorator_self:'<__main__.ClassMethodDecorator object at 0xc46cd0>'
    inside func 'SomeClass.nameprinter'
John
    <__main__.SomeClass object at 0xc46e10>.nameprinter() return:   test return value
    inside innermost funct 'wrapper'
    decorator arguments args:these are decorator arguments  kwrags:
    argument decorated_self:'<__main__.SomeClass object at 0xc46e10>', outer decorator_self:'<__main__.ClassMethodDecoratorWithArguments object at 0xc46dd0>'
    inside func 'SomeClass.nameprinter2'
Lennon
    <__main__.SomeClass object at 0xc46e10>.nameprinter2() return:  test return value 2

装饰系统函数

本例子中用到了自定义计时函数timeit，其定义在dictFetchAll.py文件中，其具体定义为：

import time
def timeit(f):
        '''decorator for function execution timing'''
        def timed(*args, **kw):

            ts = time.time()
            result = f(*args, **kw)
            te = time.time()

            print 'func:%r\targs:[args: %r\tkwargs: %r]\ttook: %2.6f sec' % \ 
                            (f.__name__, args, kw, te-ts)
            return result

        return timed

class cls_timeit(object):
        ''' timing decorator made for clsss method, 
        using with:'@cls_timeit()' right BEFORE 'def class_function_name(*args, **kwargs):pass' WITHIN class definition
        ref: https://stackoverflow.com/questions/2181275/python-how-do-i-access-an-decorated-classs-instance-from-inside-a-class-decora
        '''
        def __call__(self, func):
                def wrapper(that, *args, **kwargs):
                        ## you can access the "self" of func here through the "that" parameter
                        ## and hence do whatever you want        
                        return func(that, *args, **kwargs)
                return wrapper

示例代码

print
print "decorating system module function now..."

import inspect
import math
from dictFetchAll import *

for n, v in inspect.getmembers(math, inspect.isroutine):
        print "\tdecorating func '%s'(%s) with func 'timeit'" % (n,v)
        setattr(math, n, timeit(v))
print "math.sqrt(9) return:\t%s" % (math.sqrt(9))
print "math.pow(3.0, 2.0) return:\t%s" % (math.pow(3.0,2.0))

输出

decorating system module function now...
    decorating func 'acos'(<built-in function acos>) with func 'timeit'
    decorating func 'acosh'(<built-in function acosh>) with func 'timeit'
    decorating func 'asin'(<built-in function asin>) with func 'timeit'
    decorating func 'asinh'(<built-in function asinh>) with func 'timeit'
    decorating func 'atan'(<built-in function atan>) with func 'timeit'
    decorating func 'atan2'(<built-in function atan2>) with func 'timeit'
    decorating func 'atanh'(<built-in function atanh>) with func 'timeit'
    decorating func 'ceil'(<built-in function ceil>) with func 'timeit'
    decorating func 'copysign'(<built-in function copysign>) with func 'timeit'
    decorating func 'cos'(<built-in function cos>) with func 'timeit'
    decorating func 'cosh'(<built-in function cosh>) with func 'timeit'
    decorating func 'degrees'(<built-in function degrees>) with func 'timeit'
    decorating func 'erf'(<built-in function erf>) with func 'timeit'
    decorating func 'erfc'(<built-in function erfc>) with func 'timeit'
    decorating func 'exp'(<built-in function exp>) with func 'timeit'
    decorating func 'expm1'(<built-in function expm1>) with func 'timeit'
    decorating func 'fabs'(<built-in function fabs>) with func 'timeit'
    decorating func 'factorial'(<built-in function factorial>) with func 'timeit'
    decorating func 'floor'(<built-in function floor>) with func 'timeit'
    decorating func 'fmod'(<built-in function fmod>) with func 'timeit'
    decorating func 'frexp'(<built-in function frexp>) with func 'timeit'
    decorating func 'fsum'(<built-in function fsum>) with func 'timeit'
    decorating func 'gamma'(<built-in function gamma>) with func 'timeit'
    decorating func 'hypot'(<built-in function hypot>) with func 'timeit'
    decorating func 'isinf'(<built-in function isinf>) with func 'timeit'
    decorating func 'isnan'(<built-in function isnan>) with func 'timeit'
    decorating func 'ldexp'(<built-in function ldexp>) with func 'timeit'
    decorating func 'lgamma'(<built-in function lgamma>) with func 'timeit'
    decorating func 'log'(<built-in function log>) with func 'timeit'
    decorating func 'log10'(<built-in function log10>) with func 'timeit'
    decorating func 'log1p'(<built-in function log1p>) with func 'timeit'
    decorating func 'modf'(<built-in function modf>) with func 'timeit'
    decorating func 'pow'(<built-in function pow>) with func 'timeit'
    decorating func 'radians'(<built-in function radians>) with func 'timeit'
    decorating func 'sin'(<built-in function sin>) with func 'timeit'
    decorating func 'sinh'(<built-in function sinh>) with func 'timeit'
    decorating func 'sqrt'(<built-in function sqrt>) with func 'timeit'
    decorating func 'tan'(<built-in function tan>) with func 'timeit'
    decorating func 'tanh'(<built-in function tanh>) with func 'timeit'
    decorating func 'trunc'(<built-in function trunc>) with func 'timeit'
func:'sqrt' args:[args: (9,)    kwargs: {}] took: 0.000004 sec
math.sqrt(9) return:    3.0
func:'pow'  args:[args: (3.0, 2.0)  kwargs: {}] took: 0.000007 sec
math.pow(3.0, 2.0) return:  9.0

本例中为系统的math库中的所有数学函数增加了运行结束时输出函数名、参数列表、运行时间的功能。

示例程序源码

上述全部示例程序deco_test.py

参考

• http://www.artima.com/weblogs/viewpost.jsp?thread=240808
• http://www.artima.com/weblogs/viewpost.jsp?thread=240845
• https://stackoverflow.com/questions/2365701/decorating-python-class-methods-how-do-i-pass-the-instance-to-the-decorator
• https://stackoverflow.com/questions/3191799/decorate-a-whole-library-in-python
• https://stackoverflow.com/questions/1263451/python-decorators-in-classes
• http://thecodeship.com/patterns/guide-to-python-function-decorators/
• https://stackoverflow.com/questions/2181275/python-how-do-i-access-an-decorated-classs-instance-from-inside-a-class-decora
• https://stackoverflow.com/questions/5929107/python-decorators-with-parameters