使用烧杯进行缓存？ 为什么要切换到dogpile.cache

Continuing on where I left off regarding Beaker in October (see Thoughts on Beaker), my new replacement for Beaker caching, dogpile.cache, has had a bunch of early releases. While I’m still considering it “alpha” until I know a few people have taken it around the block, it should be pretty much set for early testing and hopefully can be tagged as production quality in the near future.

继续我十月份关于Beaker的讨论（请参阅Thoughts on Beaker ），我对Beaker缓存的新替代品dogpile.cache具有许多早期版本。 虽然我一直认为它是“ alpha”，直到我知道有人将它带到街区，但它应该为早期测试做好准备，并有望在不久的将来被标记为生产质量。

The core of Beaker’s caching mechanism is based on code I first wrote in 2005. It was adapted from what was basically my first Python program ever, a web template engine called Myghty, which in turn was based on a Perl system called HTML::Mason. The caching scenarios Beaker was designed for were primarily that of storing data in files, such as DBM files. A key assumption made at that time was that the backends would all provide some system of returning a flag whether or not a key was present, which would precede the actual fetch of the value from the cache. Another assumption made was that the actual lock applied to these backends to deal with the dogpile situation would be at its most “distributed” scope a file-based lock, using flock().

Beaker缓存机制的核心是基于我在2005年首次编写的代码。它基本上是我第一个Python程序改编而成的，该程序是一个名为Myghty的Web模板引擎，该引擎又基于名为HTML :: Mason的Perl系统。 。 Beaker设计的缓存方案主要是将数据存储在文件（例如DBM文件）中。 当时做出的一个关键假设是，后端都将提供某种返回标志的系统，无论该标志是否存在，都将在从缓存中实际获取值之前进行。 做出的另一个假设是，使用flock（）应用于这些后端以处理令人讨厌的情况的实际锁在其最“分布式”范围内将是基于文件的锁。

When memcached support was added to Beaker, these assumptions proved to be architectural shortcomings. There is no “check for a key” function in memcached; there’s only get(). Beaker’s dogpile lock calls “check for a key” twice. As a result, Beaker will in general pull a value out of memcached three times, each time resulting in an “unpickle” of a pickled object. The upshot of this is that Beaker pulls over the network and unpickles your object three times times on every cache hit. Users of Beaker are also well familiar with the awkward lock files Beaker insists on generating, even though there are more appropriate ways to lock for distributed caches.

当将Memcached支持添加到Beaker时，这些假设被证明是体系结构上的缺陷。 memcached中没有“检查密钥”功能。 只有get（）了 。 烧杯的吊钩式锁叫两次“检查钥匙”。 结果，Beaker通常会从内存缓存中提取3次值，每次都会导致“腌制”对象“变酸”。 这样做的结果是， 在每次缓存命中时 ， Beaker都会拉动网络并为您的对象解开3次 。 Beaker的用户也非常熟悉Beaker坚持要生成的笨拙的锁定文件，即使有更适当的方法来锁定分布式缓存也是如此。

So for no other reason than these, dogpile.cache’s entirely new and extremely simplified architecture is an improvement of vast proportions. The test program below illustrates the improvement in unpickling behavior, as well as dogpile.cache’s simplified API:

因此，除了这些原因之外，dogpile.cache的全新且极其简化的架构是对很大一部分的改进。 下面的测试程序说明了析举行为以及dogpile.cache的简化API的改进：

class class WidgetWidget (( objectobject ):
    ):
    """Sample object to be cached.

"""Sample object to be cached.

    Counts pickles and unpickles.

    Counts pickles and unpickles.

    """
        """
    pickles pickles = = 0
    0
    unpickles unpickles = = 0

    0

    def def __init____init__ (( selfself , , idid ):
        ):
        selfself .. id id = = id

    id

    def def __getstate____getstate__ (( selfself ):
        ):
        WidgetWidget .. pickles pickles +=+= 1
        1
        return return selfself .. __dict__

    __dict__

    def def __setstate____setstate__ (( selfself , , statestate ):
        ):
        WidgetWidget .. unpickles unpickles +=+= 1
        1
        selfself .. __dict____dict__ .. updateupdate (( statestate )

)

def def test_beakertest_beaker ():
    ():
    from from beaker beaker import import cache

    cache

    cache_manager cache_manager = = cachecache .. CacheManagerCacheManager (( cache_regionscache_regions == {
    {
    'default' 'default' :{
            :{
            'type''type' :: 'memcached''memcached' ,
            ,
            'url''url' :: '127.0.0.1:11211''127.0.0.1:11211' ,
            ,
            'expiretime''expiretime' :: 11 ,
            ,
            'lock_dir''lock_dir' :: '.''.' ,
            ,
            'key_length''key_length' :: 250
        250
        }
    }
    })

    })

    @cache_manager.region@cache_manager.region (( "default""default" , , "some_key""some_key" )
    )
    def def get_widget_beakerget_widget_beaker (( idid ):
        ):
        return return WidgetWidget (( idid )

    )

    _run_test_run_test (( get_widget_beakerget_widget_beaker )

)

def def test_dogpiletest_dogpile ():
    ():
    from from dogpile.cache dogpile.cache import import make_region
    make_region
    from from dogpile.cache.util dogpile.cache.util import import sha1_mangle_key

    sha1_mangle_key

    region region = = make_regionmake_region (( key_manglerkey_mangler == sha1_mangle_keysha1_mangle_key )) .. configureconfigure (
        (
        'dogpile.cache.memcached''dogpile.cache.memcached' ,
        ,
        expiration_time expiration_time = = 11 ,
        ,
        arguments arguments = = {
            {
            'url''url' :[:[ "127.0.0.1:11211""127.0.0.1:11211" ],
        ],
        },
    },
    )

    )

    @region.cache_on_arguments@region.cache_on_arguments ()
    ()
    def def get_widget_dogpileget_widget_dogpile (( idid ):
        ):
        return return WidgetWidget (( idid )

    )

    _run_test_run_test (( get_widget_dogpileget_widget_dogpile )

)

def def _run_test_run_test (( get_widgetget_widget ):
    ):
    """Store an object, retrieve from the cache.

"""Store an object, retrieve from the cache.

    Wait two seconds, then exercise a regeneration.

    Wait two seconds, then exercise a regeneration.

    """
        """
    import import time

    time

    WidgetWidget .. pickles pickles = = WidgetWidget .. unpickles unpickles = = 0

    0

    # create and cache a widget.
    # create and cache a widget.
    # no unpickle necessary.
    # no unpickle necessary.
    w1 w1 = = get_widgetget_widget (( 22 )

    )

    # get it again.  one pull from cache
    # get it again.  one pull from cache
    # equals one unpickle needed.
    # equals one unpickle needed.
    w1 w1 = = get_widgetget_widget (( 22 )

    )

    timetime .. sleepsleep (( 22 )

    )

    # get from cache, will pull out the
    # get from cache, will pull out the
    # object but also the fact that it's
    # object but also the fact that it's
    # expired (costs one unpickle).
    # expired (costs one unpickle).
    # newly generated object
    # newly generated object
    # cached and returned.
    # cached and returned.
    w1 w1 = = get_widgetget_widget (( 22 )

    )

    print print "Total pickles:""Total pickles:" , , WidgetWidget .. pickles
    pickles
    print print "Total unpickles:""Total unpickles:" , , WidgetWidget .. unpickles

unpickles

print print "beaker"
"beaker"
test_beakertest_beaker ()

()

print print "dogpile"
"dogpile"
test_dogpiletest_dogpile ()
()

Running this with a clean memcached you get:

使用干净的memcached运行此命令，您将获得：

beaker
Total pickles: 2
Total unpickles: 6
dogpile
Total pickles: 2
Total unpickles: 2

Run it a second time, so that the Widget is already in the cache. Now you get ten unpickles with Beaker compared to dogpile.cache’s three:

再次运行它，以便该小部件已在缓存中。 现在，与dogpile.cache的三个相比，Beaker的十个优点：

beaker
Total pickles: 2
Total unpickles: 10
dogpile
Total pickles: 2
Total unpickles: 3

The advantages of dogpile.cache go way beyond that:

dogpile.cache的优点远不止于此：

• dogpile.cache includes distinct memcached backends for pylibmc, memcache and bmemcached. These are all explicitly available via different backend names, in contrast to Beaker’s approach of deciding for you which memcached backend it wants to use.
• A dedicated API-space for backend-specific arguments, such as all the special arguments pylibmc offers.
• A Redis backend is provided.
• The system of “dogpile locking” is completely modular, and in the case of memcached and Redis, a “distributed lock” option is provided which will use the “set key if not exists” feature of those backends to provide the dogpile lock. A plain threaded mutex can be specified also.
• Cache regions and function decorators are open ended. You can plug in your own system of generating cache keys from decorated functions, as well as what kind of “key mangling” you’d like to apply to keys going into the cache (such as encoding, hashing, etc.)
• No lockfiles whatsoever unless you use the provided DBM backend; and there, you tell it exactly where to put the lockfile, or tell it to use a regular mutex instead.
• New backends are ridiculously simple to write, and can be popped in using regular setuptools entry points or in-application using the register_backend() function.
• Vastly simplified scope – there’s no dilution of the task at hand with session, cookie, or encryption features.
• Python 3 compatible in-place with no 2to3 step needed.

• dogpile.cache包括用于pylibmc ， memcache和bmemcached的不同的memcached后端。 这些都可以通过不同的后端名称显式地使用，这与Beaker决定要使用哪个内存缓存后端的方法形成对比。
• 一个专用的API空间，用于特定于后端的参数，例如pylibmc提供的所有特殊参数。
• 提供了Redis后端。
• “ dogpile锁定”系统是完全模块化的，如果使用memcached和Redis，则提供“分布式锁定”选项，该选项将使用这些后端的“ set key（如果不存在）”功能来提供dogpile锁定。 也可以指定普通螺纹互斥锁。
• 缓存区域和函数装饰器是开放式的。 您可以插入自己的系统，该系统从修饰后的函数生成缓存键，以及您要对进入缓存的键应用哪种“键修改”（例如编码，哈希等）。
• 除非您使用提供的DBM后端，否则都没有锁文件。 在那儿，您可以准确地告诉它放置锁文件的位置，或者告诉它使用常规互斥锁。
• 新的后端非常简单易写，可以使用常规的setuptools入口点弹出，也可以使用register_backend（）函数在应用程序中弹出。
• 范围大大简化–会话，cookie或加密功能不会对您的任务造成任何影响。
• 就地兼容Python 3，无需2to3步骤。

So I’m hoping we can all soon get modernized onto dogpile.cache.

因此，我希望我们所有人都能尽快升级到dogpile.cache。

dogpile.cache documentation.

dogpile.cache文档 。

翻译自: https://www.pybloggers.com/2012/04/using-beaker-for-caching-why-youll-want-to-switch-to-dogpile-cache/