django框架由lru_cache导致的bug，Testing Client-Side Applications with Django Post Mortem

I had the opportunity to give a webcast for O’Reilly Media during which I encountered a presenter’s nightmare: a broken demo. Worse than that it was a test failure in a presentation about testing. Is there any way to salvage such an epic failure?

我有机会为O 'Reilly Media做一个网络直播，在这个过程中，我遇到了一个演讲者的噩梦:一个破碎的演示。更糟糕的是，在关于测试的演示中出现了测试失败。有什么办法挽救这样一个史诗般的失败吗?

What Happened

发生了什么事

It was my second webcast and I chose to use the same format for both. I started with some brief introductory slides but most of the time was spent as a screen share, going through the code as well as running some commands in the terminal. Since this webcast was about testing this was mostly writing more tests and then running them. I had git branches setup for each phase of the process and for the first forty minutes this was going along great. Then it came to the grand finale. Integrate the server and client tests all together and run one last time. And it failed.

这是我的第二次网络直播，我选择了同样的格式。我从一些简短的介绍性幻灯片开始，但大部分时间都花在屏幕共享上，浏览代码并在终端中运行一些命令。由于这个网络广播是关于测试的，所以主要是编写更多的测试，然后运行它们。我为这个过程的每个阶段都设置了git分支，在开始的40分钟内进展得非常顺利。然后到了最后的压轴戏。将服务器和客户机测试集成在一起，最后运行一次。它失败了。

I quickly abandoned the idea of attempting to live debug this error and since I was at the end away I just went into my wrap up. Completely humbled and embarrassed I tried to answer the questions from the audience as gracefully as I could while inside I wanted to just curl up and hide.

我很快就放弃了尝试实时调试这个错误的想法，因为我在最后离开了，所以我只是进入了我的wrap up。我感到十分谦卑和尴尬，试图尽可能优雅地回答观众的问题，而内心却只想蜷缩起来，躲起来。

Tracing the Error

跟踪误差

The webcast was the end of the working day for me so when I was done I packed up and headed home. I had dinner with my family and tried not to obsess about what had just happened. The next morning with a clearer head I decided to dig into the problem. I had done much of the setup on my personal laptop but ran the webcast on my work laptop. Maybe there was something different about the machine setups. I ran the test again on my personal laptop. Still failed. I was sure I had tested this. Was I losing my mind?

对我来说，网络直播是一天工作的结束，所以当我结束时，我收拾好行李回家了。我和家人共进晚餐，尽量不去纠结刚刚发生的事情。第二天早上头脑清醒了，我决定深入研究这个问题。我在我的个人笔记本电脑上做了很多设置，但在我的工作笔记本电脑上进行了网络直播。也许机器的设置有些不同。我在我的个人笔记本电脑上再次进行了测试。仍然失败了。我确信我已经测试过了。我是不是疯了?

I looked through my terminal history. There it was and I ran it again.

我浏览了一下我的临终记录。就在那儿，我又看了一遍。

It passed! I’m not crazy! But what does that mean? I had run the test in isolation and it passed but when run in the full suite it failed. This points to some global shared state between tests. I took another look at the test.

它通过了!我不是疯了!但这意味着什么呢?我已经独立运行了测试，它通过了，但是当在完整的套件中运行时，它失败了。这指向测试之间的一些全局共享状态。我又看了看试卷。

 

import os

from django.conf import settings
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from django.test.utils import override_settings

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.support.ui import WebDriverWait


@override_settings(STATICFILES_DIRS=(
    os.path.join(os.path.dirname(__file__), 'static'), ))
class QunitTests(StaticLiveServerTestCase):
    """Iteractive tests with selenium."""

    @classmethod
    def setUpClass(cls):
        cls.browser = webdriver.PhantomJS()
        super().setUpClass()

    @classmethod
    def setUpClass(cls):
        cls.browser = webdriver.PhantomJS()
        super().setUpClass()

    @classmethod
    def tearDownClass(cls):
        cls.browser.quit()
        super().tearDownClass()

    def test_qunit(self):
        """Load the QUnit tests and check for failures."""

        self.browser.get(self.live_server_url + settings.STATIC_URL + 'index.html')
        results = WebDriverWait(self.browser, 5).until(
            expected_conditions.visibility_of_element_located(
                (By.ID, 'qunit-testresult')))
        total = int(results.find_element_by_class_name('total').text)
        failed = int(results.find_element_by_class_name('failed').text)
        self.assertTrue(total and not failed, results.text)

It seemed pretty isolated to me. The test gets its own webdriver instance. There is no file system manipulation. There is no interaction with the database and even if it did Django runs each test in its own transaction and rolls it back. Maybe this shared state wasn’t in my code.

这对我来说似乎很孤立。测试获得它自己的webdriver实例。没有文件系统操作。与数据库没有交互，即使它确实运行了Django，它也会在自己的事务中运行每个测试并回滚它。也许这个共享状态不在我的代码中。

Finding a Fix

找到一个解决

I’ll admit when people on IRC or Stackoverflow claim to have found a bug in Django my first instinct is to laugh. However, Django does have some shared state in its settings configuration. The test is using the override_settingsdecorator but perhaps there was something preventing it from working. I started to dig into the staticfiles code and that’s where I found it. Django was using the lru_cache decorator for the construction of the staticfiles finders. This means they were being cached after their first access. Since this test was running last in the suite it meant that the change to STATICFILES_DIRS was not taking effect. To fix my test meant that I simply needed to bust this cache at the start of my test.

我承认，当IRC或Stackoverflow上的用户声称发现了Django中的一个bug时，我的第一反应是大笑。不过，Django的设置配置中确实有一些共享状态。测试使用的是override_settingsdecorator，但是可能有什么东西阻止了它的工作。我开始钻研staticfiles代码，这就是我找到它的地方。Django使用lru_cache装饰器来构建静态文件查找器。这意味着它们在第一次访问之后被缓存。由于这个测试是最后一个在套件中运行的，这意味着对STATICFILES_DIRS的更改没有生效。要修复我的测试，意味着我只需要在测试开始时破坏这个缓存。

...
from django.contrib.staticfiles import finders, storage
...
from django.utils.functional import empty
...
class QunitTests(StaticLiveServerTestCase):
...
    def setUp(self):
        # Clear the cache versions of the staticfiles finders and storage
        # See https://code.djangoproject.com/ticket/24197
        storage.staticfiles_storage._wrapped = empty
        finders.get_finder.cache_clear()

Fixing at the Source

震源定位

Digging into this problem, it became clear that this wasn’t just a problem with the STATICFILES_DIRS setting but was a problem with using override_settingswith most of the contrib.staticfiles related settings. In fact I found the easiest fix for my test case by looking at Django’s own test suite. I decided this really needed to be fixed in Django so that this issue wouldn’t bite any other developers. I opened a ticket and a few days later I created a pull request with the fix. After some helpful review from Tim Graham it was merged and was included in the recent 1.8 release.

深入研究这个问题，很明显，这不仅仅是STATICFILES_DIRS设置的问题，而且是在大多数设计中使用override_settings的问题。staticfiles相关设置。事实上，通过查看Django自己的测试套件，我发现了对我的测试用例最简单的修复方法。我决定在Django中解决这个问题，这样这个问题就不会影响到其他开发人员。我打开了一个票据，几天后我创建了一个带有修复的pull请求。在Tim Graham的一些有用的评论之后，它被合并并包含在最近的1.8版本中。

What’s Next

接下来是什么

Having a test which passes alone and fails when running in the suite is a very frustrating problem. It wasn’t something that I planned to demonstrate when I started with this webcast but that’s where I ended up. The problem I experienced was entirely preventable if I had prepared for the webcast better. However, my own failing lead to a great example of tracking down global state in a test suite and ultimately helped to improve my favorite web framework in just the slightest amount. All together I think it makes the webcast better than I could have planned it.

在套件中运行一个单独通过并失败的测试是一个非常令人沮丧的问题。当我开始这个网络直播的时候，我并没有打算展示它，但这就是我结束的地方。如果我对网络直播有更好的准备，我所经历的问题是完全可以避免的。然而，我自己的失败导致了在测试套件中跟踪全局状态的一个很好的例子，并最终帮助改进了我最喜欢的web框架。总之，我认为它使网络广播比我原来计划的更好。