有条件覆盖

Recently I had to add python3.8 for our Python linter (the strictest one in existence): wemake-python-styleguide. And during this straight-forward (at first look) task, I have found several problems with test coverage that were not solved in Python community at all.

最近，我不得不为我们的Python python3.8 (现有最严格的工具)添加python3.8 ： wemake-python-styleguide 。 在这个直接的任务(乍看之下)中，我发现了一些测试覆盖率问题，这些问题在Python社区中根本没有解决。

Let's dig into it.

让我们深入研究。

基于环境的逻辑 (Environment-based logic)

The thing about this update was that python3.8 introduced a new API. Previously we had visit_Num, visit_Str, visit_NameConstant methods (we don't really care about what they do, only names are important), but now we have to use visit_Constant method instead.

关于此更新的事情是python3.8引入了新的API。 以前我们有visit_Num ， visit_Str ， visit_NameConstant方法(我们实际上并不关心它们的作用，只有名称很重要)，但是现在我们不得不改用visit_Constant方法。

Well, we will have to write some compatibility layer in our app to support both new python3.8, previous python3.6, and python3.7 releases.

好吧，我们将必须在我们的应用程序中编写一些兼容性层，以支持新的python3.8 ，以前的python3.6和python3.7版本。

My first attempt was to create a route_visit function to do the correct thing for all releases. To make it work I also had to define PY38 constant like so:

我的第一个尝试是创建一个route_visit函数，以对所有发行版执行正确的操作。 为了使其正常工作，我还必须像这样定义PY38常数：

import sys

PY38 = sys.version_info >= (3, 8)

if PY38:
    route_visit = ...  # new logic: 3.8+
else:
    route_visit = ...  # old logic: 3.6, 3.7

And it worked pretty well. I was able to run my tests successfully. The only thing broken was coverage. We use pytest and pytest-cov to measure coverage in our apps, we also enforce --cov-branch and --cov-fail-under=100 policies. Which enforce us to cover all our code and all branches inside it.

而且效果很好。 我能够成功运行测试。 唯一破坏的是覆盖范围。 我们使用pytest和pytest-cov来衡量应用程序的覆盖率，我们还执行--cov-branch和--cov-fail-under=100策略。 这迫使我们覆盖我们所有的代码以及其中的所有分支。

Here's the problem with my solution: I was either covering if PY38: branch on python3.8 build or else: branch on other releases. I was never covering 100% of my program. Because it is literally impossible.

这是我的解决方案的问题：我正在研究if PY38:在python3.8 build上if PY38:分支，还是在其他发行版上进行了else:分支。 我从来没有覆盖我的程序的100％。 因为这实际上是不可能的。

常见的陷阱 (Common pitfalls)

Open-source libraries usually face this problem. They are required to work with different python versions, 3rd party API changes, backward compatibility, etc. Here are some examples that you probably have already seen somewhere:

开源库通常会遇到此问题。 必须使用它们才能使用不同的python版本，第三方API更改，向后兼容性等。以下是您可能已经在某处看到的一些示例：

try:
    import django
    HAS_DJANGO = True
except ImportError:
    HAS_DJANGO = False

Or this was a popular hack during 2 / 3 days:

或者，这是一个在流行的黑客2 / 3日：

try:
    range_ = xrange  # python2
except NameError:
    range_ = range  # python3

With all these examples in mind, one can be sure that 100% of coverage is not possible. The common scenario to still achieve a feeling of 100% coverage for these cases was:

考虑到所有这些示例，可以确定不可能100％覆盖。 对于这些情况，仍然可以实现100％覆盖的感觉的常见情况是：

1. Using # pragma: no cover magic comment to exclude a single line or a whole block from coverage

   使用# pragma: no cover魔术注释可从覆盖范围中排除单行或整块

2. Or writing every compatibility related check in a special compat.py that were later ommited from coverage

   或将所有与兼容性相关的检查写在特殊的compat.py中，以后再从覆盖范围中删除

Here's how the first way looks like:

这是第一种方式的样子：

try:  # pragma: no cover
    import django
    HAS_DJANGO = True
except ImportError:  # pragma: no cover
    HAS_DJANGO = False

Let's be honest: these solutions are dirty hacks. But, they do work. And I personally used both of them countless times in my life.

坦白地说：这些解决方案是肮脏的骇客。 但是，它们确实起作用。 而且我个人一生都使用了无数次。

Here's the interesting thing: aren't we supposed to test these complex integrational parts with the most precision and observability? Because that's where our application breaks the most: integration parts. And currently, we are just ignoring them from coverage and pretending that this problem does not exist.

这是一件有趣的事情：我们不应该以最精确和可观察的方式测试这些复杂的集成零件吗？ 因为那是我们的应用程序最大的缺点：集成部分。 当前，我们只是从覆盖范围中忽略它们，并假装不存在此问题。

And for this reason, this time I felt like I am not going to simply exclude my compatibility logic. I got an idea for a new project.

由于这个原因，这次我觉得我不会简单地排除兼容性逻辑。 我有一个新项目的主意。

有条件覆盖 (Conditional coverage)

My idea was that # pragma comments can have more information inside them. Not just no cover, but no cover when?. That's how coverage-conditional-plugin was born. Let's use it and see how it works!

我的想法是# pragma注释注释可以在其中包含更多信息。 不仅no cover ， no cover when?也no cover when? 。 这就是coverage-conditional-plugin诞生。 让我们使用它，看看它是如何工作的！

First, we would need to install it:

首先，我们需要安装它：

pip install coverage-conditional-plugin  # pip works, but I prefer poetry

And then we would have to configure coverage and the plugin itself:

然后我们必须配置 coverage和插件本身：

[coverage:run]
# Here we specify plugins for coverage to be used:
plugins =
  coverage_conditional_plugin

[coverage:coverage_conditional_plugin]
# Here we specify our pragma rules:
rules = # we are going to define them later.

Notice this rules key. It is the most important thing here. The rule (in this context) is some predicate that tells: should we include lines behind this specific pragma in our coverage or not. Here are some examples:

注意此rules键。 这是最重要的事情。 该规则(在这方面)是一些谓词告诉：我们应该包括这个特定的背后线路pragma在我们研究范围或没有。 这里有些例子：

[coverage:coverage_conditional_plugin]
# Here we specify our pragma rules:
rules =
  "sys_version_info >= (3, 8)": py-gte-38
  "sys_version_info < (3, 8)": py-lt-38
  "is_installed('django')": has-django
  "not is_installed('django')": has-no-django

It is pretty clear what we are doing here: we are defining pairs of predicates to include this code if some condition is true and another code in the opposite case.

很清楚我们在这里做什么：如果某些条件为真，我们将定义成对谓词以包含此代码，而在相反情况下，则要包含另一条谓词。

Here's how our previous examples would look like with these magic comments:

这些神奇的注释如下所示，这就是我们前面的示例：

import sys

PY38 = sys.version_info >= (3, 8)

if PY38:  # pragma: py-lt-38
    route_visit = ...  # new logic: 3.8+
else:  # pragma: py-gte-38
    route_visit = ...  # old logic: 3.6, 3.7

What does it say? If we are running on py-lt-38 ignore if PY38: part. But, cover else: case. Because it is going to be executed and we know it. And we need to know how good did we cover it. On the other hand, if we are running on py-gte-38 then cover if PY38: case and leave else: alone.

它说什么？ 如果我们在py-lt-38上运行，请忽略if PY38: part。 但是， else:案例。 因为它将要执行，我们知道。 而且我们需要知道我们覆盖了它的质量。 另一方面，如果我们在py-gte-38上运行，则覆盖if PY38: case， if PY38: else:独自。

And we can test that everything works correctly. Let's add some nonsense into our PY38 branch to see if it is going to be covered by python3.8 build:

我们可以测试一切正常。 让我们在我们的PY38分支中添加一些废话，看看它是否将被python3.8 build覆盖：

As we can see: green signs show which lines were fully covered, the yellow line indicates that branch coverage was not full, and the red line indicates that the line was not covered at all. And here's an example of grey or ignored lines under the opposed condition:

正如我们所看到的：绿色标志表明哪些行被完全覆盖，黄线表明分支覆盖不充分，红线表明该行根本没有被覆盖。 这是相反情况下的灰色或忽略线条的示例：

Here you can find the full real-life source code for this sample.

在这里，您可以找到此示例的完整的真实源代码。

And here's one more example with django to show you how external packages can be handled:

这是django的另一个示例，向您展示如何处理外部软件包：

try:  # pragma: has-no-django
    import django
    HAS_DJANGO = True
except ImportError:  # pragma: has-django
    HAS_DJANGO = False

We use the same logic here. Do we have django installed during tests (we have a little helper function is_installed to tell us)? If so, cover try:. If not, cover except ImportError: branch. But always cover something.

我们在这里使用相同的逻辑。 在测试过程中是否已安装django (我们已安装了一个is_installed辅助函数来告诉我们)？ 如果是这样，请try:覆盖try: 。 如果不是，则覆盖except ImportError:分支except ImportError: 。 但是总要掩盖一些东西 。

结论 (Conclusion)

I hope you got the idea. Conditional coverage allows you to add or ignore lines based on predicates and collecting required bits of coverage from every run, not just ignoring complex conditions and keeping our eyes wide shut. Remember, that the code we need to cover the most!

我希望你有主意。 有条件的覆盖率使您可以基于谓词添加或忽略行，并从每次运行中收集所需的覆盖率位，而不仅仅是忽略复杂的条件并使我们睁大眼睛。 请记住，我们需要涵盖的代码最多！

By the way, we have all kinds of helpers to query your environment:

顺便说一下，我们有各种各样的帮助者来查询您的环境：

• os_environ for env variables

  os_environ用于环境变量

• patform_release and platform_version for OS-based values

  patform_release和platform_version用于基于OS的值

• pkg_version that returns package version information (as its name says)

  pkg_version返回软件包版本信息(如其名称所示)

• and many others!
  还有很多！

This little plugin is going to be really helpful for library authors that have to deal with compatibility and unfixed environments. And coverage-conditional-plugin will surely cover their backs! Give it a star on Github if you like this idea. And read the project docs to learn more.

这个小插件将对必须处理兼容性和非固定环境的图书馆作者很有帮助。 coverage-conditional-plugin一定会掩盖他们的后背！ 如果您喜欢这个想法，请在Github上给它加星标 。 并阅读项目文档以了解更多信息。

翻译自: https://habr.com/en/post/491242/