Pytest插件pytest-rerunfailures失败重跑

安装

pip install pytest-rerunfailures

doc

https://github.com/pytest-dev/pytest-rerunfailures

https://pypi.org/project/pytest-rerunfailures/#description

• 当前最新版本11.0(2023-1-12)
• python>=3.7
• pytest 6.0以上

使用方法

第一种用法：装饰器 @pytest.mark.flaky

• 示例代码

  import pytest
  from time import ctime
  
  
  @pytest.mark.flaky(reruns=3, reruns_delay=2)
  def test_a():
      print(ctime())
      import random
  
      assert random.choice([True, False])  # 这个代码你可能直接passed了，随机的
  
  if __name__ == '__main__':
      pytest.main(['-sv',__file__])
  

• 示例输出

  test_demo.py::test_a Sun Jan 29 09:39:35 2023
  RERUN
  test_demo.py::test_a Sun Jan 29 09:39:37 2023
  PASSED
  
  ========================= 1 passed, 1 rerun in 2.13s ==========================
  

• 装饰器中的参数

  • reruns=重跑次数，如果都失败那么这个用例就失败了
  • reruns_delay就是重跑的间隔

• 结果会记录你rerun的次数

• 如果把assert改为

   assert random.choice([1, 0, 0, 0, 0])
  

• 那你的输出很可能就是如下的

  test_demo.py:10: AssertionError
  =========================== short test summary info ===========================
  FAILED test_demo.py::test_a - AssertionError: assert 0
  ========================= 1 failed, 3 rerun in 6.20s ==========================
  

---

• flaky还有一个参数

  • condition：有点类似于skipif中的条件

• 示例代码

  import sys
  
  import pytest
  from time import ctime
  
  @pytest.mark.flaky(reruns=3, reruns_delay=2,condition=sys.platform.startswith('linux'))
  def test_a():
      print(ctime())
      import random
      assert random.choice([1, 0])
  
  
  if __name__ == '__main__':
      pytest.main(['-sv', __file__])
  
  

• 你测试多次会发现，遇到失败的情况压根就不会重跑的，因为condition不满足

第二种用法：命令行

• 跟多数插件一样，它也支持命令行的用法

• 你可以这样用

  $ pytest --reruns 5 --reruns-delay 1
  

• 但是condition并没有这个命令行，它变成了–only-rerun(确切的说也不是变，有点不太一样了)

  # 遇到AssertionError错误就重跑
  $ pytest --reruns 5 --only-rerun AssertionError
  # 遇到AssertionError或者ValueError 就重跑
  $ pytest --reruns 5 --only-rerun AssertionError --only-rerun ValueError
  
  

• 示例代码

  def test_a():
      assert int('a')  # 会产生一个ValueError
  

  pytest -sv --reruns 2 --reruns-delay 1 --only-rerun ValueError test_demo.py
  

  test_demo.py:4: ValueError
  ==================== short test summary info =================================================
  FAILED test_demo.py::test_a - ValueError: invalid literal for int() with base 10: 'a'
  ==================== 1 failed, 2 rerun in 0.06s ===============================================
  

• –only-rerun的意思很明确，只有遇到ValueError才重跑

• 同样的代码，换个参数–rerun-except，除了ValueError才会重跑，遇到ValueError并不重跑

  pytest -sv --reruns 2 --reruns-delay 1 --rerun-except ValueError test_demo.py
  

  test_demo.py:4: ValueError
  ======================== short test summary info =================================================
  FAILED test_demo.py::test_a - ValueError: invalid literal for int() with base 10: 'a'
  ======================== 1 failed in 0.06s ====================================================
  
  

测试AssertionError的时候 貌似跟我预期的不太一样，可能是我眼花了。

• 如果命令行没有-v显示的是R标记

  test_demo.py RRF  # 重跑了2次后失败了 ， 对应底部的1 failed, 2 rerun in 0.06s 
  
  

部分源码

• 命令行

  # command line options
  def pytest_addoption(parser):
      group = parser.getgroup(
          "rerunfailures", "re-run failing tests to eliminate flaky failures"
      )
      group._addoption(
          "--only-rerun",
          action="append",
          dest="only_rerun",
          type=str,
          default=None,
          help="If passed, only rerun errors matching the regex provided. "
          "Pass this flag multiple times to accumulate a list of regexes "
          "to match",
      )
      group._addoption(
          "--reruns",
          action="store",
          dest="reruns",
          type=int,
          default=0,
          help="number of times to re-run failed tests. defaults to 0.",
      )
      group._addoption(
          "--reruns-delay",
          action="store",
          dest="reruns_delay",
          type=float,
          default=0,
          help="add time (seconds) delay between reruns.",
      )
  

• 装饰器参数

  def get_reruns_count(item):
      ...
      if "reruns" in rerun_marker.kwargs:
          ...
          
          
  def get_reruns_delay(item):
  	...
      if "reruns_delay" in rerun_marker.kwargs:
          ...
          
  def get_reruns_condition(item):
      ...
  	    if rerun_marker is not None and "condition" in rerun_marker.kwargs:
  			...