在使用pytest的时候,希望在teardown阶段能够产生最终fail的结果,但是不要影响teardown中所有步骤的执行,立即想到了pytest-assume,然而验证的结果出乎我的意料之外。
基本case
- 基本测试用例如下
pytest@Ubuntu18:~$ cat test_demo.py
import pytest
class Test_Demo():
def setup_method(self):
pass
def teardown_method(self):
pass
def test_case(self):
pass
pytest@Ubuntu18:~$
- 运行结果肯定是PASS
pytest@Ubuntu18:~$ pytest -s test_demo.py
============================================================================================================ test session starts ============================================================================================================
platform linux2 -- Python 2.7.17, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: /home/pytest
plugins: assume-2.4.3
collected 1 item
test_demo.py .
========================================================================================================= 1 passed in 0.01 seconds ==========================================================================================================
pytest@Ubuntu18:~$
在setup使用assume
- 修改脚本,在setup中加入assume
pytest@Ubuntu18:~$ cat test_demo.py
import pytest
class Test_Demo():
def setup_method(self):
pytest.assume(False)
def teardown_method(self):
pass
def test_case(self):
pass
pytest@Ubuntu18:~$
- 运行结果提示在setup的时候fail了
pytest@Ubuntu18:~$ pytest -s test_demo.py
============================================================================================================ test session starts ============================================================================================================
platform linux2 -- Python 2.7.17, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: /home/pytest
plugins: assume-2.4.3
collected 1 item
test_demo.py F
================================================================================================================= FAILURES ==================================================================================================================
____________________________________________________________________________________________________________ Test_Demo.test_case ____________________________________________________________________________________________________________
self = <pytest_assume.plugin.AssumeContextManager object at 0x7fcbfa93a450>, expr = False, msg = ''
def __call__(self, expr, msg=""):
__tracebackhide__ = True
self._enter_from_call = True
with self:
if msg:
assert expr, msg
else:
> assert expr
E FailedAssumption:
E 1 Failed Assumptions:
E
E test_demo.py:5: AssumptionFailure
E >> pytest.assume(False)
E AssertionError: assert False
.local/lib/python2.7/site-packages/pytest_assume/plugin.py:141: FailedAssumption
========================================================================================================= 1 failed in 0.04 seconds ==========================================================================================================
pytest@Ubuntu18:~$
在teardown加入assune
- 修改脚本
pytest@Ubuntu18:~$ cat test_demo.py
import pytest
class Test_Demo():
def setup_method(self):
pass
def teardown_method(self):
pytest.assume(False)
def test_case(self):
pass
pytest@Ubuntu18:~$
- 结果竟然是PASS???!!!
pytest@Ubuntu18:~$ pytest -s test_demo.py
============================================================================================================ test session starts ============================================================================================================
platform linux2 -- Python 2.7.17, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: /home/pytest
plugins: assume-2.4.3
collected 1 item
test_demo.py .
========================================================================================================= 1 passed in 0.03 seconds ==========================================================================================================
pytest@Ubuntu18:~$
改用pytest.fail
- 修改脚本
pytest@Ubuntu18:~$ cat test_demo.py
import pytest
class Test_Demo():
def setup_method(self):
pass
def teardown_method(self):
pytest.fail()
def test_case(self):
pass
pytest@Ubuntu18:~$
- 没有问题,case fail了
pytest@Ubuntu18:~$ pytest -s test_demo.py
============================================================================================================ test session starts ============================================================================================================
platform linux2 -- Python 2.7.17, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir: /home/pytest
plugins: assume-2.4.3
collected 1 item
test_demo.py .E
================================================================================================================== ERRORS ===================================================================================================================
_________________________________________________________________________________________________ ERROR at teardown of Test_Demo.test_case __________________________________________________________________________________________________
self = <test_demo.Test_Demo instance at 0x7f3dab348910>
def teardown_method(self):
> pytest.fail()
E Failed: <Failed instance>
test_demo.py:8: Failed
===================================================================================================== 1 passed, 1 error in 0.02 seconds =====================================================================================================
pytest@Ubuntu18:~$
结论
- 对于pytest.assume,如果放在teardown里面,是不会触发最后显示测试用例结果为fail的
一种解决方法
- 请出pytest-check
pytest@Ubuntu18:~$ pip3 install pytest-check
- 修改脚本
pytest@Ubuntu18:~$ cat test_demo.py
import pytest_check as check
class Test_Demo():
def setup_method(self):
pass
def teardown_method(self):
check.is_true(False)
def test_case(self):
pass
pytest@Ubuntu18:~$
- 运行
pytest@Ubuntu18:~$ pytest -s test_demo.py
============================================================================================================ test session starts ============================================================================================================
platform linux -- Python 3.6.9, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/pytest
plugins: pytest_check-1.0.2
collected 1 item
test_demo.py .E
================================================================================================================== ERRORS ===================================================================================================================
_________________________________________________________________________________________________ ERROR at teardown of Test_Demo.test_case __________________________________________________________________________________________________
FAILURE:
assert False
+ where False = bool(False)
test_demo.py:8 in teardown_method() -> check.is_true(False)
------------------------------------------------------------
Failed Checks: 1
========================================================================================================== short test summary info ==========================================================================================================
ERROR test_demo.py::Test_Demo::test_case
======================================================================================================== 1 passed, 1 error in 0.02s =========================================================================================================
pytest@Ubuntu18:~$
- 可见pytest-check对比pytest-assume的优势为
- 可以显示出更加详细的出错信息
- 可以克服在teardown阶段调用无法触发case fail的缺陷