-
如下图,我们可以看到
allure
报告的总览,里边的一些特性是可以自定义设置的。
1 Environment设置
-
Environment
可以理解为环境变量; -
默认为空;
-
可以自己设置。
1.1 设置方法
-
在存放测试报目录下创建
environment.properties
或者environment.xml
文件; -
而测试报告目录是使用
--alluredir
指定的目录,比如:
--alluredir allure-results
-
比如之前提到的用例:
pytest -n auto --alluredir=allure-results test_xdist.py
在这里插入图片描述
1.2 创建文件
-
environment.properties
:
-
Browser=Chrome
-
Browser.Version=111.0.5563.65
-
Env=Test
-
IP=192.168.1.133
-
Allure-Pytest.Version=2.8.12
-
运行后查看
Environment
: -
或者创建
environment.xml
:
-
environment>
-
<parameter>
-
<key>Browser</key>
-
<value>Chrome</value>
-
</parameter>
-
<parameter>
-
<key>Browser.Version</key>
-
<value>111.0.5563.65</value>
-
</parameter>
-
<parameter>
-
<key>Env</key>
-
<value>Test</value>
-
</parameter>
-
<parameter>
-
<key>IP</key>
-
<value>192.168.1.133</value>
-
</parameter>
-
<parameter>
-
<key>Allure-Pytest.Version</key>
-
<value>2.8.12</value>
-
</parameter>
-
</environment>
2 Categories设置
-
Categories
即分类,测试用例结果的分类; -
默认有两种分类:
-
# Product defects 产品缺陷(测试结果:failed)
-
# Test defects 测试缺陷(测试结果:error/broken)
-
可以自定义分类。
2.1 设置方式
-
和
environment
方式一样,在allure-results
目录中创建categories.json
文件
2.2 创建文件
在这里插入图片描述
-
[
-
{
-
"name": "Ignored tests",
-
"matchedStatuses": ["skipped"]
-
},
-
{
-
"name": "Infrastructure problems",
-
"matchedStatuses": ["broken", "failed"],
-
"messageRegex": ".*bye-bye.*"
-
},
-
{
-
"name": "Outdated tests",
-
"matchedStatuses": ["broken"],
-
"traceRegex": ".*FileNotFoundException.*"
-
},
-
{
-
"name": "Product defects",
-
"matchedStatuses": ["failed"]
-
},
-
{
-
"name": "Test defects",
-
"matchedStatuses": ["broken"]
-
}
-
]
-
参数说明:
-
name:分类名称
-
matchedStatuses:测试用例的运行状态,默认["failed", "broken", "passed", "skipped", "unknown"]
-
messageRegex:测试用例运行的错误信息,默认.* ,通过正则匹配
-
traceRegex:测试用例运行的错误堆栈信息,默认.* ,通过正则匹配
-
写一个用例,验证下:
-
# -*- coding:utf-8 -*-
-
# 作者:虫无涯
-
# 日期:2023/3/20
-
# 文件名称:test_yyy.py
-
# 作用:allure特性categories验证
-
# 联系:VX(NoamaNelson)
-
# 博客:https://blog.csdn.net/NoamaNelson
-
import pytest
-
import time
-
class TestCase01():
-
def test_case_01(self):
-
time.sleep(1)
-
print("case01$$$$$$$$$$$$$$$$$$$$$")
-
assert1 == 2
-
def test_case_02(self):
-
time.sleep(1)
-
print("case02$$$$$$$$$$$$$$$$$$$$$")
-
assert3 == 3
-
def test_case_03(self):
-
time.sleep(1)
-
print("case03$$$$$$$$$$$$$$$$$$$$$")
-
assert"is"in"is_you"
-
def test_case_04(self):
-
time.sleep(1)
-
print("case04$$$$$$$$$$$$$$$$$$$$$")
-
assert5 < 10
-
def test_case_05(self):
-
time.sleep(1)
-
print("case05$$$$$$$$$$$$$$$$$$$$$")
-
assert222 == 333
-
def test_case_06(self):
-
time.sleep(1)
-
print("case06$$$$$$$$$$$$$$$$$$$$$")
-
assert444 > 666
-
class TestCase02():
-
def test_case_07(self):
-
time.sleep(1)
-
print("case07$$$$$$$$$$$$$$$$$$$$$")
-
assert10/2 == 5.0
-
def test_case_08(self):
-
time.sleep(1)
-
print("case08$$$$$$$$$$$$$$$$$$$$$")
-
assert"num"in"num_list"
-
def test_case_09(self):
-
time.sleep(1)
-
print("case08$$$$$$$$$$$$$$$$$$$$$")
-
assert"num1"in"num_list"
-
if __name__ == '__main__':
-
pytest.main(["-s", "test_yyy.py"])
-
运行命令:
-
pytest -n auto --alluredir=allure-results test_yyy.py
-
运行命令:
allure serve allure-results
-
查看结果:
3 关于Flaky test
3.1 Flaky test介绍
-
Flaky test
在被测对象和测试条件都不变的情况下,有时候失败、有时候成功的测试; -
实际上就是不稳定的测试,或者随机失败(随机成功)的测试;
-
标记成Flaky是为了当用例失败的情况下,我们能获取足够详细的信息。
3.2 产生Flaky Tests的原因
-
异步等待;
-
并发;
-
资源泄露;
-
远程服务;
-
测试依赖性。
3.3 Flaky安装
pip3 install pytest-ignore-flaky
-
C:\Users\Administrator>pip3 install pytest-ignore-flaky
-
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
-
Collecting pytest-ignore-flaky
-
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/22/bf/4a670d28c8c37569e26536c068d83b37a01aea9fff9a45a03ae3be5344b9/pytest_ignore_flaky-2.0.0-py3-none-any.whl (3.9 kB)
-
Requirement already satisfied: pytest>=6.0in d:\python37\lib\site-packages (from pytest-ignore-flaky) (6.2.4)
-
Requirement already satisfied: py>=1.8.2in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (1.10.0)
-
Requirement already satisfied: colorama in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (0.4.4)
-
Requirement already satisfied: attrs>=19.2.0in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (20.3.0)
-
Requirement already satisfied: atomicwrites>=1.0in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (1.4.0)
-
Requirement already satisfied: iniconfig in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (1.1.1)
-
Requirement already satisfied: importlib-metadata>=0.12in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (2.1.1)
-
Requirement already satisfied: toml in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (0.10.2)
-
Requirement already satisfied: packaging in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (20.8)
-
Requirement already satisfied: pluggy<1.0.0a1,>=0.12in d:\python37\lib\site-packages (from pytest>=6.0->pytest-ignore-flaky) (0.13.1)
-
Requirement already satisfied: zipp>=0.5in d:\python37\lib\site-packages (from importlib-metadata>=0.12->pytest>=6.0->pytest-ignore-flaky) (1.2.0)
-
Requirement already satisfied: pyparsing>=2.0.2in d:\python37\lib\site-packages (from packaging->pytest>=6.0->pytest-ignore-flaky) (2.4.7)
-
Installing collected packages: pytest-ignore-flaky
-
Successfully installed pytest-ignore-flaky-2.0.0
3.4 Flaky使用
-
再写一个用例:
-
# -*- coding:utf-8 -*-
-
# 作者:虫无涯
-
# 日期:2023/3/20
-
# 文件名称:test_yyy.py
-
# 作用:allure特性categories验证
-
# 联系:VX(NoamaNelson)
-
# 博客:https://blog.csdn.net/NoamaNelson
-
import pytest
-
import time
-
class TestCase01():
-
def test_case_01(self):
-
time.sleep(1)
-
print("case01$$$$$$$$$$$$$$$$$$$$$")
-
assert1 == 2
-
def test_case_02(self):
-
time.sleep(1)
-
print("case02$$$$$$$$$$$$$$$$$$$$$")
-
assert3 == 3
-
def test_case_03(self):
-
time.sleep(1)
-
print("case03$$$$$$$$$$$$$$$$$$$$$")
-
assert"is"in"is_you"
-
def test_case_04(self):
-
time.sleep(1)
-
print("case04$$$$$$$$$$$$$$$$$$$$$")
-
assert5 < 10
-
def test_case_05(self):
-
time.sleep(1)
-
print("case05$$$$$$$$$$$$$$$$$$$$$")
-
assert222 == 333
-
def test_case_06(self):
-
time.sleep(1)
-
print("case06$$$$$$$$$$$$$$$$$$$$$")
-
assert444 > 666
-
class TestCase02():
-
def test_case_07(self):
-
time.sleep(1)
-
print("case07$$$$$$$$$$$$$$$$$$$$$")
-
assert10/2 == 5.0
-
def test_case_08(self):
-
time.sleep(1)
-
print("case08$$$$$$$$$$$$$$$$$$$$$")
-
assert"num"in"num_list"
-
@pytest.mark.flaky
-
def test_case_09(self):
-
time.sleep(1)
-
print("case08$$$$$$$$$$$$$$$$$$$$$")
-
assert"num1"in"num_list"
-
if __name__ == '__main__':
-
pytest.main(["-s", "test_yyy.py"])
-
使用命令直接运行用例:
pytest -n auto --alluredir=allure-results test_yyy.py
-
========================================== short test summary info ===========================================
-
FAILED test_yyy.py::TestCase01::test_case_06 - assert444 > 666
-
FAILED test_yyy.py::TestCase01::test_case_01 - assert1 == 2
-
FAILED test_yyy.py::TestCase01::test_case_05 - assert222 == 333
-
FAILED test_yyy.py::TestCase02::test_case_09 - AssertionError: assert'num1'in'num_list'
-
==================================== 4 failed, 5 passed, 1 rerun in5.99s ====================================
-
从上发现被我们使用
@pytest.mark.flaky
标记的用例,断言是失败的,也正常标准失败: -
命令行加上代码:
--ignore-flaky
重新运行:
pytest -n auto --alluredir=allure-results test_yyy.py --ignore-flaky
在这里插入图片描述
-
发现被标记的用例变成了
xfailed
而不是失败了:
3.5 小结
小结1
-
默认情况下,
@pytest.mark.flaky
装饰器标记的测试用例默认会执行; -
当用例执行结果成功时正常执行正常显示用例结果;
-
当用例执行结果失败时,测试用例默认失败重跑一次。
小结2
-
pytest
命令行参数--ignore-flaky
运行@pytest.mark.flaky
标记的测试用例:当用例执行成功时执行结果显示正常; -
当用例执行失败时执行结果显示
XFAIL(skip flaky test failure)
。
行动吧,在路上总比一直观望的要好,未来的你肯定会感 谢现在拼搏的自己!如果想学习提升找不到资料,没人答疑解惑时,请及时加入扣群: 320231853,里面有各种软件测试+开发资料和技术可以一起交流学习哦。
最后感谢每一个认真阅读我文章的人,礼尚往来总是要有的,虽然不是什么很值钱的东西,如果你用得到的话可以直接拿走:
这些资料,对于【软件测试】的朋友来说应该是最全面最完整的备战仓库,这个仓库也陪伴上万个测试工程师们走过最艰难的路程,希望也能帮助到你!