开源Python-单元测试

python开源系列文章

• 开源Python-单元测试
• 开源python-文档撰写
• 开源python-代码规范与检查
• 开源python-打包发布
• 开源python-自动集成
• 开源python-向transformers学习

单元测试

测试可以保证上传的代码没有明显硬伤。比如我之前在github.com/LongxingTan/Time-series-prediction改动之后，跑不了了，导致issue里被提了很多非常基础的错误。就显得很不专业了

基本思路

• unittest，integration test
• 采用pytest或unittest等单元测试框架
• 结合github actions进行自动测试

基本概念

• test fixture ：表示执行一个或多个测试所需的准备, 以及任何关联的清理操作。例如, 创建临时或代理数据库、目录或启动服务器进程。setUp（）和tearDown（）方法
• test case ：测试用例是最小的测试单元。它检查特定的输入集的响应。单元测试提供了一个基类测试用例TestCase, 可用于创建新的测试用例。
• test suite ：测试套件是测试用例、测试套件或两者的集合 TestSuite实例。它用于聚合应一起执行的测试。
• test runner ：测试运行程序是协调测试执行并向用户提供结果的组件。运行程序可以使用图形界面、文本界面或返回特殊值来指示执行测试的结果。TextTestRunner类提供的run（）方法

实践

我这里采用tensorflow框架，所以具体可以参考transformer和tensorflow/models的实践。由于是接触不深，这里决定采用unittest自带的测试框架进行测试。

• 注意整个结构，tests文件夹和源码文件夹并列，每个文件夹增加__init__.py以包的形式
• test文件夹和文件名以test_开头
• unittest继承unitTest.TestCase，并且类下没有初始化init方法

示例

• unittest + codecov + CI
• unittest + codecov + CI
• pytest + codecov + CI
• pytest + codecov + CI

pytest

• conftest 存放fixture的配置文件

code coverage

检查运行自动化测试的时候，哪些代码行、块真正被执行了。是衡量 unittest 全面程度的重要指标。

• https://github.com/settings/installations/30030808

• quick start

• github tuturial

步骤

• 1 从setup中得到token，加入到github repo secrets
• 2 repo中添加codecov.yml配置
• 3 CI github actions中增加上传结果

transformers

https://github.com/huggingface/transformers/blob/main/tests/models/bert/test_modeling_tf_bert.py

其中有关于tf模型如何做测试，单个文件里主要有如下三个类

• TFBertModelTester
  • 准备config和input
• TFBertModelTest
  • 测试config
  • 测试model
• TFBertModelIntegrationTest
  • 测试了inference model?

tensorflow-addons

其中，关于为开源做贡献的部分，写了一段关于测试的

• If you add a new feature, you should add tests to ensure that new code changes doesn’t introduce bugs.

• If you fix a bug, you should add a tests which fails before your patch and passes after your patch.

• We use Pytest to write tests. We encourage you to read the documentation, but you’ll find a quick summary here:

  • If you’re testing code written in xxx.py, your tests should be in xxx_test.py.
  • In xxx_test.py, all functions starting with test_ are collected and run by Pytest.
  • Tests are run with the TF 2.x behavior, meaning eager mode my default, unless you use a tf.function.
  • Ensure something is working by using assert. For example: assert my_variable in my_list.
  • When comparing numpy arrays, use the testing module of numpy. Note that since TensorFlow ops often run with float32 of float16, you might need to increase the default atol and rtol. You can take a look at the default values used in the TensorFlow repository.
  • Prefer using your code’s public API when writing tests. It ensures future refactoring is possible without changing the tests.
  • When testing multiple configurations, prefer using parametrize rather than for loops for a clearer error report.
  • Running all the tests in a single file should take no more than 5 seconds. You very rarely need to do heavy computation to test things. Your tests should be small and focused on a specific feature/parameter.
  • Don’t be afraid to write too many tests. This is fine as long as they’re fast.

参考

• https://github.com/philschmid/github-actions/blob/master/python/run-unittest-on-pr-open.yaml
• huggingface-transformers
• pytroch-forecasting
• https://theaisummer.com/unit-test-deep-learning/
• test-driven development with python
• Python.Testing.with.pytest.2017.9
• Unit Testing Principles, Practices, and Patte
• unitest: https://zhuanlan.zhihu.com/p/48609722