u3d测试运行时暂停
When I started my professional career I had no idea what a unit test looked like. I think I skipped that part on W3Schools. It didn’t matter, because my first employer was not using tests. Before pushing a commit to the SVN repository, I checked some random pages on my local development environment. And after every deployment I made sure the exceptions coming in to my Thunderbird email client were not caused by my code change.
当我开始职业生涯时，我不知道单元测试是什么样的。 我想我在W3Schools上跳过了这一部分。 没关系，因为我的第一位雇主没有使用测试。 在将提交推送到SVN存储库之前，我检查了本地开发环境中的一些随机页面。 并且在每次部署后，我确保进入Thunderbird电子邮件客户端的异常不是由我的代码更改引起的。
When I applied at Mollie almost 7 years ago, they gave me a small coding assignment. I had to create a login screen and add some tests to prove that it worked. The login screen was working in 30 minutes and the rest of the day I was learning how to prove this with automated tests.
当我大约7年前在Mollie申请时，他们给了我一个小的编码任务。 我必须创建一个登录屏幕，并添加一些测试以证明其有效。 登录屏幕在30分钟内可以正常工作，一天的其余时间里，我正在学习如何通过自动化测试来证明这一点。
In my first weeks at Mollie, I quickly learned why tests are so important. Never in my life had I felt so much confidence before releasing a new feature. And because Mollie is processing financial information, confidence in your code is a very nice thing to have.
在莫利(Mollie)的头几个星期，我很快就知道了为什么测试如此重要。 在发布新功能之前，我从未有过如此自信。 而且由于Mollie正在处理财务信息，因此对代码有信心是一件很高兴的事情。
不同类型的测试 (Different types of tests)
Over the years, we’ve added lots of new features. And with every new feature, the number of tests grew as well. In our biggest application, we have over 30 thousand tests of different types:
多年来，我们添加了许多新功能。 随着每一项新功能的推出，测试数量也在增加。 在最大的应用程序中，我们有3万多种不同类型的测试：
Unit tests (27,687x) — This is the biggest group of tests that we have. They test a single unit of our code base, and it does not depend on external services like a database.
单元测试(27,687x) -这是我们拥有的最大的测试组。 他们测试我们代码库的一个单元，它不依赖于数据库等外部服务。
Integration tests (5,348x) — In our integration tests we test bigger parts of the system together. The tests use MySQL, Redis, RabbitMQ and Elasticsearch to make sure everything works and is configured correctly.
集成测试(5,348x) —在我们的集成测试中，我们一起测试系统的更大部分。 测试使用MySQL，Redis，RabbitMQ和Elasticsearch来确保一切正常并且配置正确。
Headless tests (39x) — These tests confirm that the end-to-end flow works as expected. We spin up three servers: one that serves the Mollie application, one that mocks our suppliers and one that acts as one of our customers. We also use a headless Chromium browser to imitate the consumer completing a transaction. Then we verify that the correct calls are made to suppliers, that we call the webhooks to our clients correctly, and many more things.
无头测试(39x) -这些测试证实了端到端流按预期工作。 我们启动了三台服务器：一台为Mollie应用程序服务，一台模拟我们的供应商，另一台充当我们的客户之一。 我们还使用无头的Chromium浏览器来模仿完成交易的消费者。 然后，我们验证是否已正确致电给供应商，是否正确地向客户致电了webhooks以及更多其他内容。
Other tests — We also have some more specific tests for smaller subsystems with specific tooling. For example, we test that our frontend assets are built correctly and we check that our SVG images don’t contain any malicious code.
其他测试 -我们还针对具有特定工具的较小子系统提供了一些更具体的测试。 例如，我们测试了我们的前端资产是否正确构建，并且检查了我们的SVG映像是否不包含任何恶意代码。
For every commit we push to our Git repository, we run all the tests in our self-hosted GitLab instance. Not only for master, but also in our branches. This gives our software developers immediate feedback. To keep the feedback loop useful, we’ve always tried to run all tests in under 5 minutes. Over the years we had to keep improving our test pipelines to make this 5-minute target. We use the following techniques to speed things up:
对于我们推送到Git存储库的每次提交，我们都在自托管的GitLab实例中运行所有测试。 不仅对于大师，而且对于我们的分支机构。 这为我们的软件开发人员提供了立即的反馈。 为了使反馈循环保持有用，我们一直尝试在5分钟内运行所有测试。 多年来，我们必须不断改进测试管道，以实现5分钟的目标。 我们使用以下技术来加快处理速度：
伞兵 (Paratest)
We use the brianium/paratest tool to run our PHPUnit tests in parallel. Paratest is an extension on top of PhpUnit that adds support for parallel testing. For our unit tests, we could just add Paratest and it works without any configuration!
我们使用brianium / paratest工具并行运行我们PHPUnit测试。 Paratest是PhpUnit之上的扩展，增加了对并行测试的支持。 对于我们的单元测试，我们可以只添加Paratest，它无需任何配置即可工作！
Because our integration tests are using external services, we had to duplicate these services as well if we want to use Paratest. For MySQL and Redis we use separate databases on the same server. We decided to mock the RabbitMQ service completely, and with our use of Elasticsearch it was not a problem to run multiple tests on the same server at the same time.
因为我们的集成测试使用的是外部服务，所以如果要使用Paratest，我们也必须复制这些服务。 对于MySQL和Redis，我们在同一服务器上使用单独的数据库。 我们决定完全模拟RabbitMQ服务，并且通过使用Elasticsearch，可以在同一服务器上同时运行多个测试不是问题。
Running the tests in parallel really speeds up your tests, but now setting up a clean database for every parallel thread was becoming the bottleneck. So instead of creating all the databases from scratch by running the database migrations, we just run them once and use some bash foodoo to mysqldump them into the other databases in parallel as well.
并行运行测试确实可以加快测试速度，但是现在为每个并行线程设置一个干净的数据库已成为瓶颈。 因此，与其通过运行数据库迁移从头开始创建所有数据库，我们只运行它们一次，并使用一些bash foodoo将mysql并行将它们转储到其他数据库中。
We learned the hard way that we have to explicitly wait for the mysql imports to finish. In the past we had some tests fail randomly, because not all the tables existed at the start of the test suite. This only happened when the pipeline runners were busy, causing the imports to take more time than usual.
我们了解了必须明确等待mysql导入完成的艰难方法。 过去，有些测试是随机失败的，因为并不是所有表都存在于测试套件的开头。 这仅在管道运行者很忙时才发生，导致进口花费的时间比平时更多。
并行管道 (Parallel pipelines)
Another way to run tests in parallel is by using fully independent parallel pipeline steps. Every type of test runs in their own step. And our integration tests are even split up over two separate steps. We created a custom Paratest runner that only runs 1/Nth of the tests. Using the parallel property in .gitlab-ci.yml we can specify in how many steps we want to split the tests.
并行运行测试的另一种方法是使用完全独立的并行管道步骤。 每种类型的测试都有自己的步骤。 我们的集成测试甚至分为两个单独的步骤。 我们创建了一个自定义的Paratest运行器，它仅运行1 / N个测试。 使用.gitlab-ci.yml的parallel属性，我们可以指定要拆分测试的步骤。
parallel: 2
to our GitLab pipeline configuration is now enough to split tests over multiple steps. And we can increase this number even more at the cost of more resources.
parallel: 2
即可将测试划分为多个步骤。
而且我们可以以更多的资源为代价来增加这个数目。
MySQL事务 (MySQL transactions)
We run all our integration tests inside a database transaction. This allows us to rollback any changes made by the tests. We just run the SQL query BEGIN before we start a test and run ROLLBACK at the end. This is way faster than running TRUNCATE on all the tables that changed.
我们在数据库事务中运行所有集成测试。 这使我们可以回滚测试所做的任何更改。 我们只是在开始测试之前运行SQL查询BEGIN然后在最后运行ROLLBACK 。 这比在所有已更改的表上运行TRUNCATE更快。
检查慢速测试 (Check for slow tests)
We use an open source PHPUnit listener johnkary/phpunit-speedtrap that outputs which of the tests take longer than a specified threshold. It’s very easy to install and configure. Our output looks like this:
我们使用开源PHPUnit侦听器johnkary / phpunit-speedtrap ，该侦听器输出哪些测试花费的时间超过指定的阈值。 安装和配置非常容易。 我们的输出如下所示：
When you know which tests are slow, you can investigate how to make them faster. Maybe you are doing requests to external services, or maybe there’s a sleep() method you need to mock away.
当您知道哪些测试速度很慢时，您可以研究如何使其更快。 也许您正在向外部服务发出请求，或者可能需要嘲弄一个sleep()方法。
GitLab运行程序规格 (GitLab runner specs)
Because we host our own GitLab in Google Cloud, we have the freedom to give the pipeline runners the specs that we want. We used to have instances of the type n2-standard-16 for our runners (16 vCPUs and 64GB ram), but we realised that for tests we only needed the CPUs. We switched to instances of the n2-highcpu-16 type, which has 16 vCPUs and only 16GB of ram. This saved us quite some money without slowing the pipelines down at all. With auto scaling we try to always have runners available to pick up new pipelines, without having a lot of unused resources.
由于我们在Google Cloud中托管了自己的GitLab，因此我们可以自由地为管道运行者提供所需的规格。 我们曾经为跑步者使用n2-standard-16类型的实例(16个vCPU和64GB内存)，但是我们意识到，对于测试，我们只需要CPU。 我们切换到n2-highcpu-16类型的实例，该实例具有16个vCPU和仅16GB的RAM。 这为我们节省了很多钱，而丝毫没有减慢管道的速度。 通过自动缩放，我们尝试始终使运行程序可以使用新的管道，而不会有大量未使用的资源。
管道之间的缓存 (Caching between pipelines)
One easy way to speed up your pipelines is to use caching whenever possible. We cache our vendor/ and node_modules/ directories for respectively our PHP and Node dependencies.
一种加快管道速度的简单方法是尽可能使用缓存。 我们分别为我们PHP和Node依赖项缓存我们的vendor/和node_modules/目录。
On the master branch we use cache policy "pull-push" so new dependencies are also pushed to the cache. For branches we use cache policy "pull" to prevent adding dependencies to the cache that are not merged to master yet. The downside is that new dependencies always will be downloaded when the pipeline runs in a branch, but we don’t add new dependencies that often. Using a dedicated cache per branch would slow down all first pipelines for a new branch, and caching for example the Composer cache directory would slow down all runs in master.
在master分支上，我们使用缓存策略"pull-push"因此新的依赖项也被推送到缓存中。 对于分支，我们使用缓存策略"pull"来防止向缓存添加尚未合并到主服务器的依赖项。 不利的一面是，当管道在分支中运行时，总是会下载新的依赖关系，但是我们并不经常添加新的依赖关系。 每个分支使用专用的缓存会减慢新分支的所有第一个管道，例如缓存Composer缓存目录会减慢master中所有运行的速度。
When using caches, don’t forget that these caches also have to be downloaded into the runner. Sometimes it’s faster to quickly build something than to use a cache.
使用缓存时，请不要忘记这些缓存也必须下载到运行程序中。 有时，快速构建某些东西比使用缓存要快。
将本地运行的测试与您的IDE集成在一起 (Integrate running tests locally with your IDE)
Although the tests run faster in the CI pipelines, being able to run just a couple of tests locally can save a lot of time. Most of our developers use PhpStorm and the integration with PHPUnit and xDebug work out of the box for us. This also helps to save resources for other pipelines.
尽管测试在CI管道中的运行速度更快，但是仅在本地运行几个测试就可以节省大量时间。 我们的大多数开发人员都使用PhpStorm，并且与PHPUnit和xDebug的集成对我们来说是开箱即用的。 这也有助于节省其他管道的资源。
结论 (Conclusion)
One of the best practises defined by extreme programming (XP) is to keep your build and tests under 10 minutes. Martin Fowler writes:
极限编程(XP)定义的最佳实践之一是将构建和测试时间控制在10分钟以内。 马丁·福勒写道 ：
“It’s worth putting in concentrated effort to make it happen, because every minute you reduce off the build time is a minute saved for each developer every time they commit.”
“值得全力以赴来实现这一目标，因为每减少一分钟的构建时间，就为每个开发人员每次提交节省了一分钟。”
I hope this article gave you some good suggestions on how to speed up your build.
希望本文为您提供有关如何加快构建速度的良好建议。
Are you looking for a nice place to work? We are hiring engineers! Check out our vacancies on https://jobs.mollie.com/.
您是否在寻找工作的好地方？ 我们正在招聘工程师！ 在https://jobs.mollie.com/上查看我们的空缺。
翻译自: https://blog.mollie.com/how-to-run-over-30k-tests-in-under-5-minutes-b43907e88d51
u3d测试运行时暂停
12万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
