减少Maven构建执行时间

最新推荐文章于 2021-12-22 15:38:43 发布

cunxiedian8614

最新推荐文章于 2021-12-22 15:38:43 发布

阅读量380

点赞数

文章标签： java 数据库测试

Current state

我们有一个包含约100k行代码的多模块Spring Web MVC 4应用程序。我们使用Maven和Azure DevOps。一个简单的构建管道可以构建并运行所有的单元测试-大约2.8k。好吧，说实话，我将其中大多数称为组件测试，甚至称为集成测试。让我们在开始时清除一些定义。

What is a unit test?

There are many definitions of a unit test. I like the one by Roy Osherove:

“单元测试是一段自动化的代码，它调用要测试的工作单元，然后检查有关该单元的单个最终结果的一些假设。单元测试几乎总是使用单元测试框架编写的。它可以轻松编写并快速运行。它是值得信赖，可读和可维护的。只要生产代码未更改，其结果就一致。”

There is a nice article about unit tests in Martin Fowler's bliki so I won't go into much details.

假设我们从单元测试中期望以下内容：

快速反馈（必须快速运行）简短（少量代码）测试一件事（一个逻辑断言）失败是有原因的

What is a component test?

Again a nice article in bliki.

Since our application uses Spring Data JPA, we have lots of repositories. Many of them contain custom query methods. We use 小号pring Test DBUnit to help us out testing them. The idea is simple - you setup a mockup database (H2), import some test data before a test, run a test, assert and then cleanup the DB end the end. Since this approach creates the DB and some (small) spring context with all necessary beans, I treat these tests as component tests.

这些测试比单元测试运行更长的时间。

What is an integration test?

Ťhe bliki defines the purpose of integration tests as:

顾名思义，集成测试的目的是测试是否有许多单独开发的模块可以按预期工作。

在我们的例子中，我们有Spring Web MVC控制器。这些控制器公开了RESTful API。对于这些，我们通常创建一个完整的spring上下文（带有模拟的H2数据库）并使用MockMvc对其进行检查。此类测试的好处在于，您可以测试API，包括所有基础服务（业务逻辑）和存储库（持久性）。巨大的缺点是执行时间。创建Spring上下文会花费很长时间（取决于您的应用程序的大小）。在许多情况下，上下文会被破坏并重新创建。所有这些加起来总计需要总构建时间。

What tests do I need?

If you are now biased to lean towards just unit tests, as they are fast to run, and want to ditch your component/integration tests as they are slow - hang on ... not so fast. You need The tests at all levels. In a reasonable ratio. In large applications you might split the test runs to fast/slow to speedup the feedback loop. You also need UI driven tests, system tests, performance tests ... Ok, the whole topic about high quality testing of a software is very complex and you might start reading here if you are interested. Let's get back to our topic for now.

The path to improve the situation

There is a nice article written about what you can do. It tells you to re-think whether you need that many integration tests and rewrite them with unit tests. As described above, do it with caution.

尽早定义测试策略很重要。否则，问题将变得太大，无法最终得出结论，即您要处理许多旧的测试，而这些测试现在更改成本很高。在公司和大型项目中，这种情况经常发生，以至于无法忽略。不论是否旧有，我们仍然可以通过合理的努力和积极的投资回报来做些事情。

Spring context

由于多种原因，我们的Spring环境在测试运行期间总共创建了17次。每次大约需要30秒才能开始。这是8.5分钟添加到测试运行中。您可以利用以上文章中的建议来确保仅创建一次上下文。我们可以在这里节省8分钟。

DBUnit

如果使用DBUnit，则每个测试均始于将数据导入DB并以清理结束。为此有几种策略。 Spring Test DBUnit期望在测试方法上有这些注释：

@DatabaseSetup(value = "data.xml")
@DatabaseTearDown(value = "data.xml")

默认策略是CLEAN_INSERT对于“设置”和“拆卸”，实际上是从受影响的表中删除所有数据，然后再次插入它们。使用纯DBUnit，您可以显式配置它，例如：

AbstractDatabaseTester databaseTester = new DataSourceDatabaseTester(dataSource);
databaseTester.setSetUpOperation(DatabaseOperation.CLEAN_INSERT);
databaseTester.setTearDownOperation(DatabaseOperation.DELETE_ALL);

考虑将这些策略更改为更快的速度，例如代替CLEAN_插入对于安装，仅使用插入。当然，如果先前的测试未正确清理您的表，这将导致问题。的CLEAN_插入拆解的可以替换为删除所有甚至更快的一个TRUNCATE_TABLE。有一个陷阱TRUNCATE_TABLE有关H2数据库的更多信息，请参见附录。

我们有大约1200个与DBUnit一起运行的测试。试想一下，如果我们节省了100毫秒的设置/拆卸时间。用相对较少的精力就可以节省4分钟。

Build time analysis

However, our situation was a bit more complicated. Our build execution time grew from about 9 minutes to something about 25 minutes in avarage over the past 6 months.

(the sudden peaks are caused by the fact that we recycle the build agents every now and then and hence the maven cache is gone).

此外，我们经历了一些构建需要长达45分钟的时间，而有些甚至超过1小时。

我们对构建日志进行了简单分析。测试配置为使用以下命令输出到控制台跟踪水平。在那里，我们可以看到某些操作突然花费了比平时更长的时间。例如数据库拆解：

2020-03-13T12:59:21.9660309Z INFO  DefaultPrepAndExpectedTestCase - cleanupData: about to clean up 13 tables=[...]
2020-03-13T12:59:29.0545094Z INFO  MockServletContext - Initializing Spring FrameworkServlet ''

第一条消息是一个测试的数据库拆解，第二条消息正在为另一个测试启动新的spring上下文。请注意时间戳记。 7秒清理数据库！请记住，我们做了1200次。在分析了一个构建输出（构建花费了45分钟）之后，我们计算出清理数据库所花费的总时间-1800秒。构建的2/3花费在此上。肯定不会花那么长时间。

Of course we already checked the infrastructure. The build agents are pretty decent t2.large EC2 instances with 2 vCPUs and 8GiB RAM powering Ubuntu. Should be OK.

我喜欢复杂问题的快速简便的解决方案。我们使用简单的工具（shell和excel）分析了构建输出。让我们逐步进行：

extract the timestamps when cleanup DB starts:

cat buildoutput.log | grep "DefaultPrepAndExpectedTestCase - cleanupData" | sed 's/\([0123456789T.:-]*\).*DefaultPrepAndExpectedTestCase.*/\1/g' > cleanup_start.out

(grep for the lines matching the substring, then using sed to only output the timestamp)

extract the timestamps of the next messages - this will roughly be the end time of the cleanups:
```
cat buildoutput.log | grep "DefaultPrepAndExpectedTestCase - cleanupData" -A 1 | grep -v "\-\-" | grep -v "DefaultPrepAndExpectedTestCase" | sed 's/\([0123456789T.:-]*\).*/\1/g' > cleanup_end.out
```
(grep for the lines matching the substring and 1 line after, grep out the first line and the line with '--', then using sed to only output the timestamp)
Load both files to Excel
- choose Data -> From Text/CSV (Alt+A, FT)
- select the first file cleanup_start.out
- Source and Change type steps should be added automatically, add a few more:
- the new Added column contains Time.Hour([Column1])*60*60+Time.Minute([Column1])*60+Time.Second([Column1])
- the Removed columns step just deletes the Column1
- now the excel contains the amount of seconds (incl. fraction) elapsed from the day start
- repeat the same for second file cleanup_end.out
- add diff column
Now the diff column contains roughly the duration of the DB cleanup. I simply put a sum at the end of the column to calculate the 1800 seconds mentioned above.
plot a chart
- select the whole third column
- add a new line chart
The chart now shows how log it took the DB cleanup during the build. You can see important information here. It was all fine and then we see sudden blocks of peaks. The pattern looks very suspicious.

Short stare at the chart, some experience and the 'blink' moment - the Garbage Collector!!! This explains why it takes 7 seconds to cleanup the DB. Because the full GC runs in the background.

Improving the situation

我们检查了受影响模块的POM，然后猜测surefire插件配置：

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>${surefire.version}</version>
        <configuration>
            <argLine>${surefireArgLine} -Dfile.encoding=UTF-8 -Xmx1024m
            </argLine>
        </configuration>
</plugin>

是的-Xmx1024m Java最大堆空间明确设置为1024m。这对于再测试来说是不够的。即使我们在构建管道中设置了Xmx，它也不会被拾取。这说明了一切-当堆空间达到最大时，将调用完整的GC释放一些内存。这从根本上减慢了测试的执行速度。

我们要做的第一件事-将Xmx增加到至少2048m（或将其留空以默认为RAM的1/4，以较小者为准）。

既然我们在这里，让我再告诉您一件事。如果您的测试足够好，可以单独正常运行，并且您的硬件足够强大，请使用surefire的以下配置：

<configuration>
    <forkCount>2</forkCount>
    <parallel>classes</parallel>
    <threadCountClasses>2</threadCountClasses>
</configuration>

此设置意味着将有2个surefire引导程序分支（此人实际上是在隔离的JVM中执行测试），它将使用2个线程来运行测试，并将测试套件分解为多个类以将其分发。这里要记住的重要一点是threadCountClasses在同一（分叉）JVM中使用，意味着每个fork将有2个线程。因此，如果您遇到比赛条件，只需降低threadCountClasses到1。测试仍将并行运行，但每个前叉有1个线程。而且由于每个JVM H2通常为1，所以应该没问题。除非您在JVM之外有另一种共享数据。

我们达到的结果-直到11分钟生成时间从25分钟开始，仅是之前时间的44％。

Conclusion

了解您使用的工具（例如MockMvc，DBUnit，Spring Test DBUnit，Surefire插件）。分析您的构建输出。不要仅仅说明您的应用程序在增长，测试套件在增长，因此构建时间会更长。当然，这是正确的，但是您应该始终保留一些时间来重构和改进代码。

Appendix

DBUnit H2 `TRUNCATE_TABLE` operation

Truncate table is usually faster operation than delete from. There are some catches though. Read more about it here. H2 won't allow you to truncate table if there are foreign keys to that table. However, there is a special H2 syntax that you can make use of. We implemented the following DBUnit operation:

class H2TruncateOperation extends org.dbunit.operation.AbstractOperation
{
    private final Logger logger = LoggerFactory.getLogger(H2TruncateOperation.class);

    @Override
    public void execute(IDatabaseConnection connection, IDataSet dataSet)
            throws DatabaseUnitException, SQLException
    {
        logger.debug("execute(connection={}, dataSet={}) - start", connection, dataSet);
        IDataSet databaseDataSet = connection.createDataSet();
        DatabaseConfig databaseConfig = connection.getConfig();
        IStatementFactory statementFactory = (IStatementFactory) databaseConfig
                .getProperty("http://www.dbunit.org/properties/statementFactory");
        IBatchStatement statement = statementFactory.createBatchStatement(connection);

        try
        {
            int count = 0;
            Stack<String> tableNames = new Stack<>();
            Set<String> tablesSeen = new HashSet<>();
            ITableIterator iterator = dataSet.iterator();

            String tableName;
            while(iterator.next())
            {
                tableName = iterator.getTableMetaData().getTableName();
                if(!tablesSeen.contains(tableName))
                {
                    tableNames.push(tableName);
                    tablesSeen.add(tableName);
                }
            }

            if(!tableNames.isEmpty())
            {
                statement.addBatch("SET FOREIGN_KEY_CHECKS=0");
            }

            for(; !tableNames.isEmpty(); ++count)
            {
                tableName = tableNames.pop();
                ITableMetaData databaseMetaData = databaseDataSet.getTableMetaData(tableName);
                tableName = databaseMetaData.getTableName();
                String sql = "TRUNCATE TABLE " +
                             this.getQualifiedName(connection.getSchema(), tableName,
                                                   connection) +
                             " RESTART IDENTITY";
                statement.addBatch(sql);
                if(logger.isDebugEnabled())
                {
                    logger.debug("Added SQL: {}", sql);
                }
            }

            if(count > 0)
            {
                statement.addBatch("SET FOREIGN_KEY_CHECKS=1");
                statement.executeBatch();
                statement.clearBatch();
            }
        }
        finally
        {
            statement.close();
        }
    }
}

SQL语句示例：

SET FOREIGN_KEY_CHECKS=0;
TRUNCATE TABLE table1 RESTART IDENTITY;
TRUNCATE TABLE table2 RESTART IDENTITY;
SET FOREIGN_KEY_CHECKS=1;

您可以使用将此操作传递给普通的数据库单元配置，如下所示：

AbstractDatabaseTester databaseTester = new DataSourceDatabaseTester(dataSource);
databaseTester.setTearDownOperation(new H2TruncateOperation());

或实施新的数据库操作查询（如果您使用的是Spring Test DBUnit）：

public class H2SpecificDatabaseOperationLookup extends DefaultDatabaseOperationLookup
{
    @Override
    public org.dbunit.operation.DatabaseOperation get(DatabaseOperation operation)
    {
        return operation == DatabaseOperation.TRUNCATE_TABLE ?
               new H2TruncateOperation() :
               super.get(operation);
    }
}

并使用注释：

@DbUnitConfiguration(databaseOperationLookup = H2SpecificDatabaseOperationLookup.class)

from: https://dev.to//vladonemo/reducing-the-maven-build-execution-time-1o7k