从.NET Framework到.NET Core：C＃构建场的历史

As we mentioned in the previous posts of this series on .NET Core, Criteo started as a full .NET shop.

正如我们 在.NET Core系列 文章 的 前几篇文章 中 提到的 那样 ，Criteo最初是一个完整的.NET商店。

To understand the challenges we faced when starting on .NET Core, we will go back to the 2010s and take a look at the .NET build farm we created to manage our codebase. When MSBuild 15 and Visual Studio 2017 came out, this build farm, that worked reasonably well until then, became our main challenge in successfully transitioning to .NET Core.

为了理解在.NET Core上启动时所面临的挑战，我们将回到2010年代，来看看为管理代码库而创建的.NET构建场。 当MSBuild 15和Visual Studio 2017发布时，在此之前运行良好的此构建服务器场成为成功过渡到.NET Core的主要挑战。

This article will attempt to paint a brief history of build at Criteo, with the evolutions of MSBuild that went with it.

本文将尝试介绍Criteo构建的简要历史，以及随之而来的MSBuild的演变。

我们的构建流程 (Our build pipeline)

卑微的开始 (Humble beginnings)

In 2005, our codebase started as any other codebase: MSBuild .NET Framework projects in a single repository. What works with a few projects becomes unwieldy as the size and the number of projects grow:

在2005年，我们的代码库开始于其他任何代码库：在单个存储库中的MSBuild .NET Framework项目。 随着项目规模和数量的增长，一些项目的工作变得笨拙：

• In 2011, development branches were integrated weekly and took 90 minutes to build.
  2011年，开发部门每周进行整合，耗时90分钟。
• In June 2012, the same process took 6 workdays: A weekly integration was no longer possible.
  在2012年6月，相同的过程耗时6个工作日：不再可能进行每周集成。

That’s when we realized that something needed to be done: The codebase was separated in multiple repositories built independently from the others and NuGets were used to publish artifacts.

到那时，我们意识到需要做些事情：将代码库分隔在独立于其他资源而建立的多个存储库中，并使用NuGets来发布工件。

But it was not long before it led to another kind of nightmare: Dependency hell. How to ensure that the versions of an assembly (i.e. the binary result of a built project) were consistent through all the repositories it was being used in? And for all assemblies!

但是不久就引发了另一场噩梦： 依赖地狱 。 如何确保程序集的版本(即，生成项目的二进制结果)在使用该程序集的所有存储库中保持一致？ 对于所有组件！

During 2013, the issue was so huge that we reached the point where no new feature was released for 3 months.

在2013年，问题非常严重，以至于我们达到了三个月没有发布任何新功能的地步。

从源构建 (Build from source)

At that time, an internal project called “red pill” was started to improve our release process. On the build side, it introduced a new principle for C# development: All internal code should always be built from source in a single pipeline and with the same version. Only one version (and by the way, no more versioning was used) of an assembly was used in all projects.

当时，一个名为“红色药丸”的内部项目开始了，以改善我们的释放过程。 在构建方面，它引入了C＃开发的新原理：所有内部代码应始终从源代码在单个管道中以相同的版本构建 。 在所有项目中仅使用一个版本的程序集(顺便说一句，不再使用版本控制)。

But to implement that principle at the scale of Criteo required some tooling on top of MSBuild. Enter a new command-line tool: CBS or Criteo Build System.

但是要在Criteo的规模上实现该原则，需要在MSBuild之上使用一些工具。 输入一个新的命令行工具： CBS或Criteo Build System 。

The main goal of this CLI tool is to automate cloning and building of all repositories as a single entity inside a folder called a workspace.

此CLI工具的主要目标是将所有存储库的克隆和构建作为一个单独的实体(称为工作区)自动进行构建。

But this tool also allows a developer to create workspaces with only a subset of repositories checked out. CBS in that case uses artifacts produced by the CI pipeline to avoid building the missing dependencies locally. We call this mode of operation “partial checkout”.

但是，该工具还允许开发人员创建仅签出一部分存储库的工作空间。 在这种情况下，CBS使用CI管道生成的构件来避免在本地构建丢失的依赖项。 我们将此操作模式称为“ 部分结帐 ”。

如何执行部分结帐？ (How is partial checkout implemented?)

MSBuild implements references between assemblies in a way that needs a bit of explanation before going any further.

MSBuild以某种方式在程序集之间实现引用，然后再进行进一步解释。

It all begins simply enough: MSBuild introduces the notion of “Items” that describe the inputs of the build system. The .NET SDK uses items of type “Reference” to track the references between a project and its dependent .NET assemblies.

一切都非常简单地开始：MSBuild引入了“项目”的概念，这些概念描述了构建系统的输入。 .NET SDK使用“引用”类型的项目来跟踪项目及其从属.NET程序集之间的引用。

It also uses items of type “ProjectReference” to describe references between projects. A very crude way to describe a ProjectReference is that it will be transformed into a Reference to the assembly built by another project.

它还使用“ ProjectReference”类型的项目来描述项目之间的引用。 描述ProjectReference的一种非常粗糙的方法是将其转换为对另一个项目构建的程序集的引用。

This is where CBS implements partial checkout: References between projects are declared as simple references. It can then adjust where to look for a dependency assembly whether the repository for that particular assembly is available locally for build or not.

这是CBS实现部分检出的地方：项目之间的引用被声明为简单引用。 然后，它可以调整在哪里寻找依赖程序集，无论该特定程序集的存储库在本地是否可用于构建。

In a given workspace, CBS knows which project sources are checked out and so it can handle the build order itself when running from the command line. CBS can also create a Visual Studio Solution with proper build dependencies for the developers to use in their IDE.

在给定的工作空间中，CBS知道检出了哪些项目源，因此当从命令行运行时，它可以自行处理构建顺序。 CBS还可以创建具有适当构建依赖关系的Visual Studio解决方案，以供开发人员在其IDE中使用。

外部依赖性如何？ (How about external dependencies?)

Before Visual Studio 2017 and MSBuild 15, external dependencies were handled in .NET with a separate tool called NuGet. Its role is to keep a list of external packages required by a project in a file called “packages.config”. When a package is added, the tool will also patch the content of the project to add the necessary Reference declarations, including the location where the assembly was downloaded by NuGet.

在Visual Studio 2017和MSBuild 15之前，.NET中的外部依赖项是通过名为NuGet的单独工具处理的。 它的作用是将项目所需的外部软件包列表保存在名为“ packages.config”的文件中。 添加程序包后，该工具还将修补项目内容，以添加必要的Reference声明，包括NuGet下载程序集的位置。

CBS detects the references created by the NuGet CLI and automatically installs the associated NuGet packages in the workspace.

CBS检测到由NuGet CLI创建的引用，并在工作区中自动安装关联的NuGet软件包。

传递依赖关系如何？ (How about transitive dependencies?)

We explained previously that we replaced ProjectReference by Reference to be able to implement partial checkout. But this is the root cause of two problems.

前面我们曾解释说，我们将ReferenceReference替换为ProjectReference，以便能够实现部分签出。 但这是两个问题的根本原因。

First, MSBuild no longer adds transitive dependencies to the C# compiler command line. This means that a given project must add references to all its dependencies required for compilation.

首先，MSBuild不再将可传递依赖项添加到C＃编译器命令行中。 这意味着给定的项目必须添加对编译所需的所有依赖项的引用。

Second, when MSBuild constructs the list of assemblies to collect in the output directory for runtime, it no longer has a global vision of the different projects involved in a build. It only knows about the produced assemblies for the current project.

其次，当MSBuild构造要在输出目录中收集以供运行时使用的程序集列表时，它不再对构建中涉及的不同项目有全局看法。 它只知道当前项目的生产装配。

For example, let’s consider Project A, which has a reference to Project B, itself having a dependency on NuGet C. Since A has no knowledge of C, how could it know where to look for it?

例如，让我们考虑项目A，它引用了项目B，而项目B本身又依赖于NuGetC。既然A不了解C，那么它怎么知道在哪里寻找呢？

ResolveAssemblyReference任务 (ResolveAssemblyReference task)

Enters “one of the most important tasks in the MSBuild toolset”: ResolveAssemblyReference (RAR). It is also the source of many headaches in the .NET community because of its complex job.

输入“ MSBuild工具集中最重要的任务之一 ”：ResolveAssemblyReference(RAR)。 由于其复杂的工作，它也是.NET社区中许多头痛的根源。

Its goal is to analyze all known references of a project and match them to file locations on disk. It will do so recursively by analyzing the metadata of each assembly to discover its own set of references.

其目标是分析项目的所有已知引用，并将它们与磁盘上的文件位置进行匹配。 它将通过分析每个程序集的元数据以发现其自己的引用集来进行递归操作。

CBS uses this task to give MSBuild sufficient information about the dependencies of projects from a repository missing on the local workspace. CBS gets this information from an “AssemblySet”: Built in the CI pipeline, it is a complete graph of all assemblies, references, and NuGets for all repositories.

CBS使用此任务为MSBuild提供有关本地工作区中缺少的存储库中项目依赖项的足够信息。 CBS从“ AssemblySet”中获取此信息：内置在CI管道中，它是所有存储库的所有程序集，引用和NuGet的完整图形。

When building on the developer’s machine, this AssemblySet information will be used to give the RAR task extra disk locations to look for missing assemblies.

在开发人员的计算机上构建时，此AssemblySet信息将用于为RAR任务提供额外的磁盘位置，以查找丢失的程序集。

2014胜利 (2014 victory)

After switching to the new system, our .NET build was back on track. A second version implementing a build cache of assemblies allowed us to further improve the build time. That cache uses git repository commit hashes to determine if a particular assembly needs to be rebuilt or simply downloaded.

切换到新系统后，我们的.NET构建已恢复正常。 第二个版本实现了程序集的构建缓存，这使我们可以进一步缩短构建时间。 该缓存使用git存储库提交哈希值来确定是否需要重建或简单下载特定程序集。

Evolution of our CI performance

CI绩效的演变

The previous diagram summarizes the performance of our CI. Splitting our repository to 130 repositories/pipelines allowed us to temporarily keep up on releasing every week in 2012. In 2014, the introduction of “build from source” reduced the integration time to 40 minutes. Finally, the introduction of the caching mechanism allowed us to keep the performance while doubling the number of repositories.

上图总结了CI的性能。 将我们的存储库划分为130个存储库/管道，使我们能够在2012年每周暂时停止发布。在2014年，“从源构建”的引入将集成时间缩短至40分钟。 最后，缓存机制的引入使我们能够在保持性能的同时将存储库数量加倍。

All was well until MSBuild 15 came along…

在MSBuild 15出现之前，一切都很好。

.NET Core在Criteo的开始 (The beginning of .NET Core at Criteo)

dotnet CLI和project.json (dotnet CLI and project.json)

When we started working on .NET Core in 2017, the Microsoft SDK was using a new build system based on JSON files. To be able to run our first tests on .NET Core without redesigning completely our tooling, we started simple: We created a separate pipeline that would clone all sources and simply call the dotnet CLI to build what we had already ported.

当我们在2017年开始使用.NET Core时，Microsoft SDK使用的是基于JSON文件的新构建系统。 为了能够在.NET Core上运行我们的第一个测试而无需完全重新设计我们的工具，我们从简单开始：我们创建了一个单独的管道，该管道将克隆所有源，并只需调用dotnet CLI来构建我们已经移植的内容。

Because there were very few projects ported to .NET Core at the beginning, this proved sufficient to begin our first experiments.

由于开始时很少有项目移植到.NET Core，因此这足以开始我们的第一个实验。

更新我们的外部依赖 (Updating our external dependencies)

The second step was to start working on our external dependencies to make sure that they were compatible with .NET Core. We already had a tool to check how far behind we were on the latest available NuGets so we simply improved it to include information about .NET Standard compatibility:

第二步是开始研究我们的外部依赖关系，以确保它们与.NET Core兼容。 我们已经有了一个工具来检查与最新可用的NuGet的差距，因此我们只是对其进行了改进，以包含有关.NET Standard兼容性的信息：

Visual Studio 2017 (Visual Studio 2017)

With the release of Visual Studio 2017 and MSBuild 15, .NET Core officially deprecated project.json to go back to MSBuild as the build system for the .NET Core SDK. This helped us with the migration, but the new .NET SDK projects proved challenging to integrate in our custom design for CBS.

随着Visual Studio 2017和MSBuild 15的发布，.NET Core正式弃用project.json，作为.NET Core SDK的生成系统返回到MSBuild。 这帮助我们进行了迁移，但是事实证明，新的.NET SDK项目很难集成到我们的CBS定制设计中。

MSBuild中的更改 (The changes in MSBuild)

To support .NET Core and the principles it introduced, MSBuild was modified quite significantly. A new notion of SDK was introduced to allow automatic import of tasks and targets into a project. The primary SDK introduced, called “Microsoft.Net.Sdk”, was designed to build assemblies targeting .NET Standard, .NET Core, and .NET Framework.

为了支持.NET Core及其引入的原理，对MSBuild进行了相当大的修改。 引入了新的SDK概念，以允许将任务和目标自动导入到项目中。 引入的主要SDK(称为“ Microsoft.Net.Sdk”)旨在构建针对.NET Standard，.NET Core和.NET Framework的程序集。

But projects that used the new SDK have a structure quite different from the old ones:

但是使用新SDK的项目的结构与旧SDK的结构完全不同：

• They rely a lot more on convention rather than configuration: the list of source files is gone, almost all properties have sane default values, some items metadata are simplified.
  他们更多地依赖约定而不是配置：源文件列表不见了，几乎所有属性都具有合理的默认值，简化了某些项目元数据。
• A new type of item is introduced: PackageReference. It brings tight integration to NuGet inside MSBuild. It also means that a new target is introduced: “Restore”. Its job is to solve the package dependency graph and download the needed NuGet packages.
  引入了一种新的项目类型：PackageReference。 它与MSBuild内部的NuGet紧密集成。 这也意味着引入了一个新的目标：“还原”。 它的工作是解决程序包依赖关系图并下载所需的NuGet程序包。

These changes required modifying CBS in order to support them. As CBS was directly parsing .csproj files as raw XML, the first thing that we had to do was to improve the parser to support both formats. We also had to include the new Restore target in the sequence of targets that CBS would call during the build.

这些更改需要修改CBS以支持它们。 由于CBS直接将.csproj文件解析为原始XML，所以我们要做的第一件事就是改进解析器以支持两种格式。 我们还必须在构建过程中CBS调用的目标序列中包括新的Restore目标。

At the same time, we also started to work on a tool that could automatically convert old projects into new ones.

同时，我们也开始研究可以自动将旧项目转换为新项目的工具。

.NET标准 (.NET Standard)

.NET Standard is the definition of a set of APIs that are common to the implementations of both .NET Framework and .NET Core.

.NET标准是.NET Framework和.NET Core的实现所共有的一组API的定义。

As there is no true .NET Standard library (the implementation of these APIs is different in both frameworks), a bit of magic happens. When a project targets .NET Standard, MSBuild needs to provide assemblies to the C# compiler so that it can properly compile the code and only use the shared API.

由于没有真正的.NET标准库(两个框架中这些API的实现都不同)，因此发生了一些不可思议的事情。 当一个项目面向.NET Standard时，MSBuild需要为C＃编译器提供程序集，以便它可以正确地编译代码并且仅使用共享的API。

To do so, the SDK contains a set of assemblies (called references assemblies) used for compilation only. They are simply classes and methods with empty implementation.

为此，SDK包含一组仅用于编译的程序集(称为引用程序集 )。 它们只是具有空实现的类和方法。

At runtime, when the final framework is known, these facade assemblies are replaced by implementations that contain type forwarding declarations, so that the actual type used is the one from the proper framework.

在运行时，当最终框架已知时，这些外观组合将替换为包含类型转发声明的实现，以便所使用的实际类型是正确框架中的类型。

The SDK that shipped with MSBuild and .NET Core contains extra code to deal with this specific issue. But it effectively broke our approach in handling transitive dependencies: as most of.NET Standard dependencies are shipped as NuGet packages, they are treated as such by our tooling that has no notion of these new types of assemblies and how to use them to create an appropriate search path for the RAR task. We ended up with DLLs incorrectly selected for the final target framework: for example, reference assemblies sometimes appeared in the final runtime folder, instead of their true implementations.

MSBuild和.NET Core附带的SDK包含额外的代码来解决此特定问题。 但这有效地打破了我们处理传递性依赖关系的方法：由于大多数.NET Standard依赖关系都以NuGet软件包的形式提供，因此我们的工具将它们视作此类，这些工具不了解这些新类型的程序集以及如何使用它们来创建组件。 RAR任务的适当搜索路径。 我们最终为最终目标框架选择了错误的DLL：例如，参考程序集有时出现在最终的运行时文件夹中，而不是它们的真实实现。

双重建造 (Double build)

Our first impression was that we could use .NET Standard to progressively convert all our projects to the new SDK and .NET Standard. But due to the implementation details of both our tooling and the magic detailed above, it would not have been possible without a major redesign of our CI system, which we didn’t have time for.

我们的第一印象是我们可以使用.NET Standard将所有项目逐步转换为新的SDK和.NET Standard。 但是，由于我们的工具和上述魔术的实现细节，如果不对CI系统进行重大重新设计(我们没有时间进行)，就不可能实现。

The solution we ended up with was to create extra MSBuild projects, next to our existing .NET Framework projects, with a “.netcore” suffix.

我们最终得到的解决方案是在现有的.NET Framework项目旁边创建额外的MSBuild项目，并带有“ .netcore”后缀。

These projects simply reference the sources from the other folder but are completely separate .NET Standard/Core projects, that produce an assembly with a different name.

这些项目仅引用其他文件夹中的源代码，但是它们是完全独立的.NET Standard / Core项目，它们生成具有不同名称的程序集。

This allowed us to get developers started and make sure that their code was compatible with .NET Standard. By the way, this is still how we run our build today.

这使我们可以使开发人员入门，并确保他们的代码与.NET Standard兼容。 顺便说一句，这仍然是我们今天运行构建的方式。

我们正在做什么 (What we are working on)

未决问题 (Pending issues)

Though we have successfully created a build farm that can support all our .NET needs, it still has a few limitations.

尽管我们已经成功创建了一个可以满足我们所有.NET需求的构建场，但它仍然存在一些局限性。

The first obvious problem with our current approach is the duplication of projects. Having 2 projects means 2 builds, twice the build time, and poor user experience in the Visual Studio IDE. Each new dependency must be added to 2 projects. This will probably disappear as we move away from .NET Framework, but it may take a long time to get there.

当前方法的第一个明显问题是项目重复。 具有2个项目意味着2个构建，是构建时间的两倍，并且在Visual Studio IDE中的用户体验较差。 每个新的依赖项必须添加到2个项目中。 当我们离开.NET Framework时，这可能会消失，但是可能要花很长时间才能到达那里。

The second issue is related to the way we handle dependencies. CBS has gone too far from the MSBuild system and requires us to duplicate parts of MSBuild logic in our tooling. The main issue we are facing is that we incorrectly propagate dependencies to client projects. The developers have to add extra dependencies that are not strictly needed at their level but required by some other NuGet/assembly they are depending on. It also means that, with each new release of MSBuild, significant work needs to be done to issue a new version of CBS.

第二个问题与我们处理依赖关系的方式有关。 CBS离MSBuild系统太远了，它要求我们在工具中复制MSBuild逻辑的各个部分。 我们面临的主要问题是我们错误地将依赖项传播到客户端项目。 开发人员必须添加额外的依赖关系，这些依赖关系在其级别上并不是严格需要的，但它们依赖的其他一些NuGet /程序集则需要。 这也意味着，对于每个新版本的MSBuild，都需要进行大量工作才能发布新版本的CBS。

The last problem we have to struggle with is consistency: Because we provide MSBuild with only a partial view of the dependency graph, all the safeties around package version conflicts and incompatibilities are effectively not enforced and would be too painful to reimplement and maintain in our tooling. These problems usually show when an incorrect version of a particular assembly is selected by the RAR task, but not properly selected at runtime.

我们必须努力解决的最后一个问题是一致性：因为我们仅向MSBuild提供依赖关系图的部分视图，所以围绕软件包版本冲突和不兼容性的所有安全性都无法得到有效实施，并且在重新实现和维护工具时将非常痛苦。 这些问题通常显示在RAR任务选择了特定程序集的不正确版本时，但在运行时未正确选择时。

我们的下一步 (Our next move)

We have created a new version of our build system, completely MSBuild-based. It relies on the new .NET SDK type of project, and implements partial checkout in the following way:

我们已经创建了完全基于MSBuild的构建系统的新版本。 它依赖于新的.NET SDK类型的项目，并通过以下方式实现部分签出：

• References to internal projects that are present locally are replaced by ProjectReferences early in the build.
  在构建早期，对本地存在的内部项目的引用将被ProjectReferences替换。
• References to missing internal projects are replaced with ProjectReferences to “placeholder” projects. Generated from the AssemblySet, they contain only Package and Project references, and point to the downloaded artifact from the CI. The long-term goal is to replace these placeholder projects with real NuGets and PackageReferences.
  对缺少的内部项目的引用将替换为对“占位符”项目的引用。 从AssemblySet生成，它们仅包含Package和Project引用，并指向从CI下载的工件。 长期目标是用真正的NuGets和PackageReferences替换这些占位符项目。

However, during the experiments we ran on our current projects, we realized that our dependency graph was full of inconsistencies that prevented us from using it without fixing them first.

但是，在我们进行当前项目的实验过程中，我们意识到依赖图充满了不一致之处，导致我们无法在不先修复它们的情况下使用它。

Because of the size of our codebase (250 repositories and 3000 projects), this is still an ongoing effort. Our tooling may also need more work to keep our build performance at the same level it has today.

由于我们的代码库很大(250个存储库和3000个项目)，所以这仍是一项持续的工作。 我们的工具可能还需要做更多的工作才能使我们的构建性能保持在今天的水平。

结论 (Conclusion)

This article was a very brief overview of the build system for .NET at Criteo. It allows us to implement our company-wide “build from source” strategy while keeping decent CI times given the size of our codebase. The design choices made at the origin proved challenging to adapt to new MSBuild versions.

本文是Criteo的.NET构建系统的非常简短的概述。 它使我们能够实施我们公司范围内的“从源构建”策略，同时在给定代码库大小的情况下，保持良好的CI时间。 最初的设计选择被证明很难适应新的MSBuild版本。

As a result, our transition to .NET Core and MSBuild 15 Is still ongoing but allowed us to successfully build and release our first products to a Linux production platform.

因此，我们仍在继续向.NET Core和MSBuild 15过渡，但使我们能够成功构建第一个产品并将其发布到Linux生产平台。

With the latest additions to MSBuild, we hope to have a design much closer to the .NET SDK, so that later versions of MSBuild will be hopefully much smoother to integrate.

随着MSBuild的最新添加，我们希望设计与.NET SDK更加接近，从而希望更高版本的MSBuild可以更平滑地集成。

Want to read more about the topic? Check out the series:

想更多地了解这个话题？ 查看系列：

Join the journey!

加入旅程！

翻译自: https://medium.com/criteo-labs/from-net-framework-to-net-core-history-of-a-c-build-farm-6295145a437b