buildbot_如何使用Buildbot和Poudriere为FreeBSD服务器构建和部署软件包

最新推荐文章于 2024-08-23 08:59:00 发布

cukw6666

最新推荐文章于 2024-08-23 08:59:00 发布

阅读量877

点赞数

文章标签： python linux java 运维 entity

原文链接：https://www.digitalocean.com/community/tutorials/how-to-build-and-deploy-packages-for-your-freebsd-servers-using-buildbot-and-poudriere

版权

本教程介绍了如何在FreeBSD服务器上使用Poudriere和Buildbot自动化构建和部署软件包。首先，你需要在FreeBSD 11.2服务器上安装Poudriere。然后，配置Poudriere进行软件包构建，设置HTTP/HTTPS包仓库，使用Buildbot作为持续集成平台。教程还涵盖了创建包签名密钥、设置ZFS、配置Nginx以托管包仓库以及设置Buildbot构建器。最后，配置客户端以安全地从中央包仓库获取软件包。

摘要由CSDN通过智能技术生成

buildbot

介绍 (Introduction)

The FreeBSD ports and packages collection, hereafter called ports tree, is FreeBSD’s build system for external software. It offers a Makefile-based, consistent way of building packages. The port refers to the build recipe, that is the Makefile and related files; while package is the output of building one port into a binary (compressed) archive of the package files and its meta information.

FreeBSD端口和软件包集合 (以下称为端口树 )是FreeBSD的外部软件构建系统。它提供了基于Makefile的一致方式来构建软件包。该端口指的是构建配方，即Makefile和相关文件。而package是将一个端口构建为包文件及其元信息的二进制(压缩)归档的输出。

Manually building and installing a subset or all of the over 30,000 ports is possible with make install. However, the builds would run on one of your servers—not a clean environment. For production use cases, manual builds would also mean that each host needs the same revision of the ports tree, and needs to compile all packages for itself. This means repeated, error-prone work by humans and the servers. It is preferable to retrieve and use identical, pre-built binary packages on each host and serve them from a central, secure package repository.

使用make install可以手动构建和安装30,000个端口的子集或全部。但是，构建将在您的一台服务器上运行-而不是一个干净的环境。对于生产用例，手动构建还意味着每个主机都需要对ports树进行相同的修订，并且需要自己编译所有软件包。这意味着人员和服务器会重复进行，容易出错的工作。最好在每个主机上检索和使用相同的，预先构建的二进制程序包，然后从中央安全的程序包存储库中为它们提供服务。

To achieve this, Poudriere is the standard tool on FreeBSD to build, test, and audit packages as well as maintain the package repositories. Each build is run isolated in a fresh jail, running the desired version of FreeBSD, and starting with no packages installed. Only the base system, plus any explicitly specified dependencies, are available to the clean build. Poudriere takes care of rebuilding packages when necessary as well as updating the package repository after a build has finished. The poudriere command line tool is central to administering different ports trees, FreeBSD versions, port build options, and lastly, running the builds.

为此， Poudriere是FreeBSD上用于构建，测试和审核软件包以及维护软件包存储库的标准工具。每个构建都在新的监狱中隔离运行，运行所需版本的FreeBSD，并且不安装任何软件包。干净的构建仅可使用基本系统以及任何明确指定的依赖项。 Poudriere在必要时负责重建软件包，并在构建完成后更新软件包存储库。 poudriere命令行工具是管理不同端口树，FreeBSD版本，端口构建选项以及最后运行构建的核心。

In this tutorial you’ll configure Poudriere, build a set of desired packages, set up HTTP-based package hosting, and automate the build using Buildbot as a continuous integration platform. Finally, you will securely access the packages from a client machine.

在本教程中，您将配置Poudriere，构建一组所需的软件包，设置基于HTTP的软件包托管，并使用Buildbot作为持续集成平台来自动进行构建。最后，您将从客户端计算机安全地访问软件包。

Note: To cover production-like use cases, the tutorial examples use the quarterly stable branches of the ports tree. Staying on one such branch protects you from breaking changes and provides security and build fixes where necessary—if you regularly update the tree from upstream (Subversion, or its GitHub mirror). You can choose to stay on one branch for an extended period of time, depending on the pace at which your system updates can be handled by developer/infrastructure teams. The ports collection supports FreeBSD releases until they become end-of-life (EOL)—see Supported FreeBSD releases—so that OS and package updates can be handled independently. Alternatively, you could consider a local version control repository cloned from the upstream tree. That way, you can manage patches and only merge upstream changes at the time you desire.

注意：为了涵盖类似生产的用例，本教程示例使用ports树的每季度稳定分支。如果您定期从上游(Subversion或其GitHub镜像)更新树，则留在这样一个分支上可以保护您免受更改的破坏，并在需要时提供安全性和构建修补程序。您可以选择在一个分支上停留更长的时间，具体取决于开发人员/基础架构团队处理系统更新的速度。 ports集合支持FreeBSD发行版，直到它们生命周期终止(EOL)(请参阅支持的FreeBSD发行版) ，以便可以独立处理OS和软件包更新。或者，您可以考虑从上游树克隆的本地版本控制存储库。这样，您可以管理补丁程序，并且仅在需要时合并上游更改。

先决条件 (Prerequisites)

Before you begin this guide, you will need:

在开始本指南之前，您需要：

A server running FreeBSD 11.2. If you’re new to working with FreeBSD, you may find it helpful to customize this server by following our guide on How to Get Started with FreeBSD. Note: FreeBSD 12.0 currently has an issue with nested jails, which first needs to be fixed before 12.x can be used for this tutorial.
运行FreeBSD 11.2的服务器。如果您是不熟悉FreeBSD的新手，则可以通过遵循我们的FreeBSD入门指南来定制该服务器。 注意： FreeBSD 12.0当前存在嵌套jails的问题，首先需要解决此问题，然后才能将12.x用于本教程。
10 GB free disk space or more to have enough capacity to store packages and logs.
10 GB或更多的可用磁盘空间，以具有足够的容量来存储软件包和日志。
A basic Buildbot setup by completing the How To Set Up Buildbot on FreeBSD tutorial.
通过完成“ 如何在FreeBSD上设置Buildbot”教程来完成Buildbot的基本设置。
Another server running FreeBSD, the same version, which you are going to use as a client to fetch and install the packages that you’re going to automatically build and host in a HTTP/HTTPS-based package repository.
另一台运行FreeBSD的服务器(相同版本)，将用作客户端来获取并安装要在基于HTTP / HTTPS的软件包存储库中自动生成并托管的软件包。

第1步-安装Poudriere以在Buildbot Worker中使用 (Step 1 — Installing Poudriere for Use in Buildbot Worker)

After completing the prerequisite tutorial, you’ll have a working Buildbot master and worker jail plus Nginx setup. You will build upon this existing setup in the following steps. In this first step, you’re going to install the build tool Poudriere inside the worker jail, since that is where the Buildbot worker process will trigger builds later on.

完成前提条件教程后，您将拥有一个有效的Buildbot主和工作人员监狱以及Nginx设置。您将在以下步骤中基于此现有设置进行构建。在第一步中，您将在工作人员监狱中安装构建工具Poudriere，因为Buildbot worker进程稍后将在此触发构建。

Connect to your server hosting Buildbot and open a root shell in the worker jail with the following command:

连接到托管Buildbot的服务器，并使用以下命令在worker监狱中打开一个根 shell：

sudo jexec buildbot-worker0 csh
须藤jexec buildbot-worker0 csh

Install Poudriere as a package:

打包安装Poudriere：

pkg install poudriere
pkg install poudriere

Then confirm installation by pressing y and then ENTER.

然后按y ，然后按ENTER确认安装。

Note: It is preferable to use the official FreeBSD package repository for installing Buildbot, Poudriere, and so on. If you build those tool packages yourself, you start off in a chicken-and-egg situation: wanting to install external software, but requiring Poudriere installed to get cleanly built packages. Since Poudriere is a very stable and backward-compatible tool, nothing speaks against updating it regularly and independently from your production packages.

注意：最好使用官方的FreeBSD软件包系统信息库来安装Buildbot，Poudriere等。如果您自己构建这些工具包，那么您将遇到麻烦的情况：想要安装外部软件，但需要安装Poudriere才能获得干净构建的软件包。由于Poudriere是一种非常稳定且向后兼容的工具，因此没有什么可以反对定期且独立于生产软件包进行更新。

If you followed the prerequisite tutorial, this is already the case and you can continue without following this note.

如果您遵循了前提条件教程，那么情况已经如此，您可以继续操作而无需遵循本说明。

You’ve successfully installed the latest Poudriere tool and dependencies. In the next several steps, you will go through preparations to configure Poudriere.

您已经成功安装了最新的Poudriere工具和依赖项。在接下来的几个步骤中，您将准备配置Poudriere。

步骤2 —创建程序包签名密钥(可选) (Step 2 — Creating a Package Signing Key (Optional))

It’s recommended to set up digital signatures for built packages in order to provide more security. Skip this step if you want to secure your installation later, or in a different way. Otherwise, let’s go ahead and create a key pair used to sign packages (using the private key) and verify packages (using the public part).

建议为内置软件包设置数字签名，以提供更高的安全性。如果您想稍后或以其他方式保护安装安全，请跳过此步骤。否则，让我们继续创建用于对数据包进行签名(使用私钥)和验证数据包(使用公共部分)的密钥对。

Packages, by default, are built as .txz files, which are strongly compressed tarballs of the package contents. The compressed files’ checksums, together with serving the files via HTTP/HTTPS (TCP checksums), already provide some protection against corrupted data. Package contents typically comprise files and directories plus meta information such as the package name, version, and miscellaneous options. Files may even include setuid-able programs (as seen in the sudo package—though sudo is not built into FreeBSD), and the installation-time scripts run as root user. Installing from unverified sources therefore poses a security risk.

默认情况下，程序包以.txz文件的形式生成，这些文件是程序包内容的压缩包。压缩文件的校验和以及通过HTTP / HTTPS(TCP校验和)提供的文件，已经为防止损坏的数据提供了一些保护。软件包内容通常包括文件和目录以及元信息，例如软件包名称，版本和其他选项。文件甚至可能包含可设置setuid程序 (如sudo软件包所示-尽管sudo并未内置在FreeBSD中)，并且安装时脚本以root用户身份运行。因此，从未经验证的来源进行安装会带来安全风险。

By serving the packages over HTTPS, you cannot detect whether someone tampered with the packages on-disk. Integrity and authenticity of your packages can be added by configuring Poudriere to sign the package repository with an RSA private key. Signed digests and the corresponding public key are thereby stored in the package repository’s digests.txz file. The required key pair (RSA private and public key) can be kept unchanged for a long time unless the private key was lost or compromised.

通过HTTPS提供软件包，您将无法检测是否有人篡改了磁盘上的软件包。可以通过配置Poudriere用RSA私钥对软件包存储库进行签名来增加软件包的完整性和真实性。签名的摘要和相应的公钥由此存储在软件包存储库的digests.txz文件中。所需的密钥对(RSA私钥和公钥)可以长时间保持不变，除非私钥丢失或受到损害。

In this step you’ll create the key pair where the builds run (worker jail) and download the public part for later use on package clients (discussed in a later step).

在此步骤中，您将在运行构建的位置创建密钥对(工人监狱)，并下载公共部分以供以后在软件包客户端上使用(在后续步骤中讨论)。

Ensure you’re still in the worker jail root shell.

确保您仍在工作人员监狱根目录中 。

Create a new RSA private key:

创建一个新的RSA私钥：

openssl genrsa -out /usr/local/etc/poudriere.key 4096
openssl genrsa -out /usr/local/etc/poudriere.key 4096

The private key file only needs to be accessible by root—the user that runs Poudriere. Protect its access permissions:

私钥文件仅需要由root(运行Poudriere的用户)进行访问。保护其访问权限：

chmod 0600 /usr/local/etc/poudriere.key
chmod 0600 /usr/local/etc/poudriere.key

Later, you’ll need the public key part available on clients for verifying package signatures. Let’s extract the public key now:

以后，您将需要客户端上可用的公钥部分来验证软件包签名。现在让我们提取公钥：

openssl rsa -in /usr/local/etc/poudriere.key -pubout -out /tmp/poudriere.pub
openssl rsa -in /usr/local/etc/poudriere.key -pubout -out /tmp/poudriere.pub

Lastly, download the public key file from your own computer:

最后，从您自己的计算机下载公钥文件：

scp your-server:/usr/jails/buildbot-worker0/tmp/poudriere.pub /tmp/poudriere.pub
scp 您的服务器：/usr/jails/buildbot-worker0/tmp/poudriere.pub /tmp/poudriere.pub

This concludes the optional creation of a key pair for package signing. You will later configure the actual signing with Poudriere and use the downloaded public key file on clients for the verification.

这样就完成了可选的包签名密钥对的创建。稍后，您将使用Poudriere配置实际的签名，并使用客户端上下载的公共密钥文件进行验证。

Another optional step follows: if you use the ZFS filesystem, Poudriere can make use of it to speed up builds. Otherwise, you can skip to Step 4 to configure Poudriere in order to get ready for running the first build.

另一个可选步骤如下：如果您使用ZFS文件系统，Poudriere可以利用它来加快构建速度。否则，您可以跳到第4步来配置Poudriere，以准备好运行第一个构建。

步骤3 —设置ZFS(可选) (Step 3 — Setting Up ZFS (Optional))

This step only applies if you run a FreeBSD system on top of the ZFS filesystem. For instance if you’re using a DigitalOcean Droplet the image is labeled 11.2 x64 zfs (for FreeBSD 11.2). In this step, you’re going to create the filesystems that Poudriere can use to create and manage jails faster, potentially speeding up your builds.

仅当您在ZFS文件系统之上运行FreeBSD系统时，此步骤才适用。例如，如果您使用的是DigitalOcean Droplet，则图像标记为11.2 x64 zfs (对于FreeBSD 11.2 )。在此步骤中，您将创建Poudriere可以用来更快地创建和管理监狱的文件系统，从而有可能加快构建速度。

You can find out whether you’re using ZFS by listing pools. Make sure you’re on the server’s shell, not inside a jail.

您可以通过列出池来确定是否正在使用ZFS。确保您在服务器的外壳上，而不是在监狱里。

exit
出口

Run the following command to list the zpools:

运行以下命令以列出zpool：

sudo zpool list
sudo zpool列表

If any pool is available, it will print information about it:

如果有任何池可用，它将打印有关它的信息：


   
   
    
    Output
   
   NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot   148G  94.4G  54.1G        -         -    66%    63%  1.00x  ONLINE  -

Otherwise if ZFS support is not available the tool will print no pools available, or failed to initialize ZFS library. This means that none of your system is using ZFS; in this case, skip to the next step. If you have decided to use another disk or storage type, such as the UFS filesystem, you can also move on to the next step.

否则，如果没有ZFS支持，该工具将不打印no pools available ，或者failed to initialize ZFS library 。这意味着您的系统都没有使用ZFS。在这种情况下，请跳至下一步。如果您决定使用其他磁盘或存储类型，例如UFS文件系统，则也可以继续执行下一步。

If you plan to use ZFS, remember the printed pool name on which you want to store build-related data. You should plan for several gigabytes of storage.

如果计划使用ZFS，请记住要在其上存储与构建相关的数据的打印池名称。您应该计划几个GB的存储空间。

ZFS is helpful to separate the various datasets of Poudriere, like build jails, ports trees, logs, packages, and other data. These are stored independently and as a result can be deleted quickly with the certainty of not leaving free space or traces behind.

ZFS有助于分离Poudriere的各种数据集，例如构建监狱，端口树，日志，包和其他数据。它们是独立存储的，因此可以快速删除，并确保不会留下可用空间或痕迹。

For Poudriere to make use of ZFS, you need to do three things: create a parent ZFS dataset, allow for the creation and deletion of ZFS datasets (which the Buildbot worker jail, or any other jail, by default cannot do), and edit Poudriere’s configuration accordingly.

为了使Poudriere使用ZFS，您需要做三件事：创建父ZFS数据集，允许创建和删除ZFS数据集(Buildbot worker监狱或其他任何监狱，默认情况下不能这样做)，以及编辑Poudriere的配置相应。

In the prerequisite tutorial, you configured the Buildbot worker jail in /etc/jail.buildbot-worker0.conf. Open this file with your preferred text editor and add the following highlighted lines to delegate a parent dataset to allow the jail to administer ZFS datasets beneath the parent. Remember to replace zroot with your desired pool name:

在必备教程中，您在配置Buildbot工人监狱/etc/jail.buildbot-worker0.conf 。使用首选的文本编辑器打开此文件，并添加以下突出显示的行以委派父级数据集，以允许监狱管理父级下的ZFS数据集。请记住将zroot替换为所需的池名称：

sudo ee /etc/jail.buildbot-worker0.conf
须藤ee /etc/jail.buildbot-worker0.conf

/etc/jail.buildbot-worker0.conf

buildbot-worker0 {
    host.hostname = buildbot-worker0.localdomain;
    ip4.addr = "lo1|10.0.0.3/24";
    path = "/usr/jails/buildbot-worker0";
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    mount.devfs; # need /dev/*random for Python
    persist;

    exec.poststart = "/sbin/zfs jail buildbot-worker0 zroot/pdr/w0";
}

In this article we will store build-related data on the ZFS pool zroot—please adapt this ZFS-related configuration here and throughout the rest of the article if you chose a pool of a different name.

在本文中，我们将与构建相关的数据存储在ZFS池zroot —如果您选择了其他名称的池，请在此处以及在本文的其余部分中调整与ZFS相关的配置。

After adding this content, save and exit the editor. If you’re using ee, do this by pressing CTRL+C, typing exit, and pressing ENTER.

添加此内容后，保存并退出编辑器。如果使用的是ee ，请按CTRL+C ，输入exit ，然后按ENTER 。

Create the parent ZFS dataset mentioned in the configuration file:

创建配置文件中提到的父ZFS数据集：

sudo zfs create zroot/pdr
sudo zfs创建zroot / pdr
sudo zfs create zroot/pdr/w0
须藤zfs创建zroot / pdr / w0

This deliberately assumes that you may want to add more workers in the future and therefore creates a sub-dataset for your first worker. The dataset name is short on purpose, since older versions of FreeBSD (before 12.0) had a mount name limit of 88 characters.

这是故意假设您将来可能要添加更多工作人员，因此会为第一个工作人员创建一个子数据集。数据集名称的目的是简短的，因为较早版本的FreeBSD(12.0之前的版本)的装载名称限制为88个字符。

In order for a jail to take control of a parent dataset and administer any children, the dataset must be marked with the following flag:

为了使监狱能够控制父级数据集并管理任何子级，必须在数据集上标记以下标志：

sudo zfs set jailed=on zroot/pdr/w0
须藤zfs设置监禁= on zroot / pdr / w0

With the preconditions now met, the jail will start correctly with the new configuration:

在满足前提条件的情况下，监狱将以新配置正确启动：

sudo service jail restart buildbot-worker0
sudo服务监狱重启buildbot-worker0

With these instructions, you successfully created the required filesystems—ZFS datasets—and allowed the jail to manage the parent dataset. In the next step, you will configure Poudriere, which involves specifying the chosen zpool and dataset used to store build-related data.

通过这些说明，您成功创建了必需的文件系统-ZFS数据集-并允许监狱管理父数据集。在下一步中，您将配置Poudriere，其中涉及指定用于存储与构建相关的数据的所选zpool和数据集。

第4步-配置Poudriere，Build Jail和Ports树 (Step 4 — Configuring Poudriere, the Build Jail, and the Ports Tree)

Until this point, you’ve installed Poudriere and optionally covered requirements for package signing and ZFS. For Poudriere to be able to run in a “jailed” fashion—that is, functioning correctly from within the Buildbot worker jail—you need to provide certain permissions to the jail. For example, if you use ZFS, you have already delegated a parent dataset for use and administration by the jail.

到目前为止，您已经安装了Poudriere，并可以选择满足软件包签名和ZFS的要求。为了使Poudriere能够以“监禁”的方式运行(即在Buildbot工作人员监狱中正常运行)，您需要向监狱提供某些权限。例如，如果您使用ZFS，则已经委托了一个父数据集供监狱使用和管理。

Let’s first configure the loopback IP and all of the permissions, and then step through the respective meaning following the changes.

首先，我们配置回送IP和所有权限，然后按照更改进行逐步操作。

Poudriere wants to start two build jails per build: one with loopback-only networking and one with internet access. Only build stages that are supposed to reach the internet will use the latter. For example, the fetch may download source tarballs, but the build phase is not allowed internet access. The existing configuration of the worker jail has ip4.addr = "lo1|10.0.0.3/24" that allows internet access. In order to allow Poudriere to assign a loopback address to freshly started build jails, the IP must also be passed to its parent (the worker jail). For this to work, please ensure you have applied the latest version of the firewall configuration file /usr/local/etc/ipfw.rules from the prerequisite tutorial, which will block the loopback interface lo0 from opening outgoing connections through NAT.

Poudriere希望每个构建开始两个构建监狱：一个建立仅环回网络，另一个建立互联网访问。只有应该到达互联网的构建阶段才会使用后者。例如， fetch可能会下载源tarball，但不允许build阶段访问Internet。工人监狱的现有配置具有ip4.addr = "lo1|10.0.0.3/24" ，它可以访问Internet。为了允许Poudriere为新启动的构建监狱分配回送地址，还必须将IP传递给它的父监狱(工人监狱)。为此，请确保已从先决条件教程中应用了防火墙配置文件/usr/local/etc/ipfw.rules的最新版本，这将阻止回lo0接口lo0通过NAT打开传出连接。

Add the highlighted lines to your worker jail configuration:

将突出显示的行添加到您的工作监狱配置中：

sudo ee /etc/jail.buildbot-worker0.conf
须藤ee /etc/jail.buildbot-worker0.conf

/etc/jail.buildbot-worker0.conf

buildbot-worker0 {
    host.hostname = buildbot-worker0.localdomain;
    ip4.addr = "lo1|10.0.0.3/24";
    ip4.addr += "lo0|127.0.0.3";
    path = "/usr/jails/buildbot-worker0";
    exec.start = "/bin/sh /etc/rc";
    exec.stop = "/bin/sh /etc/rc.shutdown";
    mount.devfs; # need /dev/*random for Python
    persist;

    # If you followed the ZFS setup step, you have this line
    # already (keep it). For non-ZFS setup, this line must be absent.
    exec.poststart = "/sbin/zfs jail buildbot-worker0 zroot/pdr/w0";

    allow.chflags;
    allow.mount;
    allow.mount.devfs;
    allow.mount.nullfs;
    allow.mount.procfs;
    allow.mount.tmpfs;
    allow.mount.zfs; # only needed if you use ZFS
    allow.raw_sockets; # optional
    allow.socket_af; # optional
    allow.sysvipc; # optional
    children.max=16;
    enforce_statfs=1;
}

Here you’ve added the following (also see the jail(8) manpage):

在这里，您添加了以下内容(另请参见jail(8)手册页 )：

ip4.addr += "lo0|127.0.0.3" adds another IPv4 address to the jail. You will later configure Poudriere’s LOIP4 variable in order to assign this loopback address to build jails that are not supposed to talk to the internet or other machines in your network, such as during the build phase. If you ever have a build that requires internet access during build, Poudriere supports a variable ALLOW_NETWORKING_PACKAGES as a workaround. However, it is preferable to follow best practice and perform downloads, and other internet-facing tasks earlier, in the fetch phase for which Poudriere permits internet access.
ip4.addr += "lo0|127.0.0.3"将另一个IPv4地址添加到监狱。稍后，您将配置Poudriere的LOIP4变量，以便将此环回地址分配给构建不应与Internet或网络中其他计算机通信的监狱，例如在build阶段。如果您有一个在构建期间需要Internet访问的构建，则Poudriere支持使用变量ALLOW_NETWORKING_PACKAGES作为解决方法。但是，最好在Poudriere允许访问Internet的fetch存阶段中遵循最佳实践并尽早执行下载和其他面向Internet的任务。
allow.chflags allows Poudriere to render certain system files like /bin/sh immutable in the build jail.
allow.chflags允许Poudriere将某些系统文件(如/bin/sh呈现为在构建监狱中不可变。
allow.mount and the other allow.mount.* options enable Poudriere to mount certain required filesystems into the build jails.
allow.mount和其他allow.mount.*选项使Poudriere可以将某些必需的文件系统挂载到构建监狱中。
allow.raw_sockets which permits use of raw sockets, and allow.socket_af which permits use of any socket address family, are both applied to the internet-capable build jails. This is helpful so that you can run tools like ping in interactive mode, like when entering a build jail to debug problems.
allow.raw_sockets允许使用原始套接字，并allow.socket_af允许使用任何套接字地址族的，都适用于互联网功能的构建监狱。这很有用，因此您可以在交互模式下运行ping工具，例如在进入构建监狱以调试问题时。
allow.sysvipc is deprecated in favor of three separate settings sysvmsg/sysvsem/sysvshm to restrict jails to only see their own shared memory objects (via “SYS V” IPC primitives). However, Poudriere can only pass on allow.sysvipc to build jails because it cannot read the relevant sysctl information for the three separate parameters (as of FreeBSD 11.2). With this deprecated configuration, the jail could read shared memory of processes outside the jail. This is only relevant for certain software that depends on IPC features, like PostgreSQL, so chances are small for this to affect security. You can remove this configuration unless you depend on a port that requires it during build.
不赞成使用allow.sysvipc而建议使用三个单独的设置sysvmsg / sysvsem / sysvshm来限制监狱只能看到自己的共享内存对象(通过“ SYS V” IPC原语)。但是，Poudriere只能传递allow.sysvipc来构建jail，因为它无法读取三个单独参数的相关sysctl信息(从FreeBSD 11.2开始)。使用此不建议使用的配置，监狱可以读取监狱外部进程的共享内存。这仅与某些依赖IPC功能的软件有关，例如PostgreSQL，因此影响安全性的机会很小。您可以删除此配置，除非您在构建期间依赖于需要它的端口。
children.max=16 allows 16 sub-jails below the worker jail. You can raise this number later if you have a lot of CPUs and Poudriere tries to create more build jails than permitted. Each Poudriere build will try to create a reference jail and two build jails per “job”, and its default is to use the number of CPUs (as output by sysctl -n hw.ncpu) as the job count.
children.max=16允许工人监狱下方的16个子监狱。如果您有很多CPU，并且Poudriere尝试创建的构建监狱数量超过允许的数量，则可以稍后增加该数量。每个Poudriere构建都会尝试为每个“作业”创建一个参考监狱和两个构建监狱，其默认设置是使用CPU数量(由sysctl -n hw.ncpu输出)作为作业计数。
enforce_statfs=1 is required together with allow.mount in order to mount certain filesystems.
为了装入某些文件系统， allow.mount enforce_statfs=1与allow.mount一起需要。

Save and exit the configuration file.

保存并退出配置文件。

Restart the jail for its configuration to take affect immediately:

重新启动监狱，使其配置立即生效：

sudo service jail restart buildbot-worker0
sudo服务监狱重启buildbot-worker0

The respective kernel modules must be loaded so that Poudriere can perform mounts. Run the following commands to load the modules at boot time and immediately:

必须加载相应的内核模块，以便Poudriere可以执行安装。运行以下命令以在引导时立即加载模块：

sudo sysrc -f /boot/loader.conf nullfs_load=YES
须藤sysrc -f /boot/loader.conf nullfs_load = YES
sudo kldload -n nullfs
须藤kldload -n nullfs
sudo sysrc -f /boot/loader.conf tmpfs_load=YES
须藤sysrc -f /boot/loader.conf tmpfs_load = YES
sudo kldload -n tmpfs
须藤kldload -n tmpfs

You already installed the Poudriere package earlier, which has copied the sample file /usr/local/etc/poudriere.conf.sample to /usr/local/etc/poudriere.conf. Next, you will make edits to the configuration file. All possible configuration variables already exist in the sample, so uncomment or adapt the respective line in the file to set a variable to a certain value.

您之前已经安装了Poudriere软件包，该软件包已将示例文件/usr/local/etc/poudriere.conf.sample复制到/usr/local/etc/poudriere.conf 。接下来，您将对配置文件进行编辑。示例中已经存在所有可能的配置变量，因此请取消注释或修改文件中的相应行以将变量设置为某个值。

For the following commands, please ensure you are still in a root shell in the worker jail:

对于以下命令，请确保您仍在worker监狱的root shell中：

sudo jexec buildbot-worker0 csh
须藤jexec buildbot-worker0 csh

Open the file with the following command:

使用以下命令打开文件：

ee /usr/local/etc/poudriere.conf
EE /usr/local/etc/poudriere.conf

If you have decided to use ZFS, please fill in your desired zpool and parent dataset:

如果您决定使用ZFS，请填写所需的zpool和父数据集：

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# Poudriere can optionally use ZFS for its ports/jail storage. For
# ZFS define ZPOOL, otherwise set NO_ZFS=yes
#
#### ZFS
# The pool where poudriere will create all the filesystems it needs
# poudriere will use ${ZPOOL}/${ZROOTFS} as its root
#
# You need at least 7GB of free space in this pool to have a working
# poudriere.
#
ZPOOL=zroot

### NO ZFS
# To not use ZFS, define NO_ZFS=yes
#NO_ZFS=yes

# root of the poudriere zfs filesystem, by default /poudriere
ZROOTFS=/pdr/w0
. . .

Otherwise, if you decided against ZFS, please disable ZFS support:

否则，如果您决定不使用ZFS，请禁用ZFS支持：

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# Poudriere can optionally use ZFS for its ports/jail storage. For
# ZFS define ZPOOL, otherwise set NO_ZFS=yes
#
#### ZFS
# The pool where poudriere will create all the filesystems it needs
# poudriere will use ${ZPOOL}/${ZROOTFS} as its root
#
# You need at least 7GB of free space in this pool to have a working
# poudriere.
#
#ZPOOL=zroot

### NO ZFS
# To not use ZFS, define NO_ZFS=yes
NO_ZFS=yes

# root of the poudriere zfs filesystem, by default /poudriere
# ZROOTFS=/poudriere
. . .

You will later instruct Poudriere to download a FreeBSD base system and thereby bootstrap the first build jail. This requires specifying a download host, add the following highlighted line:

您稍后将指示Poudriere下载FreeBSD基本系统，从而引导第一个构建监狱。这需要指定下载主机，并添加以下突出显示的行：

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# the host where to download sets for the jails setup
# You can specify here a host or an IP
# replace _PROTO_ by http or ftp
# replace _CHANGE_THIS_ by the hostname of the mirrors where you want to fetch
# by default: ftp://ftp.freebsd.org
#
# Also note that every protocols supported by fetch(1) are supported here, even
# file:///
# Suggested: https://download.FreeBSD.org
FREEBSD_HOST=https://download.FreeBSD.org

Since Poudriere will run jailed, the mount name limit of 88 characters of FreeBSD versions before 12.0 is especially harmful, as the full path of the jail /usr/jails/buildbot-worker0 is part of each mount path. Exceeding the limit would fatally break the builds, so let’s take good care to reduce path lengths. Instead of the typical directory /usr/local/poudriere, you can use /pdr like the following:

由于Poudriere会被监禁运行，因此12.0之前的FreeBSD版本中88个字符的挂载名称限制特别有害，因为jail的完整路径/usr/jails/buildbot-worker0是每个挂载路径的一部分。超过限制将致命地破坏构建，因此，请当心减小路径长度。可以使用/pdr代替典型的目录/usr/local/poudriere ，如下所示：

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# The directory where poudriere will store jails and ports
BASEFS=/pdr

Now, create that directory:

现在，创建该目录：

mkdir /pdr
mkdir / pdr

Switch again to your editor of poudriere.conf:

再次切换到poudriere.conf的编辑器：

ee /usr/local/etc/poudriere.conf
EE /usr/local/etc/poudriere.conf

Poudriere will mount a central directory for dist files (the source code tarballs for each port) while running builds so that all builders share the same cache. The default directory is:

Poudriere将在运行构建时为dist文件安装中央目录(每个端口的源代码压缩文件)，以便所有构建器共享相同的缓存。默认目录为：

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# If set the given directory will be used for the distfiles
# This allows to share the distfiles between jails and ports tree
# If this is "no", poudriere must be supplied a ports tree that already has
# the required distfiles.
DISTFILES_CACHE=/usr/ports/distfiles

Now, create that directory:

现在，创建该目录：

mkdir -p /usr/ports/distfiles
mkdir -p / usr / ports / distfiles

If you followed Step 2 and created a package repository signing key, please enter the editor again and specify it:

如果您遵循步骤2并创建了软件包存储库签名密钥，请再次进入编辑器并指定它：

ee /usr/local/etc/poudriere.conf
EE /usr/local/etc/poudriere.conf

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# Path to the RSA key to sign the PKG repo with. See pkg-repo(8)
PKG_REPO_SIGNING_KEY=/usr/local/etc/poudriere.key

Builds will run much faster if you cache C/C++ compiler and linker outputs for next time. The ports tree supports this directly by leveraging the tool ccache. Please enable it and create the respective cache directory if you can spare at least 5GB more space (the default cache size):

如果您下次缓存C / C ++编译器和链接器输出，则构建将运行得更快。端口树通过利用工具ccache直接支持此功能。如果您可以保留至少5GB以上的空间(默认缓存大小)，请启用它并创建相应的缓存目录：

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# ccache support. Supply the path to your ccache cache directory.
# It will be mounted into the jail and be shared among all jails.
# It is recommended that extra ccache configuration be done with
# ccache -o rather than from the environment.
CCACHE_DIR=/var/cache/ccache

mkdir /var/cache/ccache
mkdir / var / cache / ccache

Building and running Linux software is uncommon, so disable it until needed:

构建和运行Linux软件并不常见，因此请在需要之前禁用它：

ee /usr/local/etc/poudriere.conf
EE /usr/local/etc/poudriere.conf

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

. . .
# Disable linux support
NOLINUX=yes

The jails should get a loopback address assigned, or Poudriere will warn about it. We can inherit the jail’s IP because it is on a loopback-only network interface (lo1). For this, please add the following line to the end of the configuration file:

监狱应该分配一个回送地址，否则Poudriere会对此发出警告。我们可以继承监狱的IP，因为它位于仅回送网络接口( lo1 )上。为此，请将以下行添加到配置文件的末尾：

/usr/local/etc/poudriere.conf (snippet)

/usr/local/etc/poudriere.conf(代码段)

LOIP4=127.0.0.3

Save and exit the configuration file.

保存并退出配置文件。

For working builds, we need two more resources: a FreeBSD base system to use as the build jail template and an up-to-date ports tree. Choose the FreeBSD version you are targeting. In this tutorial, we will tell Poudriere to download FreeBSD 11.2 for amd64 architecture. You can name the jail how you like, but a consistent naming scheme like 112amd64 is recommended. Also keep in mind the choice between quarterly, stable ports tree branches (here, we use 2019Q2) and the bleeding edge “head” branch that might lead to breaking builds after updates every now and then. FreeBSD versions newer than that on the server can’t be used in the build jail.

对于有效的构建，我们需要另外两个资源：用作构建监狱模板的FreeBSD基本系统和最新的ports树。选择您要定位的FreeBSD版本。在本教程中，我们将告诉Poudriere下载适用于amd64体系结构的FreeBSD 11.2 。您可以根据自己的喜好命名监狱，但建议使用一致的命名方案，例如112amd64 。还请记住，应在每季度稳定的端口树枝(这里我们使用2019Q2 )和最新的“ head”分支之间进行选择，这可能会导致2019Q2更新后的构建。比服务器上新的FreeBSD版本不能在构建监狱中使用。

Download and create the build jail:

下载并创建构建监狱：

poudriere jail -c -j 112amd64 -v 11.2-RELEASE -a amd64
poudriere监狱-c -j 112amd64 -v 11.2-RELEASE -a amd64

Lastly, let’s download the ports tree. The default download method is portsnap, which uses compressed snapshots of the tree without history information. Either Subversion or Git are preferable to merge upstream changes or contribute back. This is also important if you want to use a custom, self-hosted tree in a version control system. In the following command, please fill in the current year and quarter.

最后，让我们下载端口树。默认的下载方法是portsnap ，它使用树的压缩快照而没有历史记录信息。 Subversion或Git最好合并上游更改或回馈。如果要在版本控制系统中使用自定义的自托管树，这也很重要。在以下命令中，请填写当前年份和季度。

If you want to start with the upstream, official ports tree:

如果要从上游官方端口树开始：

poudriere ports -c -p 2019Q2 -m svn+https -B branches/2019Q2
Poudriere端口-c -p 2019Q2 -m svn + https -B 分支机构/ 2019Q2

The method svn+https would sync from the FreeBSD Subversion host (viewable online here). If you plan to use an alternative source, read the following note, otherwise skip it.

svn+https方法将从FreeBSD Subversion主机同步( 可在此处在线查看 )。如果您打算使用其他来源，请阅读以下说明，否则请跳过。

Note: As an alternative, the method git clones the tree from the mirror on GitHub by default.

注意：作为替代，默认情况下，方法git从GitHub上的镜像克隆树。

To use the “head” branch, replace the last parameter with -B head (for Subversion) or -B master (for Git).

要使用“ head”分支，请将最后一个参数替换为-B head (对于Subversion)或-B master (对于Git)。

If you prefer to use your own Git repository, you will have to explicitly specify your repository URL and branch name. Let’s assume you want to name your tree customtree and use the branch custom:

如果您喜欢使用自己的Git存储库，则必须明确指定存储库URL和分支名称。假设您要命名树customtree并使用branch custom ：

poudriere ports -c -p customtree -m git -B custom -U https://github.com/AndiDog/freebsd-ports.git
poudriere端口-c -p customtree -m git -B custom -U https://github.com/AndiDog/freebsd-ports.git

The example URL points to a fork of freebsd-ports on GitHub, but could be any Git or other supported type of repository to which the CI server has access.

示例URL指向GitHub上的freebsd-ports分支，但可以是CI服务器可以访问的任何Git或其他受支持的存储库类型。

Available trees can be listed with poudriere ports -l, which outputs a listing like:

可使用poudriere ports -l列出可用的树，其输出类似以下内容的列表：


   
   
    
    Output
   
   PORTSTREE METHOD    TIMESTAMP           PATH
2019Q2    svn+https 2019-04-20 19:23:19 /pdr/ports/2019Q2

You’re now done setting up Poudriere’s configuration and resources. You’ve configured Poudriere with the required data to trigger the first builds and enabled the jail to create subjails. Next, you’re going to run the first build manually to verify that the setup is working.

现在您已完成设置Poudriere的配置和资源。您已为Poudriere配置了所需的数据以触发第一个构建，并启用了监狱创建子监狱。接下来，您将手动运行第一个版本以验证设置是否正常。

步骤5 —运行手动测试版本 (Step 5 — Running a Manual Test Build)

You can use the command poudriere bulk to build one or more packages and all its dependencies. After the first build of a package, Poudriere also automatically detects if a rebuild is necessary, or otherwise leaves the existing package file untouched. While the bulk subcommand only builds packages, running a build using poudriere testport would also test the specified ports using the definition of “testing” given in the port’s Makefile. For the scope of this article, we’re only interested in providing packages for installation on clients, so we are using bulk builds.

您可以使用poudriere bulk命令来构建一个或多个软件包及其所有依赖项。在首次构建软件包之后，Poudriere还会自动检测是否需要重建，否则将保持原有的软件包文件不变。尽管bulk子命令仅生成软件包，但使用poudriere testport运行生成也将使用端口的Makefile中给出的“测试”定义来测试指定的端口。对于本文的范围，我们只希望提供用于在客户端上安装的软件包，因此我们正在使用批量构建。

Ensure you’re still in a root shell of the worker jail where you have installed Poudriere. Later on, this will also be where the Buildbot worker process will run builds automatically.

确保您仍在安装Poudriere的工作人员监狱的根外壳中。稍后，这也是Buildbot worker进程将自动运行构建的地方。

Run the build, filling in the placeholders with the build jail name and ports tree name you chose earlier:

运行构建，在占位符中填充您之前选择的构建监狱名称和端口树名称：

poudriere bulk -j 112amd64 -p 2019Q2 ports-mgmt/pkg
poudriere散装-j 112amd64 -p 2019Q2 ports-mgmt / pkg

This builds the port ports-mgmt/pkg. Ports in the official tree are stored in a <category>/<name> hierarchy, and those paths (called package origin) are used to tell Poudriere which packages should be built. For the start, we have chosen to only build the package manager pkg, which does not have any third-party dependencies and is therefore a good, quick check on the configuration. If everything runs fine, you’ll see output like this:

这将构建端口ports-mgmt/pkg 。官方树中的端口存储在<category>/<name>层次结构中，这些路径(称为包origin )用于告诉Poudriere应该构建哪些包。首先，我们选择只构建软件包管理器pkg ，它没有任何第三方依赖性，因此是对配置进行很好的快速检查。如果一切正常，您将看到以下输出：


   
   
    
    Output
   
   [00:00:00] Creating the reference jail... done
[00:00:06] Mounting system devices for 112amd64-2019Q2
[00:00:06] Mounting ports/packages/distfiles
[00:00:06] Using packages from previously failed build
[00:00:06] Mounting ccache from: /var/cache/ccache
[00:00:06] Mounting packages from: /pdr/data/packages/112amd64-2019Q2
/etc/resolv.conf -> /pdr/data/.m/112amd64-2019Q2/ref/etc/resolv.conf
[00:00:06] Starting jail 112amd64-2019Q2
[00:00:07] Logs: /pdr/data/logs/bulk/112amd64-2019Q2/2019-04-20_19h35m00s
[00:00:07] Loading MOVED for /pdr/data/.m/112amd64-2019Q2/ref/usr/ports
[00:00:08] Ports supports: FLAVORS SELECTED_OPTIONS
[00:00:08] Gathering ports metadata
[00:00:08] Calculating ports order and dependencies
[00:00:08] pkg package missing, skipping sanity
[00:00:08] Skipping incremental rebuild and repository sanity checks
[00:00:08] Cleaning the build queue
[00:00:08] Sanity checking build queue
[00:00:08] Processing PRIORITY_BOOST
[00:00:08] Balancing pool
[00:00:08] Recording filesystem state for prepkg... done
[00:00:08] Building 1 packages using 1 builders
[00:00:08] Starting/Cloning builders
[00:00:14] Hit CTRL+t at any time to see build progress and stats
[00:00:14] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.10.5_5
[00:03:24] [01] [00:03:10] Finished ports-mgmt/pkg | pkg-1.10.5_5: Success
[00:03:25] Stopping 1 builders
[00:03:25] Creating pkg repository
Creating repository in /tmp/packages: 100%
Packing files for repository: 100%
[00:03:25] Committing packages to repository
[00:03:25] Removing old packages
[00:03:25] Built ports: ports-mgmt/pkg
[112amd64-2019Q2] [2019-04-20_19h35m00s] [committing:] Queued: 1  Built: 1  Failed: 0  Skipped: 0  Ignored: 0  Tobuild: 0   Time: 00:03:18
[00:03:25] Logs: /pdr/data/logs/bulk/112amd64-2019Q2/2019-04-20_19h35m00s
[00:03:25] Cleaning up
[00:03:25] Unmounting file systems

This output shows where packages will go after build, and from where existing packages are taken in case they don’t need rebuild (here: /pdr/data/packages/112amd64-2019Q2). Also, the output shows an overview of running builds while Poudriere runs (you can press CTRL+T in an interactive shell to print the progress). In the final summary you’ll see one package was built. You can view verbose build output in the log directory (/pdr/data/logs/bulk/112amd64-2019Q2/*).

此输出显示软件包在构建/pdr/data/packages/112amd64-2019Q2何处，以及在不需要重建的情况下从何处获取现有软件包(在此处： /pdr/data/packages/112amd64-2019Q2 )。同样，输出显示Poudriere运行时正在运行的构建的概述(您可以在交互式Shell中按CTRL+T来打印进度)。在最后的摘要中，您将看到已构建一个软件包。您可以在日志目录( /pdr/data/logs/bulk/112amd64-2019Q2/* )中查看详细的构建输出。

This output confirms a successful build. If Poudriere has built at least one package successfully, it will automatically commit it to the package repository. This means that packages are only available after all builds have finished, even if other packages failed to build. You now have a working package repository at /pdr/data/packages/112amd64-2019Q2 within the Buildbot worker jail.

此输出确认构建成功。如果Poudriere成功构建了至少一个软件包，它将自动将其提交到软件包存储库。这意味着，即使其他程序包构建失败，程序包也仅在所有构建完成后才可用。您现在在Buildbot工作人员监狱中的/pdr/data/packages/112amd64-2019Q2处具有一个可用的软件包存储库。

You’ve completed all the configuration needed to return working Poudriere builds, and you’ve successfully verified with a manual build. You’ll see this same output later in the tutorial once you’ve automated the bulk build in Buildbot. In addition, a link to view the detailed logs shall be accessible from the web interface. To achieve this, and to serve the package repository to clients, you’ll set up a web server next.

您已经完成了返回工作的Poudriere构建所需的所有配置，并且已成功通过手动构建进行了验证。在Buildbot中自动完成批量构建后，您将在本教程的稍后部分看到相同的输出。此外，应可从Web界面访问查看详细日志的链接。为此，并将软件包存储库提供给客户端，接下来将设置一个Web服务器。

第6步—配置Nginx服务于Poudriere Web界面和程序包存储库 (Step 6 — Configuring Nginx to Serve the Poudriere Web Interface and Package Repository)

Poudriere provides several output artifacts that we want to host using a web server:

Poudriere提供了一些我们希望使用Web服务器托管的输出工件：

Package repositories are made available to clients so they can access them with the regular pkg update and pkg install commands, using HTTPS or HTTP as transport.
软件包存储库可供客户端使用，因此它们可以使用HTTPS或HTTP作为传输，通过常规的pkg update和pkg install命令访问它们。
Detailed build logs are helpful for developers to debug problematic builds or to investigate build output. They are stored per package and per build—in the Poudriere output from the last step, you saw that logs are stored in one directory per build, labeled with date and time.
详细的构建日志对于开发人员调试有问题的构建或调查构建输出很有帮助。它们按包和按构建存储，在最后一步的Poudriere输出中，您看到日志存储在按构建的目录中，并标有日期和时间。
Poudriere’s built-in web interface is a small, single HTML page per build that uses WebSockets to regularly update the status shown on the page. This is helpful to get a better overview of how far a build is, which dependencies triggered other package builds to fail, and lastly as a replacement for the command line output, which only shows a summary at the end unless you specifically make it print the current build progress.
Poudriere的内置Web界面是每个构建的单个HTML小页面，该页面使用WebSockets定期更新页面上显示的状态。这有助于更好地了解构建的程度，哪些依赖项触发了其他软件包的构建失败，以及最后作为命令行输出的替代，命令行输出仅在末尾显示摘要，除非您明确地将其打印出来。当前的构建进度。

The configuration change in Nginx is short, as only static files need to be served. Since you’ll serve them to the outside world, you’re now going to configure the existing Nginx instance on the server, outside the jails, to serve the mentioned files from paths within the worker jail.

Nginx中的配置更改很短，因为仅需要提供静态文件。由于将它们提供给外部环境，因此现在您将在监狱之外的服务器上配置现有的Nginx实例，以从工作监狱中的路径提供上述文件。

Please exit the jail shell since you’re now going to work on the server:

因为您现在要在服务器上工作，所以请退出监狱外壳：

exit
出口

Open an editor with the Nginx configuration /usr/local/etc/nginx/nginx.conf:

使用Nginx配置/usr/local/etc/nginx/nginx.conf打开编辑器：

sudo ee /usr/local/etc/nginx/nginx.conf
须藤ee /usr/local/etc/nginx/nginx.conf

Add the following locations inside the server { block:

在server {块内添加以下位置：

/usr/local/etc/nginx/nginx.conf

. . .
http {
    . . .
    server {
        . . .
        location / {
            root /usr/local/www/nginx;
            index index.html index.htm;
        }

        # poudriere logs
        location ~ ^/logs(/(.*))?$ {
            include mime.types;
            types {
                text/plain log;
            }

            alias /usr/jails/buildbot-worker0/pdr/data/logs/bulk$1;
            index index.html index.htm;
            autoindex on;
        }

        # poudriere packages
        location ~ ^/packages(/(.*))?$ {
            alias /usr/jails/buildbot-worker0/pdr/data/packages$1;
            index no-index-file-but-required-directive-to-list-dir-contents;
            autoindex on;
        }

        location /buildbot/ {
            proxy_pass http://10.0.0.2:8010/;
        }

        . . .
    }
}
. . .

Save and close the Nginx configuration file. Then, reload the Nginx service:

保存并关闭Nginx配置文件。然后，重新加载Nginx服务：

sudo service nginx reload
sudo服务nginx重新加载

Let’s now check out the artifacts created by the first manual build. Open up your preferred web browser on your local machine to access the resources.

现在，让我们检查一下由第一次手动构建创建的工件。在本地计算机上打开首选的Web浏览器以访问资源。

The package repository is below https://your-domain/packages/ (or http://your-server-ip/). You will find meta information in the root directory, e.g. 112amd64-2019Q2, and all built packages in the subdirectory All:

软件包存储库位于http s :// your-domain /packages/ (或http:// your-server-ip / )下面。您将在根目录(例如112amd64-2019Q2找到元信息，并在子目录All所有已构建的软件包：

Detailed build logs and Poudriere’s built-in web interface can be found below https://your-domain/logs/. Click through the directory hierarchy to reach the data of your previous manual build. In this example, you might end up on a URL like https://your-domain/logs/112amd64-2019Q2/latest/build.html.

详细的构建日志和Poudriere的内置Web界面可在http s :// your-domain /logs/ 。在目录层次结构中单击以获取以前的手动构建的数据。在此示例中，您可能会以类似http s :// your-domain /logs/112amd64-2019Q2/latest/build.html的URL http s :// your-domain /logs/112amd64-2019Q2/latest/build.html 。

If you did not set up a domain name for your server, you will need to enter your server’s public IP address for these examples, e.g. http://your-server-ip/logs/.

如果您没有为服务器设置域名，则需要为这些示例输入服务器的公共IP地址，例如http:// your-server-ip /logs/ 。

This concludes all manual setup to get working builds and have visibility into the output (packages and logs). Going forward, you will automate builds to achieve continuous integration.

这样就完成了所有手动设置，可以开始构建并可以查看输出(包和日志)。展望未来，您将使构建自动化以实现持续集成。

第7步-为您的软件包设置Buildbot生成器 (Step 7 — Setting Up a Buildbot Builder for Your Packages)

Your goal in this step is to automate bulk-package builds by executing Poudriere in the same way you already have manually—by adding to the existing Buildbot sample configuration. By the end of this step, Buildbot will trigger the package build whenever the chosen branch of the ports tree changes. In this tutorial’s examples, that would be the quarterly branch 2019Q2.

您在此步骤中的目标是通过以与现有手动方式相同的方式执行Poudriere(通过添加到现有Buildbot示例配置中)来自动化批量软件包的构建。在此步骤结束时，只要端口树的选定分支发生更改，Buildbot将触发软件包的构建。在本教程的示例中，这将是2019Q2的季度分支。

All necessary changes are done in the Buildbot master configuration, so please open a root shell in the master jail:

所有必要的更改都在Buildbot主配置中完成，因此请在主监狱中打开一个根 shell：

sudo jexec buildbot-master csh
须藤jexec buildbot-master csh

First, a builder must be defined that describes the commands and actions performed to run a build. In the existing configuration /var/buildbot-master/master.cfg, you will find a section ####### BUILDERS—open an editor and replace the whole section until the next heading starting with ####### ..., with the following configuration:

首先，必须定义一个构建器 ，该构建器描述运行构建所执行的命令和操作。在现有的配置/var/buildbot-master/master.cfg ，您将找到####### BUILDERS部分- ####### BUILDERS编辑器并替换整个部分，直到下一个标题为####### ... ，具有以下配置：

ee /var/buildbot-master/master.cfg
EE /var/buildbot-master/master.cfg

/var/buildbot-master/master.cfg (snippet)

/var/buildbot-master/master.cfg(代码段)

. . .
####### BUILDERS

c['builders'] = []

PORTS_TO_BUILD = {
    'security/sudo',
    'shells/bash',
    'sysutils/tmux',
}


# Custom classes
class PoudriereLogLineObserver(util.LogLineObserver):
    _logsRe = re.compile(r'Logs: /pdr/data/logs/bulk(/[-_/0-9A-Za-z]+)$')

    def __init__(self):
        super().__init__()
        self._hadUrls = False

    def outLineReceived(self, line):
        if not self._hadUrls:
            m = self._logsRe.search(line.strip())
            if m:
                poudriereUiUrl = f'''{re.sub('/buildbot/$', '', c['buildbotURL'])}/logs{m.group(1)}'''
                self.step.addURL('Poudriere build', poudriereUiUrl)
                self.step.addURL('Poudriere logs', poudriereUiUrl + '/logs/')
                self._hadUrls = True


class PoudriereCompileStep(steps.Compile):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.addLogObserver('stdio', PoudriereLogLineObserver())


# Poudriere bulk build
bulkBuildFactory = util.BuildFactory()
bulkBuildFactory.addSteps([
    steps.ShellCommand(
        name='update ports tree',
        command=['sudo', 'poudriere', 'ports', '-u', '-p', '2019Q2', '-v'],
        haltOnFailure=True,
    ),
    PoudriereCompileStep(
        name='make bulk',
        command=['sudo', 'poudriere', 'bulk', '-j', '112amd64', '-p', '2019Q2'] + list(sorted(PORTS_TO_BUILD)),
        haltOnFailure=True,
    ),
])
c['builders'].append(util.BuilderConfig(name='bulk-112amd64-2019Q2',
                                        workernames=['worker0'],
                                        factory=bulkBuildFactory))
. . .

Note how this makes use of Buildbot’s extensibility: custom classes are used to observe and parse information from Poudriere’s log output. Namely, PoudriereLogLineObserver is added as “log observer”, i.e. gets called whenever a new log line is printed during the build. The class searches the logs for the log directory and converts that into hyperlinks. Those links will be displayed alongside the build step and take the user directly to Poudriere’s web interface and logs.

注意这是如何利用Buildbot的可扩展性的：自定义类用于观察和解析Poudriere日志输出中的信息。即， PoudriereLogLineObserver被添加为“日志观察者”，即在构建过程中每当打印新日志行时都将调用它。该类在日志中搜索日志目录，并将其转换为超链接。这些链接将在构建步骤的旁边显示，并将用户直接带到Poudriere的Web界面和日志。

In the first build step “update ports tree”, we use Poudriere’s built-in update command (ports -u) to pull the latest version of the ports tree. This will use the previously configured method automatically (for example SVN/Git). This way, you can be sure the packages are always built against the latest committed tree, which is especially helpful if you have your own versioned repository where you maintain software versions and patches.

在第一步“更新端口树”中，我们使用Poudriere的内置更新命令( ports -u )拉取端口树的最新版本。这将自动使用先前配置的方法(例如SVN / Git)。这样，您可以确保始终根据最新的提交树来构建软件包，如果您拥有自己的版本库来维护软件版本和补丁程序，这将特别有用。

At the top, the list PORTS_TO_BUILD specifies which ports should be built. It is used in the steps of the build factory specified at the bottom of the block. The build factory is a template used to instantiate a build. Buildbot creates a unique build whenever one is triggered, and the build uses a copy of the steps that were defined for the build factory at the time. In this case, we configured exactly two steps:

在顶部，列表PORTS_TO_BUILD指定应构建的端口。它在块底部指定的构建工厂的步骤中使用。构建工厂是用于实例化构建的模板。每当触发一个构建器时，Buildbot就会创建一个唯一的构建，并且该构建使用当时为构建工厂定义的步骤的副本。在这种情况下，我们恰好配置了两个步骤：

Update the ports tree. Since this example uses the quarterly branch 2019Q2, it will not receive changes very often (typically only security and build fixes).
更新端口树。由于此示例使用季度分支2019Q2 ，因此它不会经常收到更改(通常仅接收安全性和构建修补程序)。
Run the bulk build using the same tree.
使用同一棵树运行批量构建。

To make the added code block work, please add a required import to the top of the file:

为了使添加的代码块起作用，请在文件顶部添加必需的导入：

/var/buildbot-master/master.cfg (snippet)

/var/buildbot-master/master.cfg(代码段)

# -*- python -*-
# ex: set filetype=python:

import re

from buildbot.plugins import *

The re library in Python implements regular expressions, a feature to search or replace parts of a string—the PoudriereLogLineObserver class uses it to search for a line Logs: /pdr/data/logs/... that mentions the log directory.

Python中的re库实现了正则表达式 ，即搜索或替换字符串部分的功能PoudriereLogLineObserver类使用它来搜索行Logs: /pdr/data/logs/... ，其中提到了日志目录。

The build commands use sudo to run certain commands. This is required because Poudriere needs superuser privileges when running a build—in order to create, manage, and destroy the build jails—and also the ports trees managed by Poudriere are created with the root user as owner. In the previous tutorial, we configured the user that runs the Buildbot worker process with sysrc buildbot_worker_uid=buildbot-worker. Hence, we want to allow the buildbot-worker user to run exactly the necessary commands as root, but not other commands (for security reasons). Let’s install the sudo program and configure it accordingly.

构建命令使用sudo运行某些命令。这是必需的，因为Poudriere在运行构建时需要超级用户特权(以便创建，管理和销毁构建监狱)，而且Poudriere管理的端口树都是以root用户作为所有者创建的。在上一教程中，我们使用sysrc buildbot_worker_uid=buildbot-worker配置了运行Buildbot工作进程的用户。因此，我们希望允许buildbot-worker用户完全以root用户身份运行必要的命令，而不是其他命令(出于安全原因)。让我们安装sudo程序并进行相应的配置。

This needs to be done on the worker jail, not the master. Please exit the master jail shell and enter the worker jail:

这需要在工人监狱而不是主人监狱上完成。请退出主监狱外壳并进入工人监狱：

exit
出口

sudo jexec buildbot-worker0 csh
须藤jexec buildbot-worker0 csh

Install the sudo package:

安装sudo软件包：

pkg install sudo
pkg install sudo

Confirm installation with y and ENTER.

使用y和ENTER确认安装。

On FreeBSD, the sudo package by default reads configuration files from /usr/local/etc/sudoers.d/. Open an editor to create a new configuration file:

在FreeBSD上， sudo软件包默认从/usr/local/etc/sudoers.d/读取配置文件。打开编辑器以创建新的配置文件：

env EDITOR=ee visudo /usr/local/etc/sudoers.d/buildbot-worker
env EDITOR = ee visudo /usr/local/etc/sudoers.d/buildbot-worker

The use of visudo is intentional, since it will warn on syntax errors and allow fixing them instead of committing a bad configuration.

visudo的使用是有意的，因为它会警告语法错误并允许修复它们，而不用提交错误的配置。

Specify which commands the buildbot-worker user can run as root without requiring any password:

指定buildbot-worker用户可以以root身份运行而无需任何密码的命令：

/usr/local/etc/sudoers.d/buildbot-worker

buildbot-worker ALL=(ALL) NOPASSWD: /usr/local/bin/poudriere bulk *
buildbot-worker ALL=(ALL) NOPASSWD: /usr/local/bin/poudriere ports -u *

Save the file and switch back to the master jail for further required configuration of the Buildbot master:

保存文件，然后切换回主监狱，以进一步构建Buildbot主文件：

exit
出口

sudo jexec buildbot-master csh
sudo jexec buildbot-master csh

You just fulfilled the requirements to get the bulk build to work. But as mentioned, each build must be triggered to run. Buildbot uses the term scheduler for an object that defines when a build is triggered, and with which extra information, such as which branch has been changed. Please remove the existing section SCHEDULERS from the configuration file, and place the following content after the BUILDERS section, so that the code can use all existing builder names:

ee /var/buildbot-master/master.cfg
ee /var/buildbot-master/master.cfg

/var/buildbot-master/master.cfg (snippet)

. . .
####### SCHEDULERS

c['schedulers'] = []

# Forceful scheduler allowed for all builders
c['schedulers'].append(schedulers.ForceScheduler(
    name='force',
    builderNames=[builder.name for builder in c['builders']]))

# Watch ports tree for changes on given branch
c['schedulers'].append(schedulers.SingleBranchScheduler(
    name='sched-bulk-112amd64-2019Q2',
    change_filter=util.ChangeFilter(project='freebsd-ports', branch='branches/2019Q2'),
    builderNames=['bulk-112amd64-2019Q2']))
. . .

This replaces the sample configuration so that a force button appears on every builder. And most importantly, it creates a scheduler that watches all changes pertaining to the given project/branch and triggers a build for each change. Yet, no such change events can occur—you first have to create a change source. Typically, those are version control systems like SVN or Git on which one can detect changes on a branch. Buildbot supports the most popular ones, so we can use its functionality to add our chosen upstream ports tree repository as source. Completely replace the section CHANGESOURCES with the following configuration:

This replaces the sample configuration so that a force button appears on every builder. And most importantly, it creates a scheduler that watches all changes pertaining to the given project / branch and triggers a build for each change. Yet, no such change events can occur—you first have to create a change source . Typically, those are version control systems like SVN or Git on which one can detect changes on a branch. Buildbot supports the most popular ones, so we can use its functionality to add our chosen upstream ports tree repository as source. Completely replace the section CHANGESOURCES with the following configuration:

/var/buildbot-master/master.cfg (snippet)

. . .
####### CHANGESOURCES

c['change_source'] = []

c['change_source'].append(changes.SVNPoller(
    'svn://svn.freebsd.org/ports/',
    project='freebsd-ports',
    split_file=util.svn.split_file_branches,
    svnbin='svnlite',
    pollInterval=4 * 3600))

# Example for Git:
# c['change_source'].append(changes.GitPoller(
#     repourl='https://github.com/AndiDog/freebsd-ports.git',
#     project='freebsd-ports',
#     branches=['custom'],
#     pollInterval=4 * 3600))
. . .

This polls the SVN repository every four hours on the Buildbot master, and any new (not seen before) changes are forwarded to matching schedulers which in turn would trigger builds that are eventually dispatched to run on our single Buildbot worker. The ports tree is very large, and at first run these pollers will download the full history (for Git, only the specified branches), which can take a few minutes and require significant space (several gigabytes).

Apply the new configuration file by restarting Buildbot:

service buildbot restart
service buildbot restart

In this example, you have used the upstream ports collection from svn://svn.freebsd.org/ports/ and builds are scheduled whenever the branch 2019Q2 changes. As noted before, quarterly branches are mostly stable and do not receive updates very often. Since you probably do not want to wait for such a change to come in before the build is triggered the first time, let’s run it once by hand.

Open your Buildbot web interface (https://your-domain/buildbot/), navigate to Builds > Builders > bulk-112amd64-2019Q2. It will not show any builds yet.

Open your Buildbot web interface ( http s :// your-domain /buildbot/ ), navigate to Builds > Builders > bulk-112amd64-2019Q2 . It will not show any builds yet.

Click the force button at the top-right and then Start Build. That will trigger the build using its default settings, i.e. reason, branch, and other values are not overridden. The “update ports tree” step might take a minute to run, and eventually the Poudriere build should also run through successfully. The web interface will show the build as successful.

Click the force button at the top-right and then Start Build . That will trigger the build using its default settings, ie reason, branch, and other values are not overridden. The “update ports tree” step might take a minute to run, and eventually the Poudriere build should also run through successfully. The web interface will show the build as successful.

Clicking one of the links (Poudriere build and Poudriere logs) will take you to the Poudriere web interface and build logs for this specific build, respectively (as shown in Step 6). Expand by clicking the arrow next to make bulk and then stdio > view all … lines to show the full output of the poudriere bulk ... command.

Clicking one of the links ( Poudriere build and Poudriere logs ) will take you to the Poudriere web interface and build logs for this specific build, respectively (as shown in Step 6). Expand by clicking the arrow next to make bulk and then stdio > view all … lines to show the full output of the poudriere bulk ... command.

Having completed the first build, the packages are now available, as configured in Nginx in Step 6. Head to https://your-domain/packages/ (or http://your-server-ip/packages/) in a browser and click through the package repository created by Poudriere. You can find the actual package files (*.txz) once you enter one of the repositories and navigate to the All/ subdirectory.

Having completed the first build, the packages are now available, as configured in Nginx in Step 6. Head to http s :// your-domain /packages/ (or http:// your-server-ip /packages/ ) in a browser and click through the package repository created by Poudriere. You can find the actual package files ( *.txz ) once you enter one of the repositories and navigate to the All/ subdirectory.

Now that packages are available over HTTPS (or HTTP if you decided so) and built automatically on ports tree changes, you can configure one or more hosts to use those packages.

Step 8 — Configuring Package Clients (Step 8 — Configuring Package Clients)

In this step, you need a second FreeBSD server and will set it up such that it can fetch and install the packages built on the CI server. We will call this second server the package client.

In this step, you need a second FreeBSD server and will set it up such that it can fetch and install the packages built on the CI server. We will call this second server the package client .

SSH into the client host. Most remaining instructions in this section will be done on the client:

SSH into the client host. Most remaining instructions in this section will be done on the client :

ssh package-client
ssh package-client

Create the directory for custom package repository configurations:

sudo mkdir -p /usr/local/etc/pkg/repos
sudo mkdir -p /usr/local/etc/pkg/repos

As root user, open an editor to create the file /usr/local/etc/pkg/repos/ci.conf, and specify how and from where to retrieve packages:

As root user, open an editor to create the file /usr/local/etc/pkg/repos/ci.conf , and specify how and from where to retrieve packages:

sudo ee /usr/local/etc/pkg/repos/ci.conf
sudo ee /usr/local/etc/pkg/repos/ci.conf

In case you chose package signing, use this content:

/usr/local/etc/pkg/repos/ci.conf

ci: {
    url: "https://your-domain/packages/112amd64-2019Q2",
    signature_type: "pubkey",
    pubkey: "/usr/local/etc/pkg/repos/ci.pub",
    enabled: yes
}

Alternatively, if you decided to go without package signing, disable signature checks as follows:

/usr/local/etc/pkg/repos/ci.conf

ci: {
    url: "https://your-domain/packages/112amd64-2019Q2",
    signature_type: "none",
    enabled: yes
}

Note: This note applies only if you followed Step 2 to create a package repository signing key. Please skip it otherwise.

From your local machine, upload the public key to the package client:

From your local machine , upload the public key to the package client:

scp /tmp/poudriere.pub package-client:/tmp/ci.pub
scp /tmp/poudriere.pub package-client :/tmp/ci.pub

Using the client shell again, move the key into place so it can verify the authenticity of packages:

sudo mv /tmp/ci.pub /usr/local/etc/pkg/repos/ci.pub
sudo mv /tmp/ci.pub /usr/local/etc/pkg/repos/ci.pub

You completed configuring the package repository and enabled it, but on a regular FreeBSD installation, the official package repository “FreeBSD” would be enabled as well. Mixing installed packages from different sources is a foolproof way to have your production software crash at some point due to incompatible software versions or differing ABI, API, or build options. All packages on a host should stem from the same source.

The default configuration of the official repository is stored in /etc/pkg/FreeBSD.conf. This file belongs to the base system and should not be touched. However, you can override its settings—namely, we want to disable the repository altogether—by adding the respective flag in a configuration file under /usr/local/etc/pkg/repos, where also your own repository is configured. Please create a new file /usr/local/etc/pkg/repos/FreeBSD.conf with an editor, and use the following content to disable the FreeBSD repository:

The default configuration of the official repository is stored in /etc/pkg/FreeBSD.conf . This file belongs to the base system and should not be touched. However, you can override its settings—namely, we want to disable the repository altogether—by adding the respective flag in a configuration file under /usr/local/etc/pkg/repos , where also your own repository is configured. Please create a new file /usr/local/etc/pkg/repos/FreeBSD.conf with an editor, and use the following content to disable the FreeBSD repository:

sudo ee /usr/local/etc/pkg/repos/FreeBSD.conf
sudo ee /usr/local/etc/pkg/repos/FreeBSD.conf

/usr/local/etc/pkg/repos/FreeBSD.conf

FreeBSD: {
    enabled: no
}

If you are on a fully pristine package client host, no packages are installed yet and you can immediately begin using your own package repository. However, if even only one package was installed from another source, you are recommended to uninstall those packages and begin from scratch using your own source. The package manager pkg itself is installed as a package—to solve the chicken-and-egg problem, FreeBSD’s base system ships with a small executable /usr/sbin/pkg, which can bootstrap the package manager. That is, download the pkg package and install it as the very first package on the system. From that point on, the executable /usr/local/sbin/pkg of that package supports you as full-blown package manager.

If you are on a fully pristine package client host, no packages are installed yet and you can immediately begin using your own package repository. However, if even only one package was installed from another source, you are recommended to uninstall those packages and begin from scratch using your own source. The package manager pkg itself is installed as a package—to solve the chicken-and-egg problem, FreeBSD's base system ships with a small executable /usr/sbin/pkg , which can bootstrap the package manager. That is, download the pkg package and install it as the very first package on the system. From that point on, the executable /usr/local/sbin/pkg of that package supports you as full-blown package manager.

Run the following command to bootstrap pkg:

Run the following command to bootstrap pkg :

sudo pkg bootstrap
sudo pkg bootstrap

In the output of pkg bootstrap, you should see that packages are taken from your own package repository which we called ci in the configuration file. If you are using a package signing key, the output will also hint about the security verification.

In the output of pkg bootstrap , you should see that packages are taken from your own package repository which we called ci in the configuration file. If you are using a package signing key, the output will also hint about the security verification.


   
   
    
    Output
   
   The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from https://your-domain/packages/112amd64-2019Q2, please wait...
Verifying signature with public key /usr/local/etc/pkg/repos/ci.pub... done
Installing pkg-1.10.5_5...
Extracting pkg-1.10.5_5: 100%

If you see this successful output, please skip to the next note block. However, if the package manager or other packages had already been installed from another source, and you get this error:


   
   
    
    Output
   
   pkg already bootstrapped at /usr/local/sbin/pkg

Then please follow the instructions in the note.

Note – only if package manager was bootstrapped already:

You can list installed packages with pkg info. In this case, you should uninstall all of them including pkg, and reinstall them later. To do that, please first list the manually installed packages with pkg query -e "%a==0" "%n". Remember which of them you want to install again later. If, for instance, you use a shell which is not part of the base system (e.g. bash is an external package), you will want to reinstall it later or you might not be able to log in again.

You can list installed packages with pkg info . In this case, you should uninstall all of them including pkg , and reinstall them later. To do that, please first list the manually installed packages with pkg query -e "%a==0" "%n" . Remember which of them you want to install again later. If, for instance, you use a shell which is not part of the base system (eg bash is an external package), you will want to reinstall it later or you might not be able to log in again.

The following commands will remove all existing packages and the package manager, bootstrap the package manager again from your own package repository, and give an example of reinstalling your desired packages such as bash. Note though that you will only be able to install packages that you have built through the CI, i.e. listed in the Buildbot master configuration (variable PORTS_TO_BUILD).

The following commands will remove all existing packages and the package manager, bootstrap the package manager again from your own package repository, and give an example of reinstalling your desired packages such as bash. Note though that you will only be able to install packages that you have built through the CI, ie listed in the Buildbot master configuration (variable PORTS_TO_BUILD ).

First, open a root shell before uninstalling the sudo package, or else you may not be able to gain superuser privileges anymore. Keep it open until you have bootstrapped pkg through the course of the tutorial and successfully reinstalled sudo:

sudo sh
sudo sh

Uninstall all packages, including pkg:

Uninstall all packages, including pkg :

pkg delete --all --force
pkg delete --all --force

Bootstrap the package manager:

pkg bootstrap
pkg bootstrap

Confirm to bootstrap the package manager by pressing y, followed by ENTER.

Confirm to bootstrap the package manager by pressing y , followed by ENTER .

In the likely case that you set up your package host using a Let’s Encrypt certificate for HTTPS, you will run into the chicken-and-egg problem where your package host is not trusted but you would need to install the package ca_root_nss (containing trustworthy root certificate authorities) to trust the Let’s Encrypt CA and thereby also trust the server hosting your custom-built packages. The same problem would arise if you used an internal CA (self-signed by you or your company). Certificate verification errors would result in error output like this when bootstrapping the package manager:

In the likely case that you set up your package host using a Let's Encrypt certificate for HTTPS, you will run into the chicken-and-egg problem where your package host is not trusted but you would need to install the package ca_root_nss (containing trustworthy root certificate authorities) to trust the Let's Encrypt CA and thereby also trust the server hosting your custom-built packages. The same problem would arise if you used an internal CA (self-signed by you or your company). Certificate verification errors would result in error output like this when bootstrapping the package manager:


   
   
    
    Output
   
   The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from https://example.com/packages/112amd64-2019Q2, please wait...
Certificate verification failed for /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
34389740104:error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed:/usr/src/crypto/openssl/ssl/s3_clnt.c:1269:
[...]

If you see this error, please follow the instructions in the note below. Otherwise, you are all set and can skip this part and continue after the note.

Note – only if using HTTPS and certificate verification failed:

There is one straight workaround: trust the security of the package signing key, hence bootstrapping pkg and installing the ca_root_nss package via unencrypted HTTP. Since this is not always an option because of privacy concerns, blocked HTTP ports etc., we should prefer a more “best practice” way. The official FreeBSD repository is also signed by Let’s Encrypt, so we cannot simply install the ca_root_nss package from there. No matter which CA it is, you are recommended to set up your package clients with a fixed set of HTTPS CAs to trust. You can achieve exactly that within the next few instructions. We are going to assume this is for Let’s Encrypt, but the instructions will work the same way for your own, self-signed CA (you’ll need its certificate chain handy).

There is one straight workaround: trust the security of the package signing key, hence bootstrapping pkg and installing the ca_root_nss package via unencrypted HTTP. Since this is not always an option because of privacy concerns, blocked HTTP ports etc., we should prefer a more “best practice” way. The official FreeBSD repository is also signed by Let's Encrypt, so we cannot simply install the ca_root_nss package from there. No matter which CA it is, you are recommended to set up your package clients with a fixed set of HTTPS CAs to trust. You can achieve exactly that within the next few instructions. We are going to assume this is for Let's Encrypt, but the instructions will work the same way for your own, self-signed CA (you'll need its certificate chain handy).

In your web browser, visit Let’s Encrypt’s certificate listing at https://letsencrypt.org/certificates/. Make sure the web site is trusted by the browser. Download the certificates under Root Certificates > Active > ISRG Root X1 (self-signed) and Intermediate Certificates > Active > Let’s Encrypt Authority X3 (Signed by ISRG Root X1) in PEM format to /tmp/root.pem and /tmp/intermediate.pem on your local computer, respectively.

In your web browser, visit Let's Encrypt's certificate listing at https://letsencrypt.org/certificates/ . Make sure the web site is trusted by the browser. Download the certificates under Root Certificates > Active > ISRG Root X1 (self-signed) and Intermediate Certificates > Active > Let's Encrypt Authority X3 (Signed by ISRG Root X1) in PEM format to /tmp/root.pem and /tmp/intermediate.pem on your local computer, respectively.

After the download has succeeded, concatenate the files into a certificate chain:

cat /tmp/intermediate.pem /tmp/root.pem >/tmp/letsencrypt-chain.pem
cat /tmp/intermediate.pem /tmp/root.pem >/tmp/letsencrypt-chain.pem

scp /tmp/letsencrypt-chain.pem package-client:/tmp/.
scp /tmp/letsencrypt-chain.pem package-client :/tmp/.

Back in the shell of the package client, you now need to specify this chain of trust in the package manager configuration /usr/local/etc/pkg.conf so it gets used for TLS verification. Add these lines using an editor, and create the file if it does not exist yet:

Back in the shell of the package client , you now need to specify this chain of trust in the package manager configuration /usr/local/etc/pkg.conf so it gets used for TLS verification. Add these lines using an editor, and create the file if it does not exist yet:

sudo ee /usr/local/etc/pkg.conf
sudo ee /usr/local/etc/pkg.conf

/usr/local/etc/pkg.conf (snippet)

pkg_env: {
    SSL_CA_CERT_FILE: "/usr/local/etc/pkg/repos/letsencrypt-chain.pem",
}

Move the CA chain into place:

sudo mv /tmp/letsencrypt-chain.pem /usr/local/etc/pkg/repos/.
sudo mv /tmp/letsencrypt-chain.pem /usr/local/etc/pkg/repos/.

If you stayed in a root shell until now because the sudo package was removed, this command must be run without sudo. The same applies to the next command within this note.

If you stayed in a root shell until now because the sudo package was removed, this command must be run without sudo . The same applies to the next command within this note.

With this setting, you can try bootstrapping once again and should not get any more TLS errors. There is one small twist: the FreeBSD built-in /usr/sbin/pkg, which bootstraps the full package manager, does not honor the configured pkg_env setting, so we have to override the respective environment variable for this one time only, using the same value as configured:

With this setting, you can try bootstrapping once again and should not get any more TLS errors. There is one small twist: the FreeBSD built-in /usr/sbin/pkg , which bootstraps the full package manager, does not honor the configured pkg_env setting, so we have to override the respective environment variable for this one time only, using the same value as configured:

sudo env SSL_CA_CERT_FILE=/usr/local/etc/pkg/repos/letsencrypt-chain.pem pkg bootstrap
sudo env SSL_CA_CERT_FILE=/usr/local/etc/pkg/repos/letsencrypt-chain.pem pkg bootstrap

If you previously deleted existing packages, it’s a good time to reinstall essential tools now (e.g. sudo), plus any other desired packages.

If you previously deleted existing packages, it's a good time to reinstall essential tools now (eg sudo), plus any other desired packages.

pkg install bash sudo
pkg install bash sudo

And drop out of the root shell, if that is still the case:

exit
出口

In order to test whether everything works, install packages from the list specified in the Buildbot master config (variable PORTS_TO_BUILD). For example, the Bash shell and sudo:

In order to test whether everything works, install packages from the list specified in the Buildbot master config (variable PORTS_TO_BUILD ). For example, the Bash shell and sudo:

sudo pkg install bash sudo tmux
sudo pkg install bash sudo tmux

Again, confirm installation by pressing y and then ENTER. The package installation should run through without any issues.

Again, confirm installation by pressing y and then ENTER . The package installation should run through without any issues.

You can use pkg info to list which packages are currently installed (including dependencies, if any). To verify that no packages from other sources are installed, possibly causing clashes or incompatibilities, you could list installed packages with these details using pkg query "%n: autoinstalled=%a from repo=%R". Mind that pkg will be shown as bootstrapped from unknown-repository—this is why previously, you verified the bootstrapping output to see that the package manager itself is also taken from your own package repository.

You can use pkg info to list which packages are currently installed (including dependencies, if any). To verify that no packages from other sources are installed, possibly causing clashes or incompatibilities, you could list installed packages with these details using pkg query "%n: autoinstalled=%a from repo=%R" . Mind that pkg will be shown as bootstrapped from unknown-repository —this is why previously, you verified the bootstrapping output to see that the package manager itself is also taken from your own package repository.

In this last step, you configured access to the CI’s package repository on a client, optionally enabled package signature verification for security purposes, ensured that packages only come from a single source to avoid compatibility issues, bootstrapped the package manager pkg, and installed your desired packages as built by the CI.

In this last step, you configured access to the CI's package repository on a client, optionally enabled package signature verification for security purposes, ensured that packages only come from a single source to avoid compatibility issues, bootstrapped the package manager pkg , and installed your desired packages as built by the CI.

结论 (Conclusion)

In this tutorial, you have installed and configured Poudriere, automated running package builds, and configured secure access to the package repository from a client host, ending up with the latest built packages installed from a single, central source. The setup puts you in an excellent position to keep your servers consistent and up-to-date, and manage version upgrades of external software packages.

To further enhance your current setup, you could consider select follow-up steps:

Private access only: By default, Droplets have a public IP address on the internet. Also, Buildbot supports authentication but is by default unprotected.
Private access only : By default, Droplets have a public IP address on the internet. Also, Buildbot supports authentication but is by default unprotected.
Alert on build problems: Check out how to set up Buildbot reporters to get started.
Alert on build problems : Check out how to set up Buildbot reporters to get started.
Keep ports tree up to date: In the examples from the tutorial, the quarterly branch 2019Q2 was used, but you should switch to a newer tree eventually or use your own version-controlled repository to apply desired patches.
Keep ports tree up to date : In the examples from the tutorial, the quarterly branch 2019Q2 was used, but you should switch to a newer tree eventually or use your own version-controlled repository to apply desired patches.
Adding builds for own projects: The FreeBSD Porter’s Handbook explains how to write a build recipe (a port) if you want to build and install your internal software as FreeBSD packages.
Adding builds for own projects : The FreeBSD Porter's Handbook explains how to write a build recipe (a port ) if you want to build and install your internal software as FreeBSD packages.
Monitor outdated packages on clients: You can compare installed packages on a client with the latest available packages on the CI using the output of sudo pkg update -q && sudo pkg version -q --not-like "=" which prints all packages whose version does not exactly match. See the manpage of pkg-version for more details.
Monitor outdated packages on clients : You can compare installed packages on a client with the latest available packages on the CI using the output of sudo pkg update -q && sudo pkg version -q --not-like "=" which prints all packages whose version does not exactly match. See the manpage of pkg-version for more details.
Add cleanup job: Over time, the Buildbot worker jail will run full of old build log files, source tarballs, and possibly deprecated packages. Use the commands poudriere {logclean,distclean,pkgclean} to clean up (see manpage of poudriere).
Add cleanup job : Over time, the Buildbot worker jail will run full of old build log files, source tarballs, and possibly deprecated packages. Use the commands poudriere {logclean,distclean,pkgclean} to clean up (see manpage of poudriere ).