Observability：在生产环境中配置 Fleet Server 和 Elastic Agent 之间的安全

Elastic 中国社区官方博客

已于 2022-05-01 15:01:12 修改

阅读量5.7k

点赞数 4

于 2022-05-01 10:10:07 首次发布

本文为博主原创文章，未经博主允许不得转载。

本文链接：https://blog.csdn.net/UbuntuTouch/article/details/124522674

版权

在我之前的文章 “Observability：使用 Elastic Agent 来摄入日志及指标 - Elastic Stack 8.0”，我详述了如何部署 Elastic Agent 来收集系统日志及指标。在那篇文章的配置中，我选择了 “Quick Start" 来配置 Fleet server 和 Elastic Agent 之前的连接。在实际的生产环境部署中，我们也应该配置为安全的连接。以 Elastic Agent 为基础的数据摄入架构图如下：

在我们之前的配置中，我们保证所有的连接到 Elasticsearch 都是 https 的连接，但是并没有配置从 Elastic Agent 到 Fleet server 之间的安全连接。在今天的教程中，我将详述如何配置这个安全的连接。

在生产环境中配置 Fleet Server 和 Elastic Agent 之间的安全

在生产环境中配置 Fleet Server 和 Elastic Agent 之间的安全_哔哩哔哩_bilibili

在今天的练习中，我将使用 Elastic Stack 8.1.2 来进行展示。我使用的测试环境如下：

前提条件

我们参考我之前的文章 “Elastic Stack 8.0 安装 - 保护你的 Elastic Stack 现在比以往任何时候都简单”。我们按照里面的步骤按照好 Elasticsearch 及 Kibana。Elasticsearch 的访问带有 https 安全配置。

为了能够让 fleet 正常工作，内置的 API service 必须启动。我们必须为 Elasticsearch 的配置文件 config/elasticsearch.yml 文件配置：

xpack.security.authc.api_key.enabled: true

配置完后，我们再重新启动 Elasticsearch。针对 Kibana，我们也需要做一个额外的配置。我们需要修改 config/kibana.yml 文件。在这个文件的最后面，添加如下的一行：


xpack.encryptedSavedObjects.encryptionKey: 'fhjskloppd678ehkdfdlliverpoolfcr'

如果你不想使用上面的这个设置，你可以使用如下的方式来获得：

./bin/kibana-encryption-keys generate

从上面的输出中，我们可以看出来，有三个输出的 key。我们可以把这三个同时拷贝，并添加到 config/kibana.yml 文件的后面。当然，我们也可以只拷贝其中的一个也可。我们再重新启动 Kibana。

这样我们对 Elasticsearch 及 Kibana 的配置就完成。针对 Elastic Stack 8.0 以前的版本安装，请阅读我之前的文章 “Observability：如何在最新的 Elastic Stack 中使用 Fleet 摄入 system 日志及指标”。

除此之外，Kibana 需要 Internet 连接才能从 Elastic Package Registry 下载集成包。确保 Kibana 服务器可以连接到https://epr.elastic.co 的端口 443 上。如果你的环境有网络流量限制，有一些方法可以解决此要求。有关详细信息，请参阅气隙环境。

目前，Fleet 只能被具有 superuser role 的用户所使用。

配置 Fleet

使用 Kibana 中的 Fleet 将日志、指标和安全数据导入 Elastic Stack。第一次使用 Fleet 时，你可能需要对其进行设置并添加 Fleet Server。

我们点击上面的链接去下载适合自己平台的 Elastic Agent：

在 Ubuntu OS 机器的 terminal 中，我下载 LINUX 64-BIT：

wget https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.1.2-linux-x86_64.tar.gz

我们可以使用如下的命令连进行接压缩：

tar xzf elastic-agent-8.1.2-linux-x86_64.tar.gz

然后，我们进入到安装目录：

liuxg@liuxgu:~/fleet$ pwd
/home/liuxg/fleet
liuxg@liuxgu:~/fleet$ cd elastic-agent-8.1.2-linux-x86_64/

我们再次回到 Fleet 的安装页面：

由于我们是为生产环境进行配置，为了确保 fleet server 和 elastic agent 之间的安全连接，我们必须为它生成证书。

为此，我们在 macOS 的新的 terminal 中打入如下的命令：

$ pwd
/Users/liuxg/test/elasticsearch-8.1.2
$ ./bin/elasticsearch-certutil ca --pem
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'ca' mode generates a new 'certificate authority'
This will create a new X.509 certificate and private key that can be used
to sign certificate when running in 'cert' mode.

Use the 'ca-dn' option if you wish to configure the 'distinguished name'
of the certificate authority

By default the 'ca' mode produces a single PKCS#12 output file which holds:
    * The CA certificate
    * The CA's private key

If you elect to generate PEM format certificates (the -pem option), then the output will
be a zip file containing individual files for the CA certificate and private key

Please enter the desired output file [elastic-stack-ca.zip]: 
$ ls 
LICENSE.txt          bin                  elastic-stack-ca.zip logs
NOTICE.txt           config               jdk.app              modules
README.asciidoc      data                 lib                  plugins
$ unzip elastic-stack-ca.zip 
Archive:  elastic-stack-ca.zip
   creating: ca/
  inflating: ca/ca.crt               
  inflating: ca/ca.key

如上所示，我们生成了一个叫做 elastic-stack-ca.zip 文件。我们通过 unzip 命令来解压缩。我们可以清楚地看到文件 elastic-stack-ca.zip 中含有 ca.crt 及 ca.key CA 证书信息。

接下来，我们使用 CA 来为 Fleet Server 生成证书：

./bin/elasticsearch-certutil cert \
  --name fleet-server \
  --ca-cert /path/to/ca/ca.crt \
  --ca-key /path/to/ca/ca.key \
  --dns your.host.name.here \
  --ip 192.0.2.1 \
  --pem

针对我的情况：

  ./bin/elasticsearch-certutil cert \
  --name fleet-server \
  --ca-cert ca/ca.crt \
  --ca-key ca/ca.key \
  --dns liuxgu \
  --ip 192.168.0.4 \
  --pem

$   ./bin/elasticsearch-certutil cert \
>   --name fleet-server \
>   --ca-cert ca/ca.crt \
>   --ca-key ca/ca.key \
>   --dns liuxgu \
>   --ip 192.168.0.4 \
>   --pem
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

The 'cert' mode generates X.509 certificate and private keys.
    * By default, this generates a single certificate and key for use
       on a single instance.
    * The '-multiple' option will prompt you to enter details for multiple
       instances and will generate a certificate and key for each one
    * The '-in' option allows for the certificate generation to be automated by describing
       the details of each instance in a YAML file

    * An instance is any piece of the Elastic Stack that requires an SSL certificate.
      Depending on your configuration, Elasticsearch, Logstash, Kibana, and Beats
      may all require a certificate and private key.
    * The minimum required value for each instance is a name. This can simply be the
      hostname, which will be used as the Common Name of the certificate. A full
      distinguished name may also be used.
    * A filename value may be required for each instance. This is necessary when the
      name would result in an invalid file or directory name. The name provided here
      is used as the directory name (within the zip) and the prefix for the key and
      certificate files. The filename is required if you are prompted and the name
      is not displayed in the prompt.
    * IP addresses and DNS names are optional. Multiple values can be specified as a
      comma separated string. If no IP addresses or DNS names are provided, you may
      disable hostname verification in your SSL configuration.


    * All certificates generated by this tool will be signed by a certificate authority (CA)
      unless the --self-signed command line option is specified.
      The tool can automatically generate a new CA for you, or you can provide your own with
      the --ca or --ca-cert command line options.


By default the 'cert' mode produces a single PKCS#12 output file which holds:
    * The instance certificate
    * The private key for the instance certificate
    * The CA certificate

If you specify any of the following options:
    * -pem (PEM formatted output)
    * -multiple (generate multiple certificates)
    * -in (generate certificates from an input file)
then the output will be be a zip file containing individual certificate/key files

Please enter the desired output file [certificate-bundle.zip]: 

Certificates written to /Users/liuxg/test/elasticsearch-8.1.2/certificate-bundle.zip

This file should be properly secured as it contains the private key for 
your instance.
After unzipping the file, there will be a directory for each instance.
Each instance has a certificate and private key.
For each Elastic product that you wish to configure, you should copy
the certificate, key, and CA certificate to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

上述命令将生成一个叫做 certificate-bundle.zip 的文件。

接下来，我们把 ca.crt 及 certificate-bundle.zip 拷贝到 Ubuntu OS 机器上的 elastic agent 安装目录中。

liuxg@liuxgu:~/fleet/elastic-agent-8.1.2-linux-x86_64$ pwd
/home/liuxg/fleet/elastic-agent-8.1.2-linux-x86_64
liuxg@liuxgu:~/fleet/elastic-agent-8.1.2-linux-x86_64$ ls
LICENSE.txt  ca.crt                  elastic-agent                fleet.yml       output.pem
NOTICE.txt   certificate-bundle.zip  elastic-agent.reference.yml  fleet.yml.lock
README.md    data                    elastic-agent.yml            http_ca.crt

如上所示，在 Ubuntu 机器上，我们可以看到 ca.crt 文件及 certificate-bundle.zip。我们通过如下的命令来解压缩 certificate-bundle.zip 文件：

liuxg@liuxgu:~/fleet/elastic-agent-8.1.2-linux-x86_64$ unzip certificate-bundle.zip 
Archive:  certificate-bundle.zip
   creating: fleet-server/
  inflating: fleet-server/fleet-server.crt  
  inflating: fleet-server/fleet-server.key

我们可以看到有 fleet-server 目录被生成。里面含有所有的证书信息。

我们接下来拷贝下来 Kibana 中的安装信息，并做相应的修改：

sudo ./elastic-agent install --url=https://192.168.0.4:8220 \
  --fleet-server-es=https://192.168.0.3:9200 \
  --fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NTEzNjk3NTIxNzQ6aXZVUHltTXhTZXlWaDNzd1hGVGlCZw \
  --fleet-server-policy=fleet-server-policy \
  --fleet-server-es-ca-trusted-fingerprint=0eee78cb2faee1556848acd6a9d3b3c828bd57d1eeb442834d047110a75c975e \
  --certificate-authorities=/home/liuxg/fleet/elastic-agent-8.1.2-linux-x86_64/ca.crt \
  --fleet-server-cert=/home/liuxg/fleet/elastic-agent-8.1.2-linux-x86_64/fleet-server/fleet-server.crt \
  --fleet-server-cert-key=/home/liuxg/fleet/elastic-agent-8.1.2-linux-x86_64/fleet-server/fleet-server.key

我们把相应的路径添加上，并在安装目录下运行：

liuxg@liuxgu:~/fleet/elastic-agent-8.1.2-linux-x86_64$ sudo ./elastic-agent install --url=https://192.168.0.4:8220 \
>   --fleet-server-es=https://192.168.0.3:9200 \
>   --fleet-server-service-token=AAEAAWVsYXN0aWMvZmxlZXQtc2VydmVyL3Rva2VuLTE2NTEzNjk3NTIxNzQ6aXZVUHltTXhTZXlWaDNzd1hGVGlCZw \
>   --fleet-server-policy=fleet-server-policy \
>   --fleet-server-es-ca-trusted-fingerprint=0eee78cb2faee1556848acd6a9d3b3c828bd57d1eeb442834d047110a75c975e \
>   --certificate-authorities=/home/liuxg/fleet/elastic-agent-8.1.2-linux-x86_64/ca.crt \
>   --fleet-server-cert=/home/liuxg/fleet/elastic-agent-8.1.2-linux-x86_64/fleet-server/fleet-server.crt \
>   --fleet-server-cert-key=/home/liuxg/fleet/elastic-agent-8.1.2-linux-x86_64/fleet-server/fleet-server.key
Elastic Agent will be installed at /opt/Elastic/Agent and will run as a service. Do you want to continue? [Y/n]:
{"log.level":"info","@timestamp":"2022-05-01T10:06:57.258+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":777},"message":"Fleet Server - Starting","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-01T10:06:59.261+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":758},"message":"Fleet Server - Running on policy with Fleet Server integration: fleet-server-policy; missing config fleet.agent.id (expected during bootstrap process)","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-01T10:06:59.971+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":455},"message":"Starting enrollment to URL: https://192.168.0.4:8220/","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2022-05-01T10:07:03.018+0800","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":255},"message":"Successfully triggered restart on running Elastic Agent.","ecs.version":"1.6.0"}
Successfully enrolled the Elastic Agent.
Elastic Agent has been successfully installed.

从上面，我们可以看出来，安装是成功的。我们再次回到 Kibana 的界面：