Http客户端不信任此服务器的证书关闭连接 – 如何解决此Elasticsearch错误

英文报错信息如下:

[2024-06-19T14:34:53,327][WARN ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node-68] http client did not trust this server's certificate, closing connection Netty4HttpChannel{localAddress=/39.134.86.100:9200, remoteAddress=/39.134.86.100:50320

简而言之，当 Elasticsearch HTTP 客户端无法验证其尝试连接到的服务器的 SSL/TLS 证书时，会发生此错误。这可能是由于证书过期、自签名证书或来自不受信任源的证书所致。要解决此问题，您可以使用受信任的证书颁发机构提供的有效证书更新证书，也可以将 Elasticsearch 客户端配置为信任服务器的证书。如果是自签名证书，则需要将其添加到客户端的信任库中。始终确保您的证书是最新的，以避免此类问题。

本指南将帮助您检查导致日志“Http client did not trust this server's certificate;关闭连接 {} “ 以显示。要了解与此日志相关的问题，请阅读以下有关以下 Elasticsearch 概念的说明：客户端和插件。

客户

概述

任何与 Elasticsearch 交互以索引、更新或搜索数据，或使用各种 API 监控和维护 Elasticsearch 的应用程序都可以被视为客户端

正确配置客户端非常重要，以确保 Elasticsearch 资源的最佳使用。

例子

有许多用于监控、警报和可视化的开源客户端应用程序，例如 ElasticHQ、Elastalert 和 Grafana 等。在 filebeat、metricbeat、logstash 和 kibana 等 Elastic 客户端应用程序之上，这些应用程序都旨在与 Elasticsearch 集成。

但是，通常需要创建自己的客户端应用程序才能与 Elasticsearch 交互。下面是 python 客户端的简单示例（摘自客户端文档）：

从 datetime 导入 datetime

从 elasticsearch 导入 Elasticsearch

es = 弹性搜索()

doc ={

'author'： '测试'，

'text'： 'Elasticsearch：很酷。 盆景很酷。，

“timestamp”：日期时间。现在（），

}

res = es。index（index=“test-index”， doc_type='tweet'， id=1， body=doc)

print（res['结果'])

res = es。get（index=“测试索引”， doc_type='推文'， id=1)

打印（res['_source'])

es。索引。refresh（index=“测试索引”)

res = es。search（index=“test-index”， body={“query”： {“match_all”： {}}})

print（“Got %d Hits：” % res['hits']['total']['value'])

for hit in res['hits']['hits']：

print（“%（timestamp）s %（作者）s： %（text）s” % hit[“_source”])

所有官方的 Elasticsearch 客户端都遵循类似的结构，作为 Elasticsearch rest API 的轻量级包装器，因此如果您熟悉 Elasticsearch 查询结构，它们通常很容易实现。

注意事项和须知事项

使用官方 Elasticsearch 库。

虽然可以使用任何 HTTP 方法（例如 curl 请求）与 Elasticsearch 连接，但官方的 Elasticsearch 库已被设计为正确实现连接池和保持活动状态。

官方 Elasticsearch 客户端可用于 java、javascript、Perl、PHP、python、ruby 和 .NET。社区版本支持许多其他编程语言。

使您的 Elasticsearch 版本和客户端版本保持同步。

为避免出现意外，请始终使客户端版本与您正在使用的 Elasticsearch 版本保持一致。始终使用 Elasticsearch 测试客户端，因为即使是较小的版本升级也可能因依赖关系或需要更改代码而导致问题。

在适当的节点之间进行负载平衡。

确保客户端在群集中的所有相应节点之间正确负载均衡。在小型集群中，这通常意味着仅跨数据节点（从不跨主节点），或者在大型集群中，所有专用协调节点（如果已实现）。

确保 Elasticsearch 应用程序正确处理异常。

在 Elasticsearch 无法处理大量请求的情况下，设计一个客户端应用程序来优雅地处理这个问题（例如通过某种排队机制）将比简单地用重复请求淹没一个陷入困境的集群要好。

 

 

插件

概述

插件用于增强 Elasticsearch 的核心功能。Elasticsearch 提供了一些核心插件作为其发布安装的一部分。除了这些核心插件之外，还可以编写自己的自定义插件。GitHub 上有几个社区插件可用于各种用例。

例子

获取插件的所有说明：

sudo bin/elasticsearch-plugin -h

安装用于在 S3 上存储 Elasticsearch 快照的 S3 插件：

sudo bin/elasticsearch-plugin install repository-s3

删除插件：

sudo bin/elasticsearch-plugin 删除存储库-s3

使用文件的路径安装插件：

sudo bin/elasticsearch-plugin 安装文件：path/to/plugin.zip

注意事项和要知道的好事

• 插件使用 elasticsearch-plugin 脚本进行安装和删除，该脚本作为 Elasticsearch 安装的一部分提供，可以在 Elasticsearch 安装路径的 bin/ 目录中找到。
• 必须在集群的每个节点上安装一个插件，并且必须重新启动每个节点才能使插件可见。
• 您也可以手动下载插件，然后使用 elasticsearch-plugin install 命令进行安装，并提供插件源文件的文件名/路径。
• 删除插件后，您需要重新启动每个 Elasticsearch 节点才能完成删除过程。

常见问题

• 在插件安装期间和之后管理权限问题是最常见的问题。如果 Elasticsearch 是使用 DEB 或 RPM 软件包安装的，则必须使用 root 用户安装插件。否则，您可以以拥有所有 Elasticsearch 文件的用户身份安装插件。
• 在安装 DEB 或 RPM 包的情况下，安装后检查插件目录的权限很重要。如果权限已使用以下命令进行修改，则可以更新该权限：

chown -R elasticsearch：elasticsearch path_to_plugin_directory

• 如果您的 Elasticsearch 节点在私有子网中运行，无法访问 Internet，则无法直接安装插件。在这种情况下，您只需下载插件并将文件复制到每个节点上 Elasticsearch 安装路径的插件目录中即可。在这种情况下，节点也必须重新启动。

日志上下文

日志“Http 客户端不信任此服务器的证书;关闭连接 {}“ 类名为 SecurityNetty4HttpServerTransport.java。
我们从 Elasticsearch 源代码中提取了以下内容，供那些寻求深入上下文的人使用：

} else if （isReceivedCertificateUnknownException（原因)) {

如果 （logger.isTraceEnabled()) {

记录。trace（new ParameterizedMessage（“http client 不信任服务器的证书;关闭连接 {}”;

CTX的。channel（））; 原因);

} 还 {

记录。warn（“HTTP 客户端不信任此服务器的证书;关闭连接 {}“; ctx。渠道());

}

CTX的。channel（） 中。关闭();

} 还 {

超。exceptionCaught（ctx; 原因);

}

原文如下

Http client did not trust this server s certificate closing connection – How to solve this Elasticsearch error

Opster Team

Aug-23, Version: 6.8-6.8

Briefly, this error occurs when the Elasticsearch HTTP client cannot verify the SSL/TLS certificate of the server it’s trying to connect to. This could be due to an expired certificate, a self-signed certificate, or a certificate from an untrusted source. To resolve this issue, you can either update the certificate with a valid one from a trusted Certificate Authority, or configure the Elasticsearch client to trust the server’s certificate. If it’s a self-signed certificate, you need to add it to the client’s truststore. Always ensure your certificates are up-to-date to avoid such issues.

This guide will help you check for common problems that cause the log ” Http client did not trust this server’s certificate; closing connection {} ” to appear. To understand the issues related to this log, read the explanation below about the following Elasticsearch concepts: client and plugin.

Client

Overview

Any application that interfaces with Elasticsearch to index, update or search data, or to monitor and maintain Elasticsearch using various APIs can be considered a client

It is very important to configure clients properly in order to ensure optimum use of Elasticsearch resources.

Examples

There are many open-source client applications for monitoring, alerting and visualization, such as ElasticHQ, Elastalerts, and Grafana to name a few. On top of Elastic client applications such as filebeat, metricbeat, logstash and kibana that have all been designed to integrate with Elasticsearch.

However it is frequently necessary to create your own client application to interface with Elasticsearch. Below is a simple example of the python client (taken from the client documentation):

from datetime import datetime

from elasticsearch import Elasticsearch

es = Elasticsearch()

doc = {

'author': 'Testing',

'text': 'Elasticsearch: cool. bonsai cool.',

'timestamp': datetime.now(),

}

res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)

print(res['result'])

res = es.get(index="test-index", doc_type='tweet', id=1)

print(res['_source'])

es.indices.refresh(index="test-index")

res = es.search(index="test-index", body={"query": {"match_all": {}}})

print("Got %d Hits:" % res['hits']['total']['value'])

for hit in res['hits']['hits']:

print("%(timestamp)s %(author)s: %(text)s" % hit["_source"])

All of the official Elasticsearch clients follow a similar structure, working as light wrappers around the Elasticsearch rest API, so if you are familiar with Elasticsearch query structure they are usually quite straightforward to implement.

Notes and Good Things to Know

Use official Elasticsearch libraries.

Although it is possible to connect with Elasticsearch using any HTTP method, such as a curl request, the official Elasticsearch libraries have been designed to properly implement connection pooling and keep-alives.   

Official Elasticsearch clients are available for java, javascript, Perl, PHP, python, ruby and .NET. Many other programming languages are supported by community versions.

Keep your Elasticsearch version and client versions in sync.

To avoid surprises, always keep your client versions in line with the Elasticsearch version you are using. Always test clients with Elasticsearch since even minor version upgrades can cause issues due to dependencies or a need for code changes. 

Load balance across appropriate nodes.

Make sure that the client properly load balances across all of the appropriate nodes in the cluster.  In small clusters this will normally mean only across data nodes (never master nodes), or in larger clusters, all dedicated coordinating nodes (if implemented) .

Ensure that the Elasticsearch application properly handles exceptions.

In the case of Elasticsearch being unable to cope with the volume of requests, designing a client application to handle this gracefully (such as through some sort of queueing mechanism) will be better than simply inundating a struggling cluster with repeated requests.

Plugin

Overview

A plugin is used to enhance the core functionalities of Elasticsearch. Elasticsearch provides some core plugins as a part of their release installation. In addition to those core plugins, it is possible to write your own custom plugins as well. There are several community plugins available on GitHub for various use cases.

Examples

Get all of the instructions for the plugin:

sudo bin/elasticsearch-plugin -h

Installing the S3 plugin for storing Elasticsearch snapshots on S3:

sudo bin/elasticsearch-plugin install repository-s3

Removing a plugin:

sudo bin/elasticsearch-plugin remove repository-s3

Installing a plugin using the file’s path:

sudo bin/elasticsearch-plugin install file:///path/to/plugin.zip

Notes and good things to know

• Plugins are installed and removed using the elasticsearch-plugin script, which ships as a part of the Elasticsearch installation and can be found inside the bin/ directory of the Elasticsearch installation path.
• A plugin has to be installed on every node of the cluster and each of the nodes has to be restarted to make the plugin visible.
• You can also download the plugin manually and then install it using the elasticsearch-plugin install command, providing the file name/path of the plugin’s source file.
• When a plugin is removed, you will need to restart every Elasticsearch node in order to complete the removal process.

Common issues

• Managing permission issues during and after plugin installation is the most common problem. If Elasticsearch was installed using the DEB or RPM packages then the plugin has to be installed using the root user. Otherwise you can install the plugin as the user that owns all of the Elasticsearch files.
• In the case of DEB or RPM package installation, it is important to check the permissions of the plugins directory after you install it. You can update the permission if it has been modified using the following command:

chown -R elasticsearch:elasticsearch path_to_plugin_directory

• If your Elasticsearch nodes are running in a private subnet without internet access, you cannot install a plugin directly. In this case, you can simply download the plugins and copy the files inside the plugins directory of the Elasticsearch installation path on every node. The node has to be restarted in this case as well.

Log Context

Log “Http client did not trust this server’s certificate; closing connection {}” classname is SecurityNetty4HttpServerTransport.java.
We extracted the following from Elasticsearch source code for those seeking an in-depth context :

} else if (isReceivedCertificateUnknownException(cause)) {

if (logger.isTraceEnabled()) {

logger.trace(new ParameterizedMessage("http client did not trust server's certificate; closing connection {}";

ctx.channel()); cause);

} else {

logger.warn("http client did not trust this server's certificate; closing connection {}"; ctx.channel());

}

ctx.channel().close();

} else {

super.exceptionCaught(ctx; cause);

}