ECK安装elasticsearch集群及es配置x-pack

目录

ECK安装elasticsearch集群

elasticsearch安装x-pack插件

Elasticsearch Springboot 开发

---

ECK安装elasticsearch集群

• ECK为elasticsearch官方提供的在k8s集群上安装es集群的方式
• 因为es集群为有状态，ECK进行了很好的封装
• 生产环境面临大的集群状态的话，尝试通过k8s进行es集群的维护会是以后的发展趋势

ECK安装的前提条件

• k8s集群
• 配置好pv，可以采用nfs和cephFS，pvc由ECK进行维护
• 配置好镜像仓库 harbor
• 将es的镜像从远程服务器下载下来，这样本地拉的时候会快很多

k8s集群部署NFS

k8s集群部署ECK

• 下载ECK定义的CRD资源和RCAB规则
  kubectl apply -f https://download.elastic.co/downloads/eck/1.4.0/all-in-one.yaml

• 查看日志
  kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

• 创建单个节点的es operator自动创建节点

  cat <<EOF | kubectl apply -f -
  apiVersion: elasticsearch.k8s.elastic.co/v1
  kind: Elasticsearch
  metadata:
  name: quickstart
  spec:
  version: 7.11.1
  nodeSets:

  • name: default
    count: 1
    config:
    node.store.allow_mmap: false
    EOF

(设置node.store.allow_mmap: false对性能有影响，应针对虚拟虚拟机一节中所述的生产工作负载进行调整。)

• 创建成功之后监控节点的健康状态
  kubectl get elasticsearch

  NAME HEALTH NODES VERSION PHASE AGE
  quickstart green 1 7.11.1 Ready 1m

• 查看pod
  kubectl get pods --selector='elasticsearch.k8s.elastic.co/cluster-name=quickstart'

• 访问es，ECK会给es创建cluster IP，ECK自动集成了X-PACK插件，分别通过kubectl get svc quickstart-es-http和PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')获取IP和密码，默认端口9200
  curl -u "elastic:$PASSWORD" -k "https://quickstart-es-http:9200"

• kubectl port-forward service/quickstart-es-http 9200
curl -u "elastic:$PASSWORD" -k "https://localhost:9200"
  得到结果

  “name” : “quickstart1-es-default-0”,
  “cluster_name” : “quickstart1”,
  “cluster_uuid” : “cqEk6G9dQHKWL4MidTQYAw”,
  “version” : {
  “number” : “7.11.1”,
  “build_flavor” : “default”,
  “build_type” : “docker”,
  “build_hash” : “ff17057114c2199c9c1bbecc727003a907c0db7a”,
  “build_date” : “2021-02-15T13:44:09.394032Z”,
  “build_snapshot” : false,
  “lucene_version” : “8.7.0”,
  “minimum_wire_compatibility_version” : “6.8.0”,
  “minimum_index_compatibility_version” : “6.0.0-beta1”
  },
  “tagline” : “You Know, for Search”
  }

elasticsearch安装x-pack插件

es6.8以上的版本默认集成了x-pack，但是需要配置开启

1.es证书生成

将证书文件生成到指定的目录下，这里指定为/etc/elasticsearch

cd $ES_HOME/bin
./elasticsearch-certutil ca -out /etc/elasticsearch/elastic-certificates.p12 -pass

./elasticsearch-certutil ca -out elastic-certificates.p12 -pass
此处操作为，在一台机器上生成，然后分发到集群中的其他机器上，文件要保持一致

2.配置elasticsearch.yml文件

集群中每台机器的配置文件都要添加配置

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/elastic-certificates.p12

3.重启es集群

重启es要用普通用户,加上-d命令在后台运行

su elastic
bin/elasticsearch -d

4.生成密码

执行下面的命令，会让输入各种密码

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords interactive

此处输入的账号和密码是elastic:OY3iPO2AK&gSz35D

6.验证

curl localhost:9200/_cat/indices?v -u "elastic:OY3iPO2AK&gSz35D"

可以查看到索引，即密码验证成功

6.坑

1.Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

原因是集群中elasticsearch证书不一致

2.ERROR: bootstrap checks failed memory locking requested for elasticsearch process but memory is not locked

在/etc/security/limits.conf中添加配置
* soft memlock unlimited 
* hard memlock unlimited

Elasticsearch Springboot 开发

1.配置文件application.properties

spring.data.elasticsearch.repositories.enabled=true
spring.elasticsearch.rest.uris=http://192.168.223.147:9200
spring.elasticsearch.rest.connection-timeout=30s

2.编写实体类entity
@Document
@Id
@Field

3.编写repository
继承ElasticsearchRepository

4.定义接口
Service

5.写接口实现方法

6.编写Controller