elastisearch安装和运维常用API

elasticsearch 是一个文档型数据库，在 k8s 运维中，通常和 kibana 和 fluentd 结合作为 k8s 上的日志数据库，本文将介绍 elasticsearch 的搭建和一些运维时常用的 api 。

搭建

elasticsearch 的最新版本官方提供的安装方式是使用 operator 安装，详细文档可以参考此链接，执行以下命令可以快速生成一个 elasticsearch。

kubectl apply -f https://openebs.github.io/charts/openebs-operator.yaml

kubectl create -f https://download.elastic.co/downloads/eck/2.13.0/crds.yaml


cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 8.14.3
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
EOF

在搭建完成后执行以下命令查看 elasticsearch 的状态信息、密码和 ip 信息

kubectl get elasticsearch

#NAME         HEALTH   NODES   VERSION   PHASE   AGE
#quickstart   green    1       8.14.3    Ready   3d21h

kubectl get service quickstart-es-http

#NAME                 TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
#quickstart-es-http   ClusterIP   10.233.5.110   <none>        9200/TCP   3d21h

PASSWORD=$(kubectl get secret quickstart-es-elastic-user -o go-template='{{.data.elastic | base64decode}}')
echo $PASSWORD

elasticsearch 运维时常用的接口

在搭建完成之后，使用下面的脚本，创建一个索引和 100 条数据，用于后面的测试：

#!/bin/bash

# Elasticsearch 服务器配置
USERNAME="elastic"
PASSWORD="$PASSWORD"
HOST="10.233.5.110"
PORT="9200"
INDEX="test"

# 创建索引
echo "Creating index ${INDEX}..."
curl -k -u ${USERNAME}:${PASSWORD} -X PUT "https://${HOST}:${PORT}/${INDEX}" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "field1": { "type": "text" },
      "field2": { "type": "keyword" },
      "field3": { "type": "date" }
    }
  }
}
'

echo "Index ${INDEX} created."

# 插入100条数据
echo "Inserting data into ${INDEX}..."
for i in {1..100}
do
  field1_value="value${i}"
  field2_value="keyword${i}"
  field3_value=$(date +%Y-%m-%dT%H:%M:%S)

  curl -k -u ${USERNAME}:${PASSWORD} -X POST "https://${HOST}:${PORT}/${INDEX}/_doc/${i}" -H 'Content-Type: application/json' -d'
  {
    "field1": "'"${field1_value}"'",
    "field2": "'"${field2_value}"'",
    "field3": "'"${field3_value}"'"
  }
  '
  echo "Inserted document ${i}"
done

echo "Data insertion complete."

在插入完成后，我们将测试一些运维时，常用的 api 接口。

查看集群状态

curl -u elastic:$PASSWORD -X GET "https://10.233.5.110:9200/_cluster/health" -k

#{"cluster_name":"quickstart","status":"yellow","timed_out":false,"number_of_nodes":1,"number_of_data_nodes":1,"active_primary_shards":3,"active_shards":3,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":3,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":50.0}

获取节点信息

curl -u elastic:$PASSWORD -X GET "https://10.233.5.110:9200/_cat/nodes?v" -k 

#ip           heap.percent ram.percent cpu load_1m load_5m load_15m node.role   master name
#10.233.90.57           37          85  41    3.10    2.94     2.78 cdfhilmrstw *      quickstart-es-default-0

查看所有索引

curl -u elastic:$PASSWORD -X GET "https://10.233.5.110:9200/_cat/indices?v" -k

#health status index uuid                   pri rep docs.count docs.deleted store.size pri.store.size dataset.size
#yellow open   test  u2XHCpISQaGl6JkmuGch8w   3   1        100            0    127.4kb        127.4kb      127.4kb

获取 index 映射信息

curl -u elastic:$PASSWORD -X GET "https://10.233.5.110:9200/test/_mapping" -k

#{"test":{"mappings":{"properties":{"field1":{"type":"text"},"field2":{"type":"keyword"},"field3":{"type":"date"}}}}}

获取 index 设置

curl -u elastic:$PASSWORD -X GET "https://10.233.5.110:9200/test/_settings"

#{"test":{"settings":{"index":{"routing":{"allocation":{"include":{"_tier_preference":"data_content"}}},"number_of_shards":"3","provided_name":"test","creation_date":"1721358001820","number_of_replicas":"1","uuid":"u2XHCpISQaGl6JkmuGch8w","version":{"created":"8505000"}}}}}

强制合并索引

curl -u elastic:$PASSWORD -X POST "https://10.233.5.110:9200/test/_forcemerge?max_num_segments=1" -k

#{"_shards":{"total":6,"successful":3,"failed":0}}

刷新索引

curl -u elastic:$PASSWORD -X POST "https://10.233.5.110:9200/test/_refresh" -k

#{"_shards":{"total":6,"successful":3,"failed":0}}

清理缓存

curl -u elastic:$PASSWORD -X POST "https://10.233.5.110:9200/test/_cache/clear" -k

#{"_shards":{"total":6,"successful":3,"failed":0}}

删除索引

curl -u elastic:$PASSWORD -X DELETE "https://10.233.5.110:9200/test" -k

#{"acknowledged":true}

可能对于运维来说，操作 elasticserach 主要使用查询操作来检查集群和索引当前的一些情况，如果集群的索引使用导致内存过大就需要通过强制合并索引、刷新索引和清理缓存这三个接口来释放 elasticserach 相关的资源，最坏的情况下可能需要删除掉索引。