7、Elasticsearch7.3.2 单点备份和还原(reindex、curl)

系统版本：centos7
软件版本：Elasticsearch7.3.2
参考：官方文档

一、方法一（reindex）

1、修改配置文件，添加白名单

在新集群的 elasticsearch.yml 中添加

reindex.remote.whitelist: 旧集群ip:端口

2、查询旧索引并利用旧索引创建新索引

1、查询旧索引customer信息

[root@dev13 ~]# curl 'http://旧ip:9200/customer?pretty=true'
{
  "customer" : {
    "aliases" : {
      "aa" : { }
    },
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "long"
        },
        "name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1618887567162",
        "number_of_shards" : "1",
        "number_of_replicas" : "1",
        "uuid" : "CRUfn1wDQ-yZlYFPuxHKrQ",
        "version" : {
          "created" : "7030299"
        },
        "provided_name" : "customer"
      }
    }
  }
}
[root@dev13 ~]#

2、创建新索引customer1、分区、mapping，别名

[root@dev13 ~]# curl -H "Content-Type: application/json" -XPUT 'http://新ip:9200/customer1?pretty' -d '
{
    "aliases" : {
	  "aa" : { }
	},
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "text"
        },
        "name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "number_of_shards" : "3",
        "number_of_replicas" : "0"
      }
    }
  }'

3、同步索引数据

curl -H "Content-Type: application/json" -XPOST 'http://新ip:9200/_reindex?pretty' -d '
{
   "source": {
      "index": "master_data",
      "remote": {
        "host": "http://旧ip:9200"
    }
   },
   "dest": {
      "index": "master_data"
   }
}'

4、查询索引数据量

[root@test1 es]# curl 'http://192.168.17.13:9200/master_data/_count?pretty'
{
  "count" : 28427618,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  }
}
[root@test1 es]#

Elasticsearch提供了snapshot API 用作备份

============================================================

二、方法二（curl）

注意：一定要用elasticsearch用户执行如下命令
流程：修改配置文件——>创建备份仓库——>开始备份索引到创建的仓库中——>恢复数据（如果存在恢复的索引需要关闭）

1 修改配置文件：

• 修改Elasticsearch的配置文件 elasticsearch.yml 中增加设置：

vim /elasticsearch-7.3.2/config/elasticsearch.yml
#修改内容如下：
path.repo: /home/elasticsearch/elasticsearch-7.3.2/data/backup

• 查看索引情况

[elasticsearch@test1 config]$ curl -X GET 'http://IP地址:9200/_cat/indices?v'
health status index       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   aa          TdvCzAU_RkmVxTKo54YtcA   3   1          0            0       849b           849b
green  open   .security-7 VMChqkv8TTOeOctpoOttcw   1   0          6            0     19.8kb         19.8kb

2 创建和查看备份仓库目录

2.1 创建备份仓库目录（可设置多个）

# 不带用户名密码验证
curl -H "Content-Type: application/json" -X PUT http://IP地址:9200/_snapshot/aa -d'{
    "type": "fs", 
    "settings": {
        "location": "/home/elasticsearch/elasticsearch-7.3.2/data/backup/esbakup"
    }
}'
# 带用户名密码验证
curl -u elastic:dXPbCs2wvLzMV672z4BX -H "Content-Type: application/json"  -X PUT http://192.168.180.46:9200/_snapshot/aa -d'{
    "type": "fs", 
    "settings": {
        "location": "/home/elasticsearch/elasticsearch-7.3.2/snapshots/esbakup"
    }
}'

2.2 查看所有备份仓库目录

# 带用户名密码验证
[elasticsearch@test1 elasticsearch-7.3.2]$ curl -u elastic:dXPbCs2wvLzMV672z4BX  -X GET http://192.168.180.46:9200/_snapshot/_all?pretty
{
  "backup" : {
    "type" : "fs",
    "settings" : {
      "location" : "/home/elasticsearch/elasticsearch-7.3.2/snapshots/esbakup"
    }
  },
  "aa" : {
    "type" : "fs",
    "settings" : {
      "location" : "/home/elasticsearch/elasticsearch-7.3.2/snapshots/esbakup"
    }
  }
}

2.3 删除某个备份仓库目录

[elasticsearch@test3 elasticsearch-7.3.2]$ curl -X DELETE "http://192.168.180.46:9200/_snapshot/es_backup?pretty"
{
  "acknowledged" : true
}

3 备份

如果想等备份完成后返回结果，需要添加wait_for_completion=true 参数；备份的名称：snapshot_20240426）

3.1 备份所有索引

# 不带用户名密码验证
curl -H "Content-Type: application/json" -X PUT http://IP地址:9200/_snapshot/aa/snapshot_20240426?wait_for_completion=true
# 带用户名密码验证
curl -u elastic:dXPbCs2wvLzMV672z4BX -H "Content-Type: application/json" -X PUT http://192.168.180.46:9200/_snapshot/aa/snapshot_20240426?wait_for_completion=true

3.2 备份指定多个索引

curl -u elastic:dXPbCs2wvLzMV672z4BX -H "Content-Type: application/json" -XPUT 'http://192.168.180.46:9200/_snapshot/aa/idxaabb' -d '{ "indices": "aa,bb"}'

4 查看仓库目录aa的索引备份情况

• 查看aa仓库目录中的所有索引备份情况

# 不带用户名密码验证
[elasticsearch@test1 root]$ curl -X GET "http://IP地址:9200/_snapshot/aa/_all?pretty"

# 带用户名密码验证
curl -u elastic:dXPbCs2wvLzMV672z4BX -X GET "http://192.168.180.46:9200/_snapshot/aa/_all?pretty"

• 查看指定仓库aa中的指定索引idxaabb备份情况

[elasticsearch@test1 elasticsearch-7.3.2]$ curl -u elastic:dXPbCs2wvLzMV672z4BX -X GET "http://192.168.180.46:9200/_snapshot/aa/idxaabb?pretty"
{
  "snapshots" : [
    {
      "snapshot" : "idxaabb",
      "uuid" : "Glny0mziQWCQCrS090WRxw",
      "version_id" : 7030299,
      "version" : "7.3.2",
      "indices" : [
        "aa",
        "bb"
      ],
      "include_global_state" : true,
      "state" : "SUCCESS",
      "start_time" : "2024-04-29T06:00:30.358Z",
      "start_time_in_millis" : 1714370430358,
      "end_time" : "2024-04-29T06:00:30.446Z",
      "end_time_in_millis" : 1714370430446,
      "duration_in_millis" : 88,
      "failures" : [ ],
      "shards" : {
        "total" : 8,
        "failed" : 0,
        "successful" : 8
      }
    }
  ]
}

5 恢复索引

恢复的索引如果存在且打开的，需要关关闭，否则报错如下

[elasticsearch@test1 elasticsearch-7.3.2]$ curl -u elastic:dXPbCs2wvLzMV672z4BX -X POST http://192.168.180.46:9200/_snapshot/aa/snapshot_20240426/_restore?wait_for_completion=true
{"error":{"root_cause":[{"type":"snapshot_restore_exception","reason":"[aa:snapshot_20240426/HfwDwOqtTgq6oA-C0oQ_EA] cannot restore index [aa] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"}],"type":"snapshot_restore_exception","reason":"[aa:snapshot_20240426/HfwDwOqtTgq6oA-C0oQ_EA] cannot restore index [aa] because an open index with same name already exists in the cluster. Either close or delete the existing index or restore the index under a different name by providing a rename pattern and replacement name"},"status":500}

5.1 恢复所有索引

恢复数据=集群的快照ID 加上后面_restore即可，（?wait_for_completion=true回调提示是否成功）

# 不带用户名密码验证
curl -X POST http://IP地址:9200/_snapshot/aa/snapshot_20191028/_restore?wait_for_completion=true
# 带用户名密码验证
curl -u elastic:dXPbCs2wvLzMV672z4BX -X POST http://192.168.180.46:9200/_snapshot/aa/snapshot_20240426/_restore?wait_for_completion=true

运行结果如下：显示已经恢复了相应的索引数据

[elasticsearch@test1 elasticsearch-7.3.2]$ curl -u elastic:dXPbCs2wvLzMV672z4BX -X POST http://192.168.180.46:9200/_snapshot/aa/snapshot_20240426/_restore?wait_for_completion=true
{"snapshot":{"snapshot":"snapshot_20240426","indices":["aa",".security-7"],"shards":{"total":4,"failed":0,"successful":4}}}

5.2 恢复指定索引

# 带用户名密码验证
curl -u elastic:dXPbCs2wvLzMV672z4BX -H 'Content-Type: application/json' -XPOST http://192.168.180.46:9200/_snapshot/aa/idxaabb/_restore -d '{"indices": "aa,bb"}'

6 删除备份目录aa中某个备份

curl -u elastic:dXPbCs2wvLzMV672z4BX -X DELETE http://192.168.180.46:9200/_snapshot/aa/idxaabb

7 验证

[elasticsearch@test1 config]$ curl -X GET 'http://IP地址:9200/_cat/indices?v'
health status index       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   aa          TdvCzAU_RkmVxTKo54YtcA   3   1          0            0       849b           849b
green  open   .security-7 VMChqkv8TTOeOctpoOttcw   1   0          6            0     19.8kb         19.8kb

三、备份es数据到minio

1、 提前在在minio中添加备份elasticsearch所用到的账号和bucket

2、修改elasticsearch配置

#添加s3仓库插件
bin/elasticsearch-plugin install repository-s3

#修改配置文件添加endpoint
vi config/elasticsearch.yml
s3.client.default.endpoint: 192.168.181.118:29000
s3.client.default.protocol: http

#添加minio的认证信息，access_key为用户名，secret_key为密码
./bin/elasticsearch-keystore add s3.client.default.access_key
./bin/elasticsearch-keystore add s3.client.default.secret_key

#重新启动elasticsearch

3、创建备份仓库

curl -H 'Content-Type: application/json' -X PUT http://192.168.180.46:9200/_snapshot/aa -d '{ 
    "type":"s3", 
    "settings":{ 
        "bucket":"es-backup", "protocol":"http", "disable_chunked_encoding":"true", "endpoint":"192.168.181.118:29000" 
    } 
}'

剩下和 方法二（curl）剩余步骤没区别

============================================================