ElasticSearch索引数据迁移ElasticSearch-Dump方式

1、安装环境

CentOS Linux release 7.5.1804 (Core)

2、安装nodejs

yum install -y nodejs

3、验证nodejs

[root@localhost yum.repos.d]# node -v
v8.12.0
[root@localhost yum.repos.d]# npm -v
6.4.1

4、安装elasticsearch-dump

npm install elasticdump --registry https://registry.npm.taobao.org

5、验证安装

进入elasticdump/bin目录下

cd node_modules/elasticdump/bin/

执行

 ./elasticdump

出现

Fri, 19 Oct 2018 07:03:15 GMT | Error Emitted => {"errors":["`input` is a required input","`output` is a required input"]}

报错是因为没有输入参数

6、使用dump迁移索引

拷贝索引

elasticdump 
	--input=http://production.es.com:9200/my_index 
	--output=http://staging.es.com:9200/my_index 
	--type=mapping

拷贝数据

elasticdump 
	--input=http://production.es.com:9200/my_index 
	--output=http://staging.es.com:9200/my_index 
	--type=data

拷贝所有索引

elasticdump  
	--input=http://production.es.com:9200/ 
	--output=http://staging.es.com:9200/ 
	--all=true  

7、迁移实战

为了方便操作写了一个迁移脚本，仅供参考。

#!/bin/bash
echo -n "源ES地址: "
read old
echo -n "目标ES地址: "
read new
echo -n "源索引名: "
read old_index
echo -n "目标索引名: "
read new_index
cd /root/node_modules/elasticdump/bin/
 ./elasticdump --input=$old/$old_index  --output=$new/$new_index  --type=mapping &>> /root/dump.log
 ./elasticdump --input=$old/$old_index  --output=$new/$new_index  --type=data &>> /root/dump.log

建议：有内网的话尽量使用内网IP

八、详细参数

# Copy an index from production to staging with analyzer and mapping:
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=http://staging.es.com:9200/my_index \
  --type=analyzer
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=http://staging.es.com:9200/my_index \
  --type=mapping
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=http://staging.es.com:9200/my_index \
  --type=data

# Backup index data to a file:
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=/data/my_index_mapping.json \
  --type=mapping
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=/data/my_index.json \
  --type=data

# Backup and index to a gzip using stdout:
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=$ \
  | gzip > /data/my_index.json.gz

# Backup the results of a query to a file
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=query.json \
  --searchBody='{"query":{"term":{"username": "admin"}}}'

# Copy a single shard data:
elasticdump \
  --input=http://es.com:9200/api \
  --output=http://es.com:9200/api2 \
  --params='{"preference" : "_shards:0"}'

# Backup aliases to a file
elasticdump \
  --input=http://es.com:9200/index-name/alias-filter \
  --output=alias.json \
  --type=alias

# Import aliases into ES
elasticdump \
  --input=./alias.json \
  --output=http://es.com:9200 \
  --type=alias

# Backup templates to a file
elasticdump \
  --input=http://es.com:9200/template-filter \
  --output=templates.json \
  --type=template

# Import templates into ES
elasticdump \
  --input=./templates.json \
  --output=http://es.com:9200 \
  --type=template

# Split files into multiple parts
elasticdump \
  --input=http://production.es.com:9200/my_index \
  --output=/data/my_index.json \
  --fileSize=10mb

# Import data from S3 into ES (using s3urls)
elasticdump \
  --s3AccessKeyId "${access_key_id}" \
  --s3SecretAccessKey "${access_key_secret}" \
  --input "s3://${bucket_name}/${file_name}.json" \
  --output=http://production.es.com:9200/my_index

# Export ES data to S3 (using s3urls)
elasticdump \
  --s3AccessKeyId "${access_key_id}" \
  --s3SecretAccessKey "${access_key_secret}" \
  --input=http://production.es.com:9200/my_index \
  --output "s3://${bucket_name}/${file_name}.json"

参考文档：
https://blog.csdn.net/u010955999/article/details/80814656
https://www.cnblogs.com/mojita/p/12011800.html