elasticdump是实现不同ElasticSearch集群之间索引迁移的工具。默认情况下,ITOA集群部署成功后,该工具存在于安装节点(客户端节点)上。elasticdump语法和常用参数选项如下:
elasticdump --input SOURCE --output DESTINATION [OPTIONS]
其中input/output表示索引数据源和数据目的地
SOURCE/ DESTINATION可以是ElasticSearch集群地址或者文件路径。
当SOURCE/ DESTINATION为ElasticSearch集群地址时格式如下:
格式: {protocol}://{host}:{port}/{index}
样例: http://127.0.0.1:9200/my_index
当SOURCE/ DESTINATION为文件路径时格式如下:
格式: {FilePath}
样例: /Users/evantahler/Desktop/dump.json
OPTIONS常用选项参数有:
--limit
用于限定每一批量操作时进行迁移的文档对象数目,默认值是100,也就是每一次从SOURCE到DESTINATION文档对象数量。
--quiet
只输出错误信息,该参数的默认值是false,如果该参数设置为true。控制台中关于数据迁移的进度信息不可见。
--type
数据迁移类型,可选值有analyzer, data, mapping三种。在ITOA由于统一版本的软件平台analyzer是相同的,所以进行索引迁移时,只需要迁移data, mapping两种类型的数据。该参数的默认选项是data
--ignore-errors
该参数用于控制数据迁移过程遇到错误数据的处理策略。默认情况下为false,表示遇到错误数据时,停止数据迁移。
--help
陈列该工具使用时的帮助信息。
以下两个实例为ElasticSearch和File之间进行索引的data和mapping互相迁移实例。
实例一:将索引(包括data和mapping)从ElasticSearch迁移至File
[root@training01 ~]# elasticdump --input=http://172.168.1.139:9200/lion-acg-example --output=/data/my_index_mapping.json --type=mapping
Sat, 17 Jun 2017 07:33:21 GMT | starting dump
Sat, 17 Jun 2017 07:33:21 GMT | got 1 objects from source elasticsearch (offset: 0)
Sat, 17 Jun 2017 07:33:21 GMT | sent 1 objects to destination file, wrote 1
Sat, 17 Jun 2017 07:33:21 GMT | got 0 objects from source elasticsearch (offset: 1)
Sat, 17 Jun 2017 07:33:21 GMT | Total Writes: 1
Sat, 17 Jun 2017 07:33:21 GMT | dump complete
[root@training01 ~]#elasticdump --input=http://172.168.1.139:9200/lion-acg-example --output=/data/my_index_data.json --type=data
Sat, 17 Jun 2017 07:34:33 GMT | starting dump
Sat, 17 Jun 2017 07:34:33 GMT | got 100 objects from source elasticsearch (offset: 0)
Sat, 17 Jun 2017 07:34:33 GMT | sent 100 objects to destination file, wrote 100
Sat, 17 Jun 2017 07:34:33 GMT | got 100 objects from source elasticsearch (offset: 100)
Sat, 17 Jun 2017 07:34:33 GMT | sent 100 objects to destination file, wrote 100
Sat, 17 Jun 2017 07:34:33 GMT | got 100 objects from source elasticsearch (offset: 200)
Sat, 17 Jun 2017 07:34:33 GMT | sent 100 objects to destination file, wrote 100
Sat, 17 Jun 2017 07:34:33 GMT | got 100 objects from source elasticsearch (offset: 300)
……………..
Sat, 17 Jun 2017 07:34:36 GMT | got 0 objects from source elasticsearch (offset: 20561)
Sat, 17 Jun 2017 07:34:36 GMT | Total Writes: 20561
Sat, 17 Jun 2017 07:34:36 GMT | dump complete
实例二:将索引(包括data和mapping)从File迁移至ElasticSearch
[root@training01 data]# elasticdump --input=/data/my_index_data.json --output=http://172.168.1.139:9200/h3c-acg-data --type=data
Sat, 17 Jun 2017 08:37:15 GMT | starting dump
Sat, 17 Jun 2017 08:37:15 GMT | got 100 objects from source file (offset: 0)
Sat, 17 Jun 2017 08:37:16 GMT | sent 100 objects to destination elasticsearch, wrote 100
Sat, 17 Jun 2017 08:37:16 GMT | got 137 objects from source file (offset: 100)
Sat, 17 Jun 2017 08:37:16 GMT | sent 137 objects to destination elasticsearch, wrote 137
Sat, 17 Jun 2017 08:37:16 GMT | got 135 objects from source file (offset: 237)
Sat, 17 Jun 2017 08:37:16 GMT | sent 135 objects to destination elasticsearch, wrote 135
………………..
Sat, 17 Jun 2017 08:37:36 GMT | Total Writes: 20561
Sat, 17 Jun 2017 08:37:36 GMT | dump complete
[root@training01 data]# elasticdump --input=/data/my_index_mapping.json --output=http://172.168.1.139:9200/h3c-acg-data --type=mapping
Sat, 17 Jun 2017 08:37:59 GMT | starting dump
Sat, 17 Jun 2017 08:37:59 GMT | got 1 objects from source file (offset: 0)
Sat, 17 Jun 2017 08:37:59 GMT | sent 1 objects to destination elasticsearch, wrote 10
Sat, 17 Jun 2017 08:37:59 GMT | got 0 objects from source file (offset: 1)
Sat, 17 Jun 2017 08:37:59 GMT | Total Writes: 10
Sat, 17 Jun 2017 08:37:59 GMT | dump complete