这是在开发环境中测试的hive到es的数据导入方法
[root@sage-bigdata-01 es6]# pwd
/opt/es6
1、上传到hdfs中
hdfs> hdfs dfs -mkdir /hive
hdfs> hdfs dfs -chown hive:hive /hive
hive> hdfs dfs -put commons-httpclient-3.1.jar /hive
hive> hdfs dfs -put elasticsearch-hadoop-6.3.0.jar /hive
2、将jar添加到hive中
hive> add jar hdfs://bigdata-01:8020/hive/elasticsearch-hadoop-6.3.0.jar;
hive> add jar hdfs://bigdata-01:8020/hive/commons-httpclient-3.1.jar;
3、创建外部表
hive> CREATE EXTERNAL TABLE persons_es (id STRING, age INT, name STRING)
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES('es.nodes'='172.16.2.115', 'es.resource' = 'hive/persons');
4、执行导入
hive> INSERT OVERWRITE TABLE persons_es select * from bigdata.persons;
Logs like this:
Query ID = hive_20180625172002_058cad13-89a3-4bad-aabd-abad5e105fb3
Total jobs = 1
Launching Job 1 out of 1
Status: Running (Executing on YARN cluster with App id application_1526267537875_0112)
--------------------------------------------------------------------------------
VERTICES STATUS TOTAL COMPLETED RUNNING PENDING FAILED KILLED
--------------------------------------------------------------------------------
Map 1 .......... SUCCEEDED 1 1 0 0 2 0
--------------------------------------------------------------------------------
VERTICES: 01/01 [==========================>>] 100% ELAPSED TIME: 16.94 s
--------------------------------------------------------------------------------
OK
Time taken: 19.925 seconds
5、help
TBLPROPERTIES Settings: https://www.elastic.co/guide/en/elasticsearch/hadoop/current/configuration.html