hive2es的一个例子

用于hive2es的es字段大小验证

1、es建库
curl -XDELETE "http://134.194.15.192:9200/tmp_peidw"
curl -X PUT 'http://134.194.15.192:9200/tmp_peidw' -d '{"settings": {"number_of_shards":5,"number_of_replicas":0,"refresh_interval": "120s"},"mappings":   {"tmp_peidw": {"properties":{ "serv_id": {"index": "not_analyzed","type": "string"},"Tag1": {"index": "not_analyzed","type": "string"},"10D001": {"index": "not_analyzed","type": "string"} }}}}'

 

2、创建扩展表

CREATE  EXTERNAL  TABLE  dic_tmp.ext_tmp_peidw(
  serv_id string,
  Tag1 string,
  10D001 String 
)  
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
TBLPROPERTIES(
'es.output.json' = 'true',
'es.index.auto.create' = 'flase',
'es.batch.write.refresh' = 'false', 
'es.batch.write.retry.wait' = '120', 
'es.batch.write.retry.count' = '6' , 
'es.nodes.wan.only' = 'true', 
'es.http.timeout' = '1m' , 
'es.http.retries' = '8' , 
'es.mapping.id' = 'serv_id' , 
'es.write.operation' = 'index' , 
'es.batch.size.bytes'='8m', 
'es.batch.size.entries'='8000', 
'es.resource' = 'tmp_peidw/tmp_peidw',
'es.mapping.names'='serv_id:serv_id,Tag1:Tag1,10D001:10D001',
'es.nodes'='134.194.15.192:9200'
);

3、创建数据源表

CREATE TABLE dic_tmp.tmp_peidw(
  `serv_id` string,
  `Tag1` string,
  `10D001` string
)  
insert into dic_tmp.tmp_peidw  values('peidw','kill -title','998');

4、hive表记录写入es
insert overwrite table      dic_tmp.ext_tmp_peidw      select serv_id,Tag1,10D001 from dic_tmp.tmp_peidw;

在es里验证
select * from tmp_peidw

这样验证是没问题，现在碰到问题是我的扩展表有上千个字段时，有些字段没有按'es.mapping.names'指定的那样转成标签编码，hive  往扩展表写入时自动转换成小写字母标签字段了。

在此向大神求解！！！！