有searchguard的es-hadoop插件安装

有searchguard的es-hadoop插件安装

   1：下载安装包

         wget https://artifacts.elastic.co/downloads/elasticsearch-hadoop/elasticsearch-hadoop-7.1.1.zip

   2：解压安装包

     把对应的elasticsearch-hadoop-7.1.1.jar放到对应的path中

    /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/jars

   cd /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/lib/spark/lib   
   ln -s ../../../jars/elasticsearch-hadoop-7.1.1.jar elasticsearch-hadoop-7.1.1.jar

    cd /opt/cloudera/parcels/CDH-6.1.1-1.cdh6.1.1.p0.875250/lib/hive/lib     

   ln -s ../../../jars/elasticsearch-hadoop-7.1.1.jar elasticsearch-hadoop-7.1.1.jar

  3：创建hive外表

   CREATE EXTERNAL TABLE es_hive_test(
     id STRING,
     name STRING,
     phone STRING
    )
  STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
  TBLPROPERTIES(
     'es.net.ssl' = 'true',
     'es.net.ssl.protocol' = 'ssl',
     'es.net.http.auth.user' = 'admin',
     'es.net.http.auth.pass' = '***',
     'es.net.ssl.keystore.location' = 'file:opt/XXX/conf/truststore.jks',
     'es.net.ssl.keystore.pass' = '***',
     'es.net.ssl.keystore.type' = 'JKS',
     'es.net.ssl.truststore.location' = 'file:opt/XXX/conf/truststore.jks',
     'es.net.ssl.truststore.pass' = '***',
     'es.resource' = 'test_hive/doc',
     'es.index.read.missing.as.empty' = 'yes',
     'es.nodes'='172.16.*.*:9200,172.16.*.*:9200',
     'es.index.auto.create' = 'true',
     'es.nodes.wan.only' = 'true',
     'es.read.metadata' = 'true',
     'es.mapping.names'='id:_metadata._id,name:name,phone:phone'
   );

然后查询该表没什么问题

4：集成spark需要引入commons-httpclient jar不然会报对应的类classNotFound

 通过踩坑搞好的，怕忘记记录一下