总结metastore 的调研工作

访问hive 命令 beeline -n ubuntu -u jdbc:hive2://192.168.30.12:10000 没有密码

hive.xml有关metastore的配置

<configuration>
<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
     <description>
          Enforce metastore schema version consistency.
          True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic
          schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures
          proper metastore schema migration. (Default)
          False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
      </description>
    </property>
    <!-- 存储在hdfs上的数据路径 -->
    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/user/hive/warehouse</value>
        <description>location of default database for the warehouse</description>
    </property>
    <!-- 本地mysql -->
    <property>
        <name>hive.metastore.local</name>
        <value>true</value>
    </property>
    <property>
       <name>javax.jdo.option.ConnectionURL</name>
       <value>jdbc:mysql://localhost:3306/hive??createDatabaseIfNotExist=tru
e&amp;useSSL=false</value>
        <description>JDBC connect string for a JDBC metastore</description>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
        <description>Driver class name for a JDBC metastore</description>
    </property>
    <property>
       <name>javax.jdo.option.ConnectionPassword</name>
       <value>hive</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
        <description>Username to use against metastore database</description>
     </property>
</configuration>

 

有Java访问metasotre的接口，官方的

https://hive.apache.org/javadocs/r2.1.1/api/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.html

https://docs.microsoft.com/en-us/openspecs/sql_server_protocols/ms-hmshttp/d94e2883-7175-484b-a107-d51c19e92a18

https://cwiki.apache.org/confluence/display/Hive/MetaStore+API+Tests#MetaStoreAPITests-IMetaStoreClientTests

https://cwiki.apache.org/confluence/display/Hive/Hive+APIs+Overview#HiveAPIsOverview-Metastore(Java)

hive

https://cwiki.apache.org/confluence/display/Hive/HiveClient#HiveClient-ThriftJavaClient

一个连接hive 的最简单的例子https://stackoverflow.com/questions/21370431/how-to-access-hive-via-python?answertab=active#tab-top

To install you'll need these libraries:

pip install sasl
pip install thrift
pip install thrift-sasl
pip install PyHive

#!/usr/bin/env python
 
from pyhive import hive
cursor = hive.connect(host="192.168.30.12", 
                      port=10000, 
                      username="ubuntu",
                      database="test"
                      ).cursor()
cursor.execute("SELECT * FROM aaaa LIMIT 10")
# columnList = [desc[0] for desc in cursor.description]
# headerStr = ",".join(columnList)
# headerTuple = tuple(headerStr.split (",")
# print(headerTuple)
# print(cursor.fetchone())
print(cursor.fetchall())
print(cursor.getSchema())

其他的：

元数据库介绍

https://www.cnblogs.com/qingyunzong/p/8710356.html#_label5_0

https://blog.csdn.net/weixin_30535843/article/details/98440564

Hive 安装 python thrift 连接hiserver2

https://blog.csdn.net/hjh00/article/details/64917226

Thrift方式连接hiveServer2+Kerberos

https://blog.csdn.net/clj198606061111/article/details/84928612?utm_medium=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-2.control&depth_1-utm_source=distribute.pc_relevant.none-task-blog-BlogCommendFromMachineLearnPai2-2.control

MetaStore Thrift

https://blog.csdn.net/ThreeAspects/article/details/108747817

python尝试连接metastore 看

http://www.voidcn.com/article/p-qyjacomz-bwe.html

https://github.com/cloudera/hue/blob/master/apps/beeswax/src/beeswax/server/hive_server2_lib.py

https://github.com/akolb1/metastore-python

https://github.com/oreh/thrift_hive_metastore

https://github.com/oreh/thrift_hive_metastore/blob/master/thrift_hive_metastore/ThriftHiveMetastore.py

https://github.com/bryanyang0528/hive_sync

hive metastore java api使用

https://blog.csdn.net/cl2010abc/article/details/104325021

https://blog.csdn.net/a794922102/article/details/84840936

https://www.jianshu.com/p/200d617b21e6

https://www.cnblogs.com/yangsy0915/p/8456806.html

java 连接hive

https://blog.csdn.net/weixin_34085658/article/details/91928285

元数据库mysql中运行语句

select a.NAME,b.TBL_NAME from DBS as a inner join TBLS as b on a.DB_ID=b.DB_ID where TBL_ID in (select distinct(TBL_ID) from PARTITIONS);

https://blog.csdn.net/david__peng/article/details/82968068