Hive安装以及问题汇总

最新推荐文章于 2023-03-01 21:46:31 发布

fuzuxian

最新推荐文章于 2023-03-01 21:46:31 发布

阅读量235

点赞数

分类专栏： Hive

本文链接：https://blog.csdn.net/image_fzx/article/details/80100027

版权

Hive 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

Hive只在一个节点上安装即可

1. 上传tar包

下载地址：点击打开链接

Hive版本与Hadoop版本：Hadoop、Hbase、Hive版本兼容性

2. 解压

tar -zxvf apache-hive-2.2.0-bin.tar.gz  -C ~/software/

mv apache-hive-2.2.0-bin  hive-2.2.0

接着配置HIVE_HOME环境变量

修改/etc/profile文件。
#vi /etc/profile
增加 
export HIVE_HOME=/home/hadoop/software/hive-2.2.0
修改 
export PATH=$JAVA_HOME/bin:$PATH:$HADOOP_HOME/bin:$HIVE_HOME/bin
保存退出
#source /etc/profile

3. 配置mysql metastore（切换到root用户）

 rpm -qa | grep mysql rpm -e mysql-libs-5.1.66-2.el6_3.i686 --nodeps rpm -ivh MySQL-server-5.1.73-1.glibc23.i386.rpm  rpm -ivh MySQL-client-5.1.73-1.glibc23.i386.rpm  修改mysql的密码 /usr/bin/mysql_secure_installation （注意：删除匿名用户，允许用户远程连接） 登陆mysql mysql -u root -p
因为我之前装了Mysql，所以。。。。
   没有MySQL-server-5.1.73-1.glibc23.i386.rpm 和 MySQL-client-5.1.73-1.glibc23.i386.rpm 
  可以在线安装：   sudo apt-get install mysql-server mysql-client
4.配置hive
     cp hive-default.xml.template hive-site.xml 
  或者重命名：

mv hive-default.xml.template hive-site.xml

 mv hive-env.sh.template hive-env.sh

配置 hive-env.sh 文件

export HADOOP_HOME=/usr/local/hadoop

# Hive Configuration Directory can be controlled by:
 export HIVE_CONF_DIR=/home/hadoop/software/hive-2.2.0/conf

# Folder containing extra ibraries required for hive compilation/execution can be controlled by:
 export HIVE_AUX_JARS_PATH=/home/hadoop/software/hive-2.2.0/lib

下载 mysql-connector-java-8.0.11.jar 地址：点击打开链接

将下载的 deb 文件解压，里面就有mysql-connector-java-8.0.11.jar，把这个JAR 包移动到 ~/software/hive-2.2.0/lib 目录下。修改权限

hadoop@Master:~/software/hive-2.2.0/lib$ chmod 777 mysql-connector-java-8.0.11.jar

配置 hive-site.xml 文件

（删除所有内容）

添加如下内容：

<configuration>

      <property>
	  <name>javax.jdo.option.ConnectionURL</name>
	  <value>jdbc:mysql://Master:3306/hive?createDatabaseIfNotExist=true</value>
	  <description>JDBC connect string for a JDBC metastore</description>
	</property>

	<property>
	  <name>javax.jdo.option.ConnectionDriverName</name>
	  <value>com.mysql.jdbc.Driver</value>
	  <description>Driver class name for a JDBC metastore</description>
	</property>

	<property>
	  <name>javax.jdo.option.ConnectionUserName</name>
	  <value>root</value>
	  <description>username to use against metastore database</description>
	</property>

	<property>
	  <name>javax.jdo.option.ConnectionPassword</name>
	  <value>000000fzx</value>
	  <description>password to use against metastore database</description>
	</property>

       <property>  
         <name>hive.metastore.schema.verification</name>  
         <value>false</value>  
       <description>  
    Enforce metastore schema version consistency.  
    True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic  
          schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures  
          proper metastore schema migration. (Default)  
    False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.  
         </description>  
      </property>  

</configuration>

5. 安装hive和mysq完成后，将mysql的连接jar包拷贝到$HIVE_HOME/lib目录下
如果出现没有权限的问题，在mysql授权(在安装mysql的机器上执行)
mysql -u root -p
#(执行下面的语句 *.*:所有库下的所有表 %：任何IP地址或主机都可以连接)
GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' WITH GRANT OPTION;
FLUSH PRIVILEGES;

启动 ./hive 问题一

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:614)
	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:232)
	at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:384)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))
	at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3793)
	at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:244)
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:227)
	... 14 more
Caused by: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql))
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3556)
	at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3528)
	at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3790)
	... 16 more

解决：

这是由于没有初始化数据库，生成元数据。
需要执行以下指令(在hive的bin目录下执行)

./schematool -initSchema -dbType mysql

最后一个参数为hive存放元数据所使用的数据库，我用的是mysql。如果使用derby，将最后的参数改为derby就可以了

启动 ./hive 问题二：

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:614)
	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:232)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
	at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1652)
	... 23 more
Caused by: MetaException(message:Version information not found in metastore. )
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:83)
	at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92)
Caused by: MetaException(message:Version information not found in metastore. )
	at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7754)

解决：

下载 mysql-connector-java-8.0.11.jar 地址：点击打开链接

将下载的 deb 文件解压，里面就有mysql-connector-java-8.0.11.jar，把这个JAR 包移动到 ~/software/hive-2.2.0/lib 目录下。修改权限

hadoop@Master:~/software/hive-2.2.0/lib$ chmod 777 mysql-connector-java-8.0.11.jar

4. 修改conf/hive-site.xml 中的 “hive.metastore.schema.verification” 值为 false 即可解决 “Caused by: MetaException(message:Version information not found in metastore. )”

conf/hive-site.xml 增加如下内容：

<property>  
   <name>hive.metastore.schema.verification</name>  
   <value>false</value>  
    <description>  
    Enforce metastore schema version consistency.  
    True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic  
          schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures  
          proper metastore schema migration. (Default)  
    False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.  
    </description>  
 </property>

fuzuxian

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
Hive安装以及问题汇总

Hive只在一个节点上安装即可1. 上传tar包下载地址：点击打开链接 Hive版本与Hadoop版本：Hadoop、Hbase、Hive版本兼容性2. 解压tar -zxvf apache-hive-2.2.0-bin.tar.gz -C ~/software/mv apache-hive-2.2.0-bin hive-2.2.0 接着配置HIVE_HOME环境变量修改...
复制链接

扫一扫