Linux下安装配置Hive

1. 安装和配置MySQL

安装和配置MySQL

2. 安装hive

hive安装包下载地址

# 解压hive
[ho@ho Downloads]$ tar -zvxf apache-hive-2.3.7-bin

# 配置环境变量
[root@ho ~]# vim /etc/profile
# 增加内容如下：
# HIVE_HOME
export HIVE_HOME=hive放置的地方
export PATH=$HIVE_HOME/bin:$PATH

3. 配置hive

3.1 复制配置文件

[ho@ho apache-hive-2.3.7-bin]# cp hive-default.xml.template hive-site.xml
[ho@ho apache-hive-2.3.7-bin]# cp hive-env.sh.template hive-env.sh

3.2 修改hive-site

[ho@ho conf]# vim hive-site.xml 

文件内容如下：

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
--><configuration>
  <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
  <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
  <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
  <!-- Hive Execution Parameters -->
        <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionDriverName</name>
                <value>com.mysql.jdbc.Driver</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionUserName</name>
                <value>名称</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionPassword</name>
                <value>密码</value>
        </property>
        <property>
                <name>hive.metastore.schema.verification</name>
                <value>false</value>
        </property>
</configuration>

3.3 修改hive-env

[ho@ho conf]# vim hive-env.sh

在# Set HADOOP_HOME to point to a specific hadoop install directory后
先去除#，并将HADOOP_HOME修改成HADOOP_HOME = hadoop的地址

4. 安装并配置Mysql的驱动程序

mysql-connector-java.jar包下载
选择Red Hat Enterprise Linux / Oracle Linux
根据自己系统的版本选择适合的。

[ho@ho Downloads]$ rpm2cpio mysql-connector-java-8.0.22-1.el7.noarch.rpm | cpio -div

# 解压后找到jar包的位置后复制到hive的lib文件夹中
[root@ho lib]# ll /usr/share/java/mysql-connector-java.jar

5. 初始化hive

[ho@ho ~]$ mysql
mysql> create database hive default charset utf8 collate utf8_general_ci;
mysql> CREATE USER 'hive'@'%' IDENTIFIED BY 'Ho';
# 若此处报错则运行FLUSH PRIVILEGES;
mysql> GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%';
mysql> FLUSH PRIVILEGES;

[ho@ho bin]# schematool -dbType mysql -initSchema
......
schemaTool completed

[root@ho bin]# hive
......
hive> 

# 测试一下hive
hive> show databases;
OK
default
Time taken: 6.162 seconds, Fetched: 1 row(s)

到此处hive启动成功就安装配置完成了！以下是配置过程中遇到的问题。

6. 遇到的问题

6.1 初始化时遇到的问题：

问题1. HiveMetaException: Failed to load driver
解决方法：
将mysql-connector-java-8.0.22-1.jar包的位置后复制到hive的lib文件夹中

[root@ho lib]# ll /usr/share/java/mysql-connector-java-8.0.22-1.jar

2. HiveMetaException: Failed to get schema version
hive-site.xml配置出错
解决方法：
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
  <!-- localhost是我的ip映射的用户名，此处填自己的，或者你自己的ip也可以 -->
</property>

3. HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
解决方法：

[ho@ho ~]$ mysql
mysql>  drop database hive;

6.2 启动hive时遇到的问题：

1. Call From ho/192.168.3.5 to ho:9000 failed on connection exception: java.net.ConnectException: Connection refused;
解决方法：
检查hadoop中core-site.xml
<property>
  <name>fs.defaultFS
  <value>hdfs://localhost:9000
  localhost是我的ip映射的用户名，此处填自己的，或者你自己的ip也可以;这里跟上面问题2要对应
</property>
2. namenode.SafeModeException): Cannot create directory /tmp/hive. Name node is in safe mode.
解决方法：

  [root@ho ~]# hdfs dfsadmin -safemode leave

3. 非root模式下启动hive报/tmp/ho/stderr: permission denied
解决方法：
重新在非root模式下复制出/tmp/ho/stderr

6.3 使用hive时遇到的问题：

1. Permission denied: user=root, access=WRITE, inode="/user":ho:supergroup:drwxr-xr-x
解决方法：

[ho@ho ~]$ hdfs dfs -ls /
[ho@ho ~]$ hdfs dfs -mkdir /root
[ho@ho ~]$ hadoop fs -chmod 777 /root
[ho@ho ~]$ hdfs dfs -ls /
drwxrwxrwx   - ho   supergroup          0 ****-****-**** 23:57 /root

2. HiveException:MetaException(message:file:… is not a directory or unable to create one);
解决方法：

[ho@ho bin]# schematool -dbType mysql -initSchema