CentOs7 Hive 部署

安装

Hive基于Hadoop，为了与Hadoop匹配，下载cdh版本的Hive，下载地址
下载完成后，上传服务器，解压

tar -zxvf hive-1.1.0-cdh5.15.1.tar.gz

配置

解压完成后，添加HIVE_HOME到系统环境变量

export HIVE_HOME=/opt/hive-1.1.0-cdh5.15.1
export PATH=$PATH:$HIVE_HOME/bin

修改Hive的配置文件

cd /opt/hive-1.1.0-cdh5.15.1/conf/

hive-env.sh
添加 HADOOP_HOME=/opt/hadoop-2.6.0-cdh5.15.1

hive-site.xml
添加以下内容
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

#添加mysql服务器所在ip和port，hadoop_hive是所创建的数据库名称
<configuration>
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://192.168.44.12:3306/hadoop_hive?createDatabaseIfNotExist=true</value>
</property>

#使用mysql数据库，就用jdbc
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>

#数据库用户名 一般是root
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>yourUserName</value>
</property>

#数据库密码 填设置的root用户的密码
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>yourPassword</value>
</property>
</configuration>

拷贝MySQL驱动包到$HIVE_HOME/lib

cp mysql-connector-java-5.1.27-bin.jar /opt/hive-1.1.0-cdh5.15.1/lib

这里有个坑，jdbc的版本使用5.1.27可以，5.1.47就不行 不知道是不是版本太高

启动

# 先启动Hadoop的所有服务 尤其是DataNode
./start-all.sh
jps
5283 NameNode
5427 DataNode
5606 SecondaryNameNode
6056 NodeManager
9512 Jps
5771 ResourceManager
# 启动hive
hive

which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/java/jdk1.8.0_261/bin:/opt/hadoop-2.6.0-cdh5.15.1/bin:/opt/hive-1.1.0-cdh5.15.1/bin:/usr/local/kafka/bin:/opt/flume/bin:/root/bin)

Logging initialized using configuration in jar:file:/opt/hive-1.1.0-cdh5.15.1/lib/hive-common-1.1.0-cdh5.15.1.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.
hive> 

创建一个数据库，会默认使用先检查是否有hive的数据库，没有的话就创建一个hadoop_hive，后面其他hive相关的都放在这里面

hive> create database hive_test;

看一下mysql服务端

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| gaohan             |
| hadoop_hive        |
| mysql              |
| performance_schema |
| sys                |
+--------------------+

mysql> use hadoop_hive;
Database changed

mysql> show tables;
+---------------------------+
| Tables_in_hadoop_hive     |
+---------------------------+
| bucketing_cols            |
| cds                       |
| columns_v2                |
| database_params           |
| dbs                       |
| func_ru                   |
| funcs                     |
| global_privs              |
| part_col_stats            |
| partition_key_vals        |
| partition_keys            |
| partition_params          |
| partitions                |
| roles                     |
| sd_params                 |
| sds                       |
| sequence_table            |
| serde_params              |
| serdes                    |
| skewed_col_names          |
| skewed_col_value_loc_map  |
| skewed_string_list        |
| skewed_string_list_values |
| skewed_values             |
| sort_cols                 |
| tab_col_stats             |
| table_params              |
| tbls                      |
| version                   |
+---------------------------+

mysql> select * from dbs \G;
*************************** 1. row ***************************
          DB_ID: 1
           DESC: Default Hive database
DB_LOCATION_URI: hdfs://node2:8020/user/hive/warehouse
           NAME: default
     OWNER_NAME: public
     OWNER_TYPE: ROLE
*************************** 2. row ***************************
          DB_ID: 2
           DESC: NULL
DB_LOCATION_URI: hdfs://node2:8020/user/hive/warehouse/hive_test.db
           NAME: hive_test
     OWNER_NAME: root
     OWNER_TYPE: USER

dbs存放着hive创建的数据库信息，可以看到hdfs的路径

hadoop fs -ls /user/hive/warehouse

NameNode启动时会进入该模式进行检测，检查数据块的完整性，处于该模式下的集群无法对HDFS进行操作，出现错误

Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot create directory /tmp/hive. 
Name node is in safe mode.

可以手动离开安全模式

hadoop dfsadmin -saftmode leave