CentOS7 手动部署Hive1.2.1

目的：新项目需求，Hadoop集群+HIVE
版本：
hadoop：2.7.2
hive：1.2.1

基础环境

和上一篇文章相同：CentOS7 手动部署Hadoop集群2.7.2

环境准备

hive的元数据信息本次使用mysql存储，因此，不是hive之前要准备好mysql
前文提到，之前的环境用到了ansible，本次mysql依然采用ansible部署，该步骤略，如需要，可自行百度
mysql版本为：

[root@test-02 ~]# mysql --version
mysql  Ver 14.14 Distrib 5.6.39, for linux-glibc2.12 (x86_64) using  EditLine wrapper

创建hive数据库

create database hivemeta character set latin1;
grant all privileges on *.* to hive@"%" identified by "hive" with grant option; 
flush privileges; 

安装hive

1、下载hive安装包

wget http://archive.apache.org/dist/hive/hive-1.2.1/apache-hive-1.2.1-bin.tar.gz
#解压安装包
tar xf apache-hive-1.2.1-bin.tar.gz 

2、添加环境变量

[root@test-01 /opt]# cd /etc/profile.d/
[root@test-01 /etc/profile.d]# cat hive.sh
export HIVE_HOME=/opt/apache-hive-1.2.1-bin
export PATH=$PATH:$HIVE_HOME/bin
#使变量生效
source /etc/profile

3、创建文件夹

mkdir /opt/apache-hive-1.2.1-bin/tmp
chmod 777 -R /opt/apache-hive-1.2.1-bin/tmp

hadoop fs -mkdir -p /user/hive/warehouse
hadoop fs -mkdir -p /tmp/hive
hadoop fs -chmod -R 777 /user/hive
hadoop fs -chmod -R 777 /tmp/hive

4、修改hive-env.sh

[root@test-01 /opt/apache-hive-1.2.1-bin/conf]# vim hive-env.sh
#添加如下参数
export HADOOP_HOME=/opt/hadoop-2.7.2
export HIVE_CONF_DIR=/opt/apache-hive-1.2.1-bin/conf
export HIVE_AUX_JARS_PATH=/opt/apache-hive-1.2.1-bin/lib
export JAVA_HOME=/opt/java

5、修改hive-site.xml

#新建 hive-site.xml
vim hive-site.xml
<!--添加以下内容-->
<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://test-02:3306/hivemeta?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hive</value>
<description>password to use against metastore database</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://test-01:9000/user/hive/warehouse</value>
<description>hive default warehousehive default warehouse, if nessecory, change it</description>
</property>
</configuration>

6、修改hive-log4j.properties

#复制模板为 hive-log4j.properties
[root@test-01 /opt/apache-hive-1.2.1-bin/conf]$ cp hive-log4j.properties.template hive-log4j.properties
#修改参数
[root@test-01 /opt/apache-hive-1.2.1-bin/conf]$ vim hive-log4j.properties
#hive.log.dir=${java.io.tmpdir}/${user.name}
hive.log.dir=/opt/apache-hive-1.2.1-bin/log

7、添加mysql驱动

将mysql驱动包放到lib目录
注：下载驱动包，连接地址为 https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.40.tar.gz
如果想下载其他版本 只需要更改后面的版本包即可，例如：
https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.48.tar.gz
https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.48.zip

[root@test-01 ~]# wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.40.tar.gz
--2020-01-03 11:17:07--  https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.40.tar.gz
Resolving dev.mysql.com (dev.mysql.com)... 137.254.60.11
Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn.mysql.com//archives/mysql-connector-java-5.1/mysql-connector-java-5.1.40.tar.gz [following]
--2020-01-03 11:17:09--  https://cdn.mysql.com//archives/mysql-connector-java-5.1/mysql-connector-java-5.1.40.tar.gz
Resolving cdn.mysql.com (cdn.mysql.com)... 104.75.165.42
Connecting to cdn.mysql.com (cdn.mysql.com)|104.75.165.42|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3911557 (3.7M) [application/x-tar-gz]
Saving to: ‘mysql-connector-java-5.1.40.tar.gz’

100%[=================================================================================>] 3,911,557   1.10MB/s   in 3.4s   

2020-01-03 11:17:14 (1.10 MB/s) - ‘mysql-connector-java-5.1.40.tar.gz’ saved [3911557/3911557]
[root@test-01 ~]# cp mysql-connector-java-5.1.40-bin.jar /opt/apache-hive-1.2.1-bin/lib

启动&验证

1、启动

nohup hive --service metastore  &
nohup hive --service hiveserver2 &

2、验证

[root@test-01 ~]# su - hive
[hive@test-01 ~]$ hive
Logging initialized using configuration in file:/opt/apache-hive-1.2.1-bin/conf/hive-log4j.properties
hive> show databases;
OK
default
Time taken: 1.152 seconds, Fetched: 2 row(s)
#创建数据库
hive> create database test01;
hive> show databases;
OK
default
test01
Time taken: 1.152 seconds, Fetched: 2 row(s)
#创建测试表
hive> use test01;
hive> create table test01.employee (eud int,name String,salary String,destination String) COMMENT 'Employee table' ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n' STORED AS TEXTFILE;
OK
hive> show tables;
OK
employee
Time taken: 0.233 seconds, Fetched: 1 row(s)
hive>

至此，hive部署完成！