第三步：hive集群搭建

hive集群搭建
一、hive安装和配置
1、在/opt路径下
[root@hadoop-93 opt]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz

2、配置环境变量
vi /etc/profile
export HIVE_HOME=/opt/apache-hive-2.3.4-bin/
export HIVE_CONF_DIR=$HIV{E}_{H}OME/confexportCLASSPATH=$CLASSPATH:$HIV{E}_{H}OME/libexportPATH=$PATH:$HIVE_HOME/bin

source /etc/profile

二、配置mysql（注：切换到root用户）
1、卸载CentOS自带的MySQL
rpm -qa | grep mysql
rpm -e mysql-libs-5.1.66-2.el6_3.i686 --nodeps

2、下载mysql的repo源
wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm

3、安装mysql-community-release-el7-5.noarch.rpm包（repo源）
rpm -ivh mysql-community-release-el7-5.noarch.rpm

4、安装MySQL
wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar

yum install mysql-server

#yum install mysql
#yum install mysql-server
#yum install mysql-devel

service mysqld restart

5、初始化MySQL
(1)修改mysql的密码(root权限执行)
cd /usr/bin
./mysql_secure_installation

(2)输入当前MySQL数据库的密码为root, 初始时root是没有密码的, 所以直接回车

Enter current password for root (enter for none):
(3)设置MySQL中root用户的密码(应与下面Hive配置一致,下面设置为123456)

Set root password? [Y/n] Y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables…
… Success!
(4)删除匿名用户

Remove anonymous users? [Y/n] Y
… Success!
(5)是否不允许用户远程连接,选择N

Disallow root login remotely? [Y/n] N
… Success!
(6)删除test数据库

Remove test database and access to it? [Y/n] Y
Dropping test database…
… Success!
Removing privileges on test database…
… Success!
(7)重装

Reload privilege tables now? [Y/n] Y
… Success!
(8)完成

All done! If you’ve completed all of the above steps, your MySQL
installation should now be secure.
Thanks for using MySQL!
(9)登陆mysql

mysql -uroot -p
grant all privileges on . to root@'192.168.32.93’identified by ‘123456’;
三、hive安装和配置
1、下载hive
wget https://mirrors.tuna.tsinghua.edu.cn/apache/hive/hive-2.3.4/apache-hive-2.3.4-bin.tar.gz

2、将hive解压到/opt下
tar -zxvf apache-hive-2.3.4-bin.tar.gz

3、修改环境变量/etc/profile：
export HIVE_HOME=/opt/apache-hive-2.3.4-bin/
export HIVE_CONF_DIR=$HIV{E}_{H}OME/confexportCLASSPATH=$CLASSPATH:$HIV{E}_{H}OME/libexportPATH=$PATH:$HIVE_HOME/bin

4、source /etc.profile

5、hive --version

成功or失败

6、修改hive-site.xml：

touch hive-site.xml
vim hive-site.xml


javax.jdo.option.ConnectionURL
jdbc:mysql://192.168.32.93:3306/hive?createDatabaseIfNotExsit=true;characterEncoding=UTF-8


javax.jdo.option.ConnectionDriverName
com.mysql.jdbc.Driver


javax.jdo.option.ConnectionUserName
root


javax.jdo.option.ConnectionPassword
123456


datanucleus.readOnlyDatastore
false


datanucleus.fixedDatastore
false


datanucleus.autoCreateSchema
true


datanucleus.autoCreateTables
true


datanucleus.autoCreateColumns
true


datanucleus.autoCreateColumns
true


hive.metastore.schema.verification
false

hive.metastore.warehouse.dir /user/hive/warehouse location of default database for the warehouse hive.downloaded.resources.dir /opt/apache-hive-2.3.4-bin/tmp/resources Temporary local directory for added resources in the remote file system. hive.exec.dynamic.partition true hive.exec.dynamic.partition.mode nonstrict hive.exec.local.scratchdir /opt/apache-hive-2.3.4-bin/tmp/HiveJobsLog Local scratch space for Hive jobs hive.downloaded.resources.dir /opt/apache-hive-2.3.4-bin/tmp/ResourcesLog Temporary local directory for added resources in the remote file system. hive.querylog.location /opt/apache-hive-2.3.4-bin/tmp/HiveRunLog Location of Hive run time structured log file hive.server2.logging.operation.log.location /opt/apache-hive-2.3.4-bin/tmp/OpertitionLog Top level directory where operation tmp are stored if logging functionality is enabled

7、配置hive-env.sh
cp hive-env.sh.template hive-env.sh

export HADOOP_HOME=/opt/hadoop-3.0.2
export HIVE_CONF_DIR=/opt/apache-hive-2.3.4-bin/conf/
export HIVE_AUX_JARS_PATH=/opt/apache-hive-2.3.4-bin/lib/

8、复制mysql的驱动程序到hive/lib下面

wget http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.47/mysql-connector-java-5.1.47.jar

9、在mysql中hive的schema（在此之前需要创建mysql下的hive数据库）
[root@s100 bin]# pwd
/usr/local/hive/bin
[root@s100 bin]# schematool -dbType mysql -initSchema

10、执行hive命令

成功进入hive界面，hive配置完成

11、查询mysql（hive这个库是在 schematool -dbType mysql -initSchema 之前创建的！）

use hive;

那我们做这些事干什么的呢，下面小段测试大家感受一下

hive测试：
备注：这里是第二个配置文件的演示：所以数据库名称是hive数据库！

1.需要知道现在的hadoop中的HDFS存了什么

[root@hadoop-93 bin]# hadoop fs -lsr /

2.进入hive并创建一个测试库和测试表

[root@hadoop-93 bin]# hive

创建库：
hive> create database hive_1;
显示库：
hive> show databases;
OK
default
hive_1
Time taken: 0.19 seconds, Fetched: 2 row(s)
创建库成功！

3.查询一下HDFS有什么变化

多了一个库hive_1

那么我们的mysql下的hahive库有什么变化
mysql -uroot -p123456mysql> use hive
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

mysql> select * from DBS;

4.在hive_1下创建一个表hive_01
hive> use hive_1;
OK
Time taken: 0.754 seconds
hive> create table hive_01 (id int,name string);
OK
Time taken: 2.447 seconds

hdfs下的情况：

mysql下：

那么在web端是什么样子的呢！
http://hadoop-93:50070
或者 192.168.32.93:50070

hive集群搭建成功！

http://www.cnblogs.com/dxxblog/p/8193967.html