Hive数据仓库搭建--远程模式

最新推荐文章于 2022-07-06 10:30:53 发布

Erase Me

最新推荐文章于 2022-07-06 10:30:53 发布

阅读量478

点赞数

分类专栏：大数据文章标签： hadoop 大数据 hive

本文链接：https://blog.csdn.net/qq_44957186/article/details/107336166

版权

大数据专栏收录该内容

7 篇文章 0 订阅

订阅专栏

结点规划

IP	Hostname	角色
192.168.0.104	master	Client
192.168.0.102	slave1	metastore
192.168.0.106	slave2	Mysql

搭建步骤

1、安装Mysql

1.1 解压安装包

tar -xvf mysql-5.7.25-1.el7.x86_64.rpm-bundle.tar

[root@slave2 software]# tar -xvf mysql-5.7.29-1.el7.x86_64.rpm-bundle.tar
mysql-community-embedded-devel-5.7.29-1.el7.x86_64.rpm
mysql-community-test-5.7.29-1.el7.x86_64.rpm
mysql-community-embedded-5.7.29-1.el7.x86_64.rpm
mysql-community-embedded-compat-5.7.29-1.el7.x86_64.rpm
mysql-community-libs-5.7.29-1.el7.x86_64.rpm
mysql-community-client-5.7.29-1.el7.x86_64.rpm
mysql-community-server-5.7.29-1.el7.x86_64.rpm
mysql-community-devel-5.7.29-1.el7.x86_64.rpm
mysql-community-libs-compat-5.7.29-1.el7.x86_64.rpm
mysql-community-common-5.7.29-1.el7.x86_64.rpm
您在 /var/spool/mail/root 中有新邮件
[root@slave2 software]#

1.2 使用 rpm 命令安装MySql组件

使用命令rpm -ivh {-file-name}进行安装操作。

按照依赖关系依次安装rpm包依赖关系依次为common→libs→client→server

rpm -ivh mysql-community-common-5.7.22-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.22-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.22-1.el7.x86_64.rpm
rpm -ivh mysql-community-server-5.7.22-1.el7.x86_64.rpm

（1）首先安装mysql-community-common （服务器和客户端库的公共文件），使用命令：

[root@slave2 software]# rpm -ivh mysql-community-common-5.7.29-1.el7.x86_64.rpm

（2）其次安装mysql-community-libs（MySQL数据库客户端应用程序的共享库），使用命令：

[root@slave2 software]# rpm -ivh mysql-community-libs-5.7.29-1.el7.x86_64.rpm

可能会报错：mysql依赖错误

[root@slave2 software]# rpm -ivh mysql-community-libs-5.7.29-1.el7.x86_64.rpm
警告：mysql-community-libs-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
错误：依赖检测失败：
	mysql-community-common(x86-64) >= 5.7.9 被 mysql-community-libs-5.7.29-1.el7.x86_64 需要
	mariadb-libs 被 mysql-community-libs-5.7.29-1.el7.x86_64 取代
[root@slave2 software]#

解决方案：

清除yum里所有mysql依赖包

[root@slave2 software]# rpm -qa|grep mysql
[root@slave2 software]# yum remove mysql-libs
已加载插件：fastestmirror
正在解决依赖关系
--> 正在检查事务
---> 软件包 mariadb-libs.x86_64.1.5.5.65-1.el7 将被 删除
--> 正在处理依赖关系 libmysqlclient.so.18()(64bit)，它被软件包 2:postfix-2.10.1-9.el7.x86_64 需要
--> 正在处理依赖关系 libmysqlclient.so.18(libmysqlclient_18)(64bit)，它被软件包 2:postfix-2.10.1-9.el7.x86_64 需要
--> 正在检查事务
---> 软件包 postfix.x86_64.2.2.10.1-9.el7 将被 删除
--> 解决依赖关系完成

依赖关系解决

===========================================================================================================
 Package                   架构                版本                           源                      大小
===========================================================================================================
正在删除:
 mariadb-libs              x86_64              1:5.5.65-1.el7                 @anaconda              4.4 M
为依赖而移除:
 postfix                   x86_64              2:2.10.1-9.el7                 @anaconda               12 M

事务概要
===========================================================================================================
移除  1 软件包 (+1 依赖软件包)

安装大小：17 M
是否继续？[y/N]：y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  正在删除    : 2:postfix-2.10.1-9.el7.x86_64                                                          1/2 
  正在删除    : 1:mariadb-libs-5.5.65-1.el7.x86_64                                                     2/2 
  验证中      : 1:mariadb-libs-5.5.65-1.el7.x86_64                                                     1/2 
  验证中      : 2:postfix-2.10.1-9.el7.x86_64                                                          2/2 

删除:
  mariadb-libs.x86_64 1:5.5.65-1.el7                                                                       

作为依赖被删除:
  postfix.x86_64 2:2.10.1-9.el7                                                                            

完毕！
[root@slave2 software]#

再重新安装`mysql-community-common-5.7.29-1.el7.x86_64.rpm`：

[root@slave2 software]# rpm -ivh mysql-community-common-5.7.29-1.el7.x86_64.rpm
警告：mysql-community-common-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-common-5.7.29-1.e################################# [100%]

接着安装`rpm -ivh mysql-community-libs-5.7.29-1.el7.x86_64.rpm`：

[root@slave2 software]# rpm -ivh mysql-community-libs-5.7.29-1.el7.x86_64.rpm
警告：mysql-community-libs-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-libs-5.7.29-1.el7################################# [100%]
[root@slave2 software]#

（3）之后安装 mysql-community-libs-compat（MySQL 之前版本的共享兼容库），使用命令：

[root@slave2 software]# rpm -ivh mysql-community-libs-compat-5.7.29-1.el7.x86_64.rpm
警告：mysql-community-libs-compat-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-libs-compat-5.7.2################################# [100%]
[root@slave2 software]#

（4）之后安装 mysql-community-client（MySQL客户端应用程序和工具），使用命令：

[root@slave2 software]# rpm -ivh mysql-community-client-5.7.29-1.el7.x86_64.rpm
警告：mysql-community-client-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-client-5.7.29-1.e################################# [100%]
[root@slave2 software]#

（5）最后安装 mysql-community-server（数据库服务器和相关工具），使用命令：

竟然能报错：

[root@slave2 software]# rpm -ivh mysql-community-server-5.7.29-1.el7.x86_64.rpm
警告：mysql-community-server-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
错误：依赖检测失败：
	/usr/bin/perl 被 mysql-community-server-5.7.29-1.el7.x86_64 需要
	libaio.so.1()(64bit) 被 mysql-community-server-5.7.29-1.el7.x86_64 需要
	libaio.so.1(LIBAIO_0.1)(64bit) 被 mysql-community-server-5.7.29-1.el7.x86_64 需要
	libaio.so.1(LIBAIO_0.4)(64bit) 被 mysql-community-server-5.7.29-1.el7.x86_64 需要
	net-tools 被 mysql-community-server-5.7.29-1.el7.x86_64 需要
	perl(Getopt::Long) 被 mysql-community-server-5.7.29-1.el7.x86_64 需要
	perl(strict) 被 mysql-community-server-5.7.29-1.el7.x86_64 需要
[root@slave2 software]#

解决方案：

缺少三个Mysql依赖，使用yum安装即可

net-tools

libaio

perl

使用yum安装对应包

[root@slave2 software]# yum -y install perl.x86_64
[root@slave2 software]# yum install -y libaio.x86_64
[root@slave2 software]# yum -y install net-tools.x86_64

安装libaio:

[root@slave2 software]# yum -y install libaio
已加载插件：fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.163.com
 * updates: mirrors.aliyun.com
base                                                                                | 3.6 kB  00:00:00     
extras                                                                              | 2.9 kB  00:00:00     
updates                                                                             | 2.9 kB  00:00:00     
正在解决依赖关系
--> 正在检查事务
---> 软件包 libaio.x86_64.0.0.3.109-13.el7 将被 安装
--> 解决依赖关系完成

依赖关系解决

===========================================================================================================
 Package                架构                   版本                             源                    大小
===========================================================================================================
正在安装:
 libaio                 x86_64                 0.3.109-13.el7                   base                  24 k

事务概要
===========================================================================================================
安装  1 软件包

总下载量：24 k
安装大小：38 k
Downloading packages:
libaio-0.3.109-13.el7.x86_64.rpm                                                    |  24 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
警告：RPM 数据库已被非 yum 程序修改。
  正在安装    : libaio-0.3.109-13.el7.x86_64                                                           1/1 
  验证中      : libaio-0.3.109-13.el7.x86_64                                                           1/1 

已安装:
  libaio.x86_64 0:0.3.109-13.el7                                                                           

完毕！
[root@slave2 software]#

再次检查libaio：

[root@slave2 software]# rpm -qa|grep libaio
libaio-0.3.109-13.el7.x86_64
[root@slave2 software]#

安装net-tools:

[root@slave2 software]# yum -y install net-tools.x86_64
已加载插件：fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.163.com
 * updates: mirrors.aliyun.com
正在解决依赖关系
--> 正在检查事务
---> 软件包 net-tools.x86_64.0.2.0-0.25.20131004git.el7 将被 安装
--> 解决依赖关系完成

依赖关系解决

===========================================================================================================
 Package                架构                版本                                   源                 大小
===========================================================================================================
正在安装:
 net-tools              x86_64              2.0-0.25.20131004git.el7               base              306 k

事务概要
===========================================================================================================
安装  1 软件包

总下载量：306 k
安装大小：917 k
Downloading packages:
net-tools-2.0-0.25.20131004git.el7.x86_64.rpm                                       | 306 kB  00:00:00     
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  正在安装    : net-tools-2.0-0.25.20131004git.el7.x86_64                                              1/1 
  验证中      : net-tools-2.0-0.25.20131004git.el7.x86_64                                              1/1 

已安装:
  net-tools.x86_64 0:2.0-0.25.20131004git.el7                                                              

完毕！

`mysql-community-server-5.7.29-1.el7.x86_64.rpm`终于安装成功：

[root@slave2 software]# rpm -ivh mysql-community-server-5.7.29-1.el7.x86_64.rpm
警告：mysql-community-server-5.7.29-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-server-5.7.29-1.e################################# [100%]

2、登录 MySQL

1. 初始化 MySQL 的数据库

安装好 MySQL 后，我们需要初始化数据库，初始化和启动数据库时最好不要使用root用户，而是使用mysql用户启动。

[root@slave2 software]# mysqld --initialize-insecure --user=mysql

2. 启动 MySQL 服务

使用如下命令开启 MySQL 服务，让其在后台运行：

[root@slave2 software]# mysqld --user=mysql &
[1] 2965

说明：一定要加“&”，才能脚本放到后台运行。

3. 登录 MySQL

使用root用户无密码登录 MySQL：

[root@slave2 software]# mysql -uroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.29 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

4. 重置 MySQL 密码

在 5.7 版本后，我们可以使用 alter user...identified by命令把root用户的密码修改为“123456”，具体命令如下所示：

mysql> alter user 'root'@'localhost' identified by '123456';
Query OK, 0 rows affected (0.00 sec)

修改完成，使用exit或者quit命令退出 MySQL，重新登录验证密码是否修改成功，具体命令如下所示：

mysql> exit
Bye
# 使用密码登录
[root@slave2 software]# mysql -uroot -p123456
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.29 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

3、增加远程登录权限

当我们的帐号不允许从远程登录，只能在localhost连接时。这个时候只要在 MySQL 服务器上，更改mysql 数据库里的 user 表里的 host 项，从localhost改成%即可实现用户远程登录。

（1）首先我们来查看 mysql 数据库下的 user表信息：

mysql> use mysql;  # 切换成mysql数据库
mysql> select user,host from user; # 查询用户信息

可以看到在user表中已创建的root用户。host字段表示登录的主机，其值可以用IP地址，也可用主机名。

（2）实现远程连接（授权法）

将 host 字段的值改为%就表示在任何客户端机器上能以root用户登录到 MySQL 服务器，建议在开发时设为%。

# 设置远程登录权限 
mysql> update user set host='%' where host='localhost';
# 刷新配置信息
mysql> flush privileges;

效果如下所示：

mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.7.29 MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

# 1、切换成mysql数据库
mysql> use mysql
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

# 2、查询用户信息
mysql> select user,host from user;
+---------------+-----------+
| user          | host      |
+---------------+-----------+
| mysql.session | localhost |
| mysql.sys     | localhost |
| root          | localhost |
+---------------+-----------+
3 rows in set (0.00 sec)

# 3、设置远程登录权限 
mysql> update user set host='%' where host='localhost';
Query OK, 3 rows affected (0.00 sec)
Rows matched: 3  Changed: 3  Warnings: 0

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

# 4、# 刷新配置信息
mysql> select user,host from user;
+---------------+------+
| user          | host |
+---------------+------+
| mysql.session | %    |
| mysql.sys     | %    |
| root          | %    |
+---------------+------+
3 rows in set (0.00 sec)

mysql>

4、Hive安装

1、解压安装包

现在已经为大家下载好了 hive2.3.4 的安装包，存放在 /root/software 目录下，首先进入此目录下，使用如下命令进行解压即可使用：

[root@slave1 software]# tar -zxvf apache-hive-3.1.2-bin.tar.gz

将其解压到当前目录下，即 /root/software 中。

2、配置环境变量

（1）首先打开 `/etc/profile` 文件：

[root@slave1 software]# vim /etc/profile

（2）将以下内容添加到配置文件的底部，添加完成输入“:wq”保存退出：

# 配置Hive的安装目录   
export HIVE_HOME=/root/software/apache-hive-3.1.2-bin  
# 在原PATH的基础上加入Hive的bin目录  
export PATH=$PATH:$HIVE_HOME/bin

（3）让配置文件立即生效，使用如下命令：

[root@slave1 software]# source /etc/profile

（4）检测 Hive 环境变量是否设置成功，使用如下命令查看 Hive 版本：

[root@slave1 software]# hive --version

执行此命令后，若是出现 Hive 版本信息说明配置成功：

[root@slave1 software]# hive --version
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/software/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/software/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive 3.1.2
Git git://HW13934/Users/gates/tmp/hive-branch-3.1/hive -r 8190d2be7b7165effa62bd21b7d60ef81fb0e4af
Compiled by gates on Thu Aug 22 15:01:18 PDT 2019
From source with checksum 0492c08f784b188c349f6afb1d8d9847
[root@slave1 software]#

3 、修改配置文件 `hive-env.sh`

切换到 ${HIVE_HOME}/conf 目录下，将 hive-env.sh.template 文件复制一份并重命名为 hive-env.sh：

cp hive-env.sh.template hive-env.sh

修改完成，使用 vi 编辑器进行编辑：

vim hive-env.sh

在文件中配置 HADOOP_HOME、HIVE_CONF_DIR 以及HIVE_AUX_JARS_PATH 参数：

# 配置Hadoop安装路径  
HADOOP_HOME=/root/software/hadoop-2.7.7  
  
# 配置Hive配置文件存放路径  
export HIVE_CONF_DIR=/root/software/apache-hive-2.3.4-bin/conf  
  
# 配置Hive运行资源库路径  
export HIVE_AUX_JARS_PATH=/root/software/apache-hive-2.3.4-bin/lib

配置完成，输入“:wq”保存退出。

4、Hive元数据配置到MySQL（`Slave1服务端`）

1. 驱动拷贝

将/root/software目录下的 MySQL 驱动包 mysql-connector-java-5.1.48-bin.jar 拷贝到 ${HIVE_HOME}/lib 目录下。

[root@slave1 mysql-connector-java-5.1.48]# echo $HIVE_HOME
/root/software/apache-hive-3.1.2-bin
[root@slave1 mysql-connector-java-5.1.48]# cp mysql-connector-java-5.1.48-bin.jar /root/software/apache-hive-3.1.2-bin/lib

2. 配置 Metastore到MySQL（`Slave1服务端`）

（1）在${HIVE_HOME}/conf目录下创建一个名为hive-site.xml的文件，并使用vi编辑器进行编辑：

[root@slave1 apache-hive-3.1.2-bin]# cd conf
[root@slave1 conf]# ls
beeline-log4j2.properties.template  hive-exec-log4j2.properties.template  llap-cli-log4j2.properties.template
hive-default.xml.template           hive-log4j2.properties.template       llap-daemon-log4j2.properties.template
hive-env.sh.template                ivysettings.xml                       parquet-logging.properties


[root@slave1 conf]# touch hive-site.xml
[root@slave1 conf]# vi hive-site.xml

（2）根据官方文档配置参数（https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+Administration），拷贝数据到 hive-site.xml 文件中：

配置一：（可能会报错）

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl"href="configuration.xsl"?>
<configuration>
        <property>
                <name>hive.metastore.warehouse.dir</name>
                <!-- 这里配置hive产生的元数据的存放位置，根据自己喜好进行配置 -->
                <value>/user/hive/warehouse</value>
 
        </property>
 
        <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <!-- 这里配置hive连接mysql服务所需要的jdbc支持 -->
                <value>jdbc:mysql://test1:3306/hive?createDatabaseIfNotExist=true</value>
        </property>
 
        <property>
                <name>javax.jdo.option.ConnectionDriverName</name>
                <!-- 配置mysql的Driver -->
                <value>com.mysql.jdbc.Driver</value>
        </property>
 
        <property>
                <name>javax.jdo.option.ConnectionUserName</name>
                <!-- 这里配置的是连接mysql时的登陆用户名 -->
                <value>root</value>
        </property>
 
        <property>
                <name>javax.jdo.option.ConnectionPassword</name>
                <!-- 这里配置的是连接mysql时的登陆密码 -->
                <value>123456</value>
        </property>
 
</configuration>

配置二：

<configuration>

        <property>
                <name>hive.metastore.schema.verification</name>
                <value>false</value>
        </property>

        <property>
                <name>hive.metastore.warehouse.dir</name>
                <!-- 这里配置hive产生的元数据的存放位置，根据自己喜好进行配置 -->
                <value>/user/hive/warehouse</value>

        </property>

        <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <!-- 这里配置hive连接mysql服务所需要的jdbc支持 -->
                <value>jdbc:mysql://slave2:3306/hive?useSSL=false</value>
        </property>

        <property>
                <name>javax.jdo.option.ConnectionDriverName</name>
                <!-- 配置mysql的Driver -->
                <value>com.mysql.jdbc.Driver</value>
        </property>

        <property>
                <name>javax.jdo.option.ConnectionUserName</name>
                <!-- 这里配置的是连接mysql时的登陆用户名 -->
                <value>root</value>
        </property>

        <property>
                <name>javax.jdo.option.ConnectionPassword</name>
                <!-- 这里配置的是连接mysql时的登陆密码 -->
                <value>123456</value>
        </property>

        <property>
                <name>datanucleus.schema.autoCreateAll</name>
                <value>true</value>
        </property>

</configuration>

5、修改`Master客户端`的配置文件

5.1安装和配置环境变量和slave1一样

5.2进入apache-hive-1.2.1-bin/conf目录，创建名为hive-site.xml的文件。

详细配置：

<?xmlversion="1.0"?>
<?xmlversion="1.0"?>
<?xml-stylesheettype="text/xsl" href="configuration.xsl"?>
 
<configuration>
        
        <property>
               <name>hive.metastore.warehouse.dir</name>
               <value>/user/hive/warehouse</value>
        </property>
 
        <property>
            	<!--  这里配置的是否启用本地服务器连接Hive，这里我们是客户端和服务端分开的一个配置，所以这里设置为false. -->
               <name>hive.metastore.local</name>               
               <value>false</value>
        </property>
 
        <property>
            <!--  这里配置的是连接Hive服务端的匹配地址 -->
               <name>hive.metastore.uris</name>
               <value>thrift://slave1:9083</value>
        </property>
 
</configuration>

6、初始化元数据库

注意：如果使用的是 2.x 版本的 Hive，那么就必须手动初始化元数据库。 使用 schematool -dbType <db type> -initSchema 命令进行初始化：

schematool -dbType mysql -initSchema

若是出现“schemaTool completed”则初始化成功。

7、启动服务

先启动Hadoop集群

[root@master conf]# start-dfs.sh
Starting namenodes on [master]
master: starting namenode, logging to /root/software/hadoop-2.7.7/logs/hadoop-root-namenode-master.out
master: starting datanode, logging to /root/software/hadoop-2.7.7/logs/hadoop-root-datanode-master.out
slave1: starting datanode, logging to /root/software/hadoop-2.7.7/logs/hadoop-root-datanode-slave1.out
slave2: starting datanode, logging to /root/software/hadoop-2.7.7/logs/hadoop-root-datanode-slave2.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /root/software/hadoop-2.7.7/logs/hadoop-root-secondarynamenode-master.out
[root@master conf]# start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /root/software/hadoop-2.7.7/logs/yarn-root-resourcemanager-master.out
slave1: starting nodemanager, logging to /root/software/hadoop-2.7.7/logs/yarn-root-nodemanager-slave1.out
slave2: starting nodemanager, logging to /root/software/hadoop-2.7.7/logs/yarn-root-nodemanager-slave2.out
master: starting nodemanager, logging to /root/software/hadoop-2.7.7/logs/yarn-root-nodemanager-master.out
[root@master conf]# jps
2176 NodeManager
2054 ResourceManager
1613 NameNode
1742 DataNode
1903 SecondaryNameNode
2367 Jps
[root@master conf]#

7.1启动Hive服务端：

在开启HiveClint和metastore的时候出现这个警告：

Tue Jul 14 12:17:28 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Jul 14 12:17:29 CST 2020 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

MySQL在高版本需要指明是否进行SSL连接。
解决方法：you need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true，这里直接在hive-site.xml中设置value为false。

    <property>
        <!-- 这里配置hive连接mysql服务所需要的jdbc支持 -->
        <name>javax.jdo.option.ConnectionURL</name>
            # <value>jdbc:mysql://slave2:3306/hive?createDatabaseIfNotExist=false</value>
        <!-- 把上面的换成 -->
        <value>jdbc:mysql://master:3306/hive?useSSL=false</value>
    </property>

7.2其次启动Hive客户端：

[root@master conf]# hive
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/software/jdk1.8.0_171/bin:/root/software/hadoop-2.7.7/bin:/root/software/hadoop-2.7.7/sbin:/root/bin:/root/software/jdk1.8.0_171/bin:/root/software/hadoop-2.7.7/bin:/root/software/hadoop-2.7.7/sbin:/root/software/apache-hive-3.1.2-bin/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/software/apache-hive-3.1.2-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/software/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 356cb5f0-9379-4e11-8114-04c77bc0ddde

Logging initialized using configuration in jar:file:/root/software/apache-hive-3.1.2-bin/lib/hive-common-3.1.2.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 7ebfd808-61fa-4117-a1bd-fe14d0637de0
hive>

8、测试hive

我们在Hive服务端输入查找数据库的命令:show databases;

hive> show databases;
OK
default
Time taken: 0.554 seconds, Fetched: 1 row(s)

Erase Me

关注

0
点赞
踩
3

收藏

觉得还不错? 一键收藏
0
评论
Hive数据仓库搭建--远程模式

结点规划IPHostname角色192.168.0.104masterClient192.168.0.102slave1metastore192.168.0.106slave2Mysql搭建步骤1、安装Mysql1.1 解压安装包tar -xvf mysql-5.7.25-1.el7.x86_64.rpm-bundle.tar[root@slave2 software]# tar -xvf mysql-5.7.29-1.el7.x86_64.
复制链接

扫一扫