Hive安装（一）

最新推荐文章于 2024-07-24 17:30:44 发布

maplea2012

最新推荐文章于 2024-07-24 17:30:44 发布

阅读量100

点赞数

分类专栏： Hive 文章标签： hive mysql 数据库

本文链接：https://blog.csdn.net/weixin_37901366/article/details/130328626

版权

Hive 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

一、Mysql安装

1、准备安装包

01_mysql-community-common-5.7.16-1.el7.x86_64.rpm

02_mysql-community-libs-5.7.16-1.el7.x86_64.rpm

03_mysql-community-libs-compat-5.7.16-1.el7.x86_64.rpm

04_mysql-community-client-5.7.16-1.el7.x86_64.rpm

05_mysql-community-server-5.7.16-1.el7.x86_64.rpm

mysql-connector-java-5.1.27-bin.jar

2、卸载已安装的mysql(虚拟机为例)

[root@master ~]# rpm -qa | grep -i -E mysql\|mariadb | xargs -n1 sudo rpm -e --nodeps

3、安装Mysql依赖

[root@master mysql]# rpm -ivh 01_mysql-community-common-5.7.16-1.el7.x86_64.rpm
警告：01_mysql-community-common-5.7.16-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-common-5.7.16-1.e################################# [100%]

[root@master mysql]# rpm -ivh 02_mysql-community-libs-5.7.16-1.el7.x86_64.rpm
警告：02_mysql-community-libs-5.7.16-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-libs-5.7.16-1.el7################################# [100%]

[root@master mysql]# rpm -ivh 03_mysql-community-libs-compat-5.7.16-1.el7.x86_64.rpm
警告：03_mysql-community-libs-compat-5.7.16-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-libs-compat-5.7.1################################# [100%]

4、安装mysql-client

[root@master mysql]# rpm -ivh 04_mysql-community-client-5.7.16-1.el7.x86_64.rpm
警告：04_mysql-community-client-5.7.16-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-client-5.7.16-1.e################################# [100%]

5、安装mysql-server

[root@master mysql]# rpm -ivh 05_mysql-community-server-5.7.16-1.el7.x86_64.rpm
警告：05_mysql-community-server-5.7.16-1.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKE
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-server-5.7.16-1.e################################# [100%]

6、启动mysql

[root@master mysql]# systemctl start mysqld

7、获取临时密码

[root@master mysql]# cat /var/log/mysqld.log | grep password
2023-04-23T11:32:30.410222Z 1 [Note] A temporary password is generated for root@localhost: b7sqxrhyy_aS

8、登录

[root@master mysql]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.7.16

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

9、修改密码

注意：需要包含大/小写、数字和特殊符号。

mysql> set password=password("asdZXC@123456");
Query OK, 0 rows affected, 1 warning (0.00 sec)

10、修改user表，修改Host表内容'%',允许任意客户端访问

mysql> use mysql
mysql> update user set host="%" where user="root";
mysql> flush privileges;

11、退出

mysql> quit;

12、客户端连接测试

13、Mysql安装目录汇总

文件夹	文件夹内容
/usr/bin	客户端和脚本（mysqladmin、mysqldump 等命令）
/usr/sbin	mysqld 服务器
/var/lib/mysql	日志文件、socket 文件和数据库
/usr/share/info	信息格式的手册
/usr/share/man	UNIX 帮助页
/usr/include/mysql	头文件
/usr/lib/mysql	库
/usr/share/mysql	错误消息、字符集、安装文件和配置文件等
/etc/rc.d/init.d/	启动脚本文件的 mysql 目录，可以用来启动和停止 MySQL 服务

>>>查看一下数据库

[root@master mysql]# cd /var/lib/mysql

可以看到默认的三个库：

对照客户端如下：不过上面的路径没看到information_schema, 这个库存放在哪里？

二、Hive安装

1、上传安装包

2、解压安装到指定目录

[root@master ~]# tar -zxvf /opt/software/apache-hive-3.1.2-bin.tar.gz -C /opt/module/
[root@master ~]# mv /opt/module/apache-hive-3.1.2-bin/ /opt/module/hive

3、添加环境变量

[root@master ~]# vim /etc/profile.d/my_env.sh

#HIVE_HOME
export HIVE_HOME=/opt/module/hive
export PATH=$PATH:$HIVE_HOME/bin

4、环境变量生效

[root@master ~]# source /etc/profile.d/my_env.sh

5、解决Log日志冲突

[root@master lib]# mv log4j-slf4j-impl-2.10.0.jar log4j-slf4j-impl-2.10.0.jar.bak

6、拷贝Mysql驱动到hive lib包

[root@master lib]# cp /opt/software/mysql/mysql-connector-java-5.1.27-bin.jar /opt/module/hive/lib/

7、配置Metastore到Mysql

[root@master conf]# cd /opt/module/hive/conf
[root@master conf]# vim hive-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://master:3306/metastore?useSSL=false</value>
    </property>

    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>

    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>root</value>
    </property>

    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>asdZXC@123456</value>
    </property>

    <property>
        <name>hive.metastore.warehouse.dir</name>
        <value>/user/hive/warehouse</value>
    </property>

    <property>
        <name>hive.metastore.schema.verification</name>
        <value>false</value>
    </property>

    <property>
    <name>hive.server2.thrift.port</name>
    <value>10000</value>
    </property>

    <property>
        <name>hive.server2.thrift.bind.host</name>
        <value>master</value>
    </property>

    <property>
        <name>hive.metastore.event.db.notification.api.auth</name>
        <value>false</value>
    </property>
    
    <property>
        <name>hive.cli.print.header</name>
        <value>true</value>
    </property>

    <property>
        <name>hive.cli.print.current.db</name>
        <value>true</value>
    </property>

</configuration>

8、初始化元数据

（1）登录Mysql，创建metastore库

[root@master conf]# mysql -uroot -pasdZXC@123456

mysql> create database metastore;

mysql> quit;
Bye

（2）初始化Hive元数据库

[root@master conf]# schematool -initSchema -dbType mysql -verbose

Metastore connection URL:	 jdbc:mysql://master:3306/metastore?useSSL=false
Metastore Connection Driver :	 com.mysql.jdbc.Driver
Metastore connection User:	 root
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.mysql.sql
Connecting to jdbc:mysql://master:3306/metastore?useSSL=false
Connected to: MySQL (version 5.7.16)
Driver: MySQL Connector Java (version mysql-connector-java-5.1.27 ( Revision: alexander.soklakov@oracle.com-20131021093118-gtm1bh1vb450xipt ))
Transaction isolation: TRANSACTION_READ_COMMITTED
0: jdbc:mysql://master:3306/metastore> !autocommit on
Autocommit status: true

...
No rows affected (0.003 seconds)
0: jdbc:mysql://master:3306/metastore> !closeall
Closing: 0: jdbc:mysql://master:3306/metastore?useSSL=false
beeline> 
beeline> Initialization script completed
schemaTool completed

（3）客户端查询Hive元数据库

三、Hive登录

1、交互模式

[root@master hive]# bin/hive

>>很不幸报错，提示Name Node处于安全模式，无法创建临时目录/tmp/hive

>>手动退出安全模式即可：

[root@master hive]# hdfs dfsadmin -safemode leave
Safe mode is OFF

>>重新启动hive:

[root@master hive]# bin/hive
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/module/jdk1.8.0_212/bin:/opt/module/hadoop-3.1.3/bin:/opt/module/hadoop-3.1.3/sbin:/opt/module/hive/bin:/root/bin)
Hive Session ID = a3300960-da28-41d4-8fe7-a508bbc35514

Logging initialized using configuration in jar:file:/opt/module/hive/lib/hive-common-3.1.2.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.

hive (default)> show databases;
OK
database_name
default
Time taken: 0.966 seconds, Fetched: 1 row(s)

2、客户端模式

（1）启动metastore服务,该服务会监听9083端口

[root@master hive]# hive --service metastore 1>/dev/null 2>&1 &

[root@master hive]# netstat -nltp | grep 9083
tcp6       0      0 :::9083                 :::*                    LISTEN      3899/java

（2）启动hiveserver2服务,该服务会监听10000端口

[root@master hive]# hive --service  hiveserver2 1>/dev/null 2>&1 &

[root@master hive]# netstat -nltp | grep 10000
tcp6       0      0 :::10000                :::*                    LISTEN      4038/java

（3）客户端连接

3-1. hive 自带的客户端工具：beeline

[root@master hive]# beeline -u jdbc:hive2://master:10000 -n root
Connecting to jdbc:hive2://master:10000
Connected to: Apache Hive (version 3.1.2)
Driver: Hive JDBC (version 3.1.2)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 3.1.2 by Apache Hive

0: jdbc:hive2://master:10000> show databases;
INFO  : Compiling command(queryId=root_20230424225248_75150784-df40-404a-92c1-fa316c92be2d): show databases
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Semantic Analysis Completed (retrial = false)
INFO  : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
INFO  : Completed compiling command(queryId=root_20230424225248_75150784-df40-404a-92c1-fa316c92be2d); Time taken: 1.303 seconds
INFO  : Concurrency mode is disabled, not creating a lock manager
INFO  : Executing command(queryId=root_20230424225248_75150784-df40-404a-92c1-fa316c92be2d): show databases
INFO  : Starting task [Stage-0:DDL] in serial mode
INFO  : Completed executing command(queryId=root_20230424225248_75150784-df40-404a-92c1-fa316c92be2d); Time taken: 0.162 seconds
INFO  : OK
INFO  : Concurrency mode is disabled, not creating a lock manager
+----------------+
| database_name  |
+----------------+
| default        |
+----------------+
1 row selected (2.16 seconds)

3-2. 第三方连接工具- DataGrip