0. 集群配置情况
注意：这里还是使用的hadoop2x版本的集群，安装了hive3.1.2，可能会有兼容性问题。
[user02@hadoop102 ~]$ start
start start-all.sh start-dfs.cmd start-pulseaudio-x11 start_udev start-yarn.cmd
start-all.cmd start-balancer.sh start-dfs.sh start-secure-dns.sh startx start-yarn.sh
[user02@hadoop102 ~]$ start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [hadoop102]
hadoop102: starting namenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-user02-namenode-hadoop102.out
hadoop103: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-user02-datanode-hadoop103.out
hadoop102: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-user02-datanode-hadoop102.out
hadoop104: starting datanode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-user02-datanode-hadoop104.out
Starting secondary namenodes [hadoop104]
hadoop104: starting secondarynamenode, logging to /opt/module/hadoop-2.7.2/logs/hadoop-user02-secondarynamenode-hadoop104.out
starting yarn daemons
starting resourcemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-user02-resourcemanager-hadoop102.out
hadoop103: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-user02-nodemanager-hadoop103.out
hadoop102: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-user02-nodemanager-hadoop102.out
hadoop104: starting nodemanager, logging to /opt/module/hadoop-2.7.2/logs/yarn-user02-nodemanager-hadoop104.out
[user02@hadoop102 ~]$ jps
3600 Jps
3474 NodeManager
3082 DataNode
2939 NameNode
[user02@hadoop102 ~]$
[user02@hadoop103 ~]$ jps
2369 NodeManager
2249 DataNode
2493 Jps
[user02@hadoop103 ~]$
[user02@hadoop104 ~]$ jps
2486 DataNode
2808 Jps
2601 SecondaryNameNode
2687 NodeManager
[user02@hadoop104 ~]$
检查了下集群本身的hive元数据的mysql
[user02@hadoop102 software]$ rpm -qa |grep mysql
mysql-libs-5.1.73-7.el6.x86_64
1. 开始安装hive3.1.2
[user02@hadoop102 ~]$ tar -zxvf apache-hive-3.1.2-bin.tar.gz -C /opt/module/
[user02@hadoop102 module]$ mv apache-hive-3.1.2-bin/ hive312
[user02@hadoop102 ~]$ sudo vim /etc/profile
添加三行
# HIVE_HOME
export HIVE_HOME=/opt/module/hive312
export PATH=$PATH:$HIVE_HOME/bin
[user02@hadoop102 ~]$ source /etc/profile
保证yarn和hdfs启动
[user02@hadoop102 ~]$ bin/hadoop fs -mkdir /tmp
[user02@hadoop102 ~]$ bin/hadoop fs -mkdir -p /user/hive/warehouse （调转到本文4.2 配置metasore到mysql）
[user02@hadoop102 ~]$ bin/hadoop fs -chmod g+w /tmp/
[user02@hadoop102 ~]$ bin/hadoop fs -chmod g+w /user/hive/warehouse
[user02@hadoop102 ~]$ hive
2. 卸载安装mysql服务端客户端
附
rpm -qa |grep mysql 检查一个安装包是否被安装，-a所有已经安装的软件包 -q查询一个包是否安装。
rpm -e --nodeps 卸载安装包，–nodeps卸载软件时，不检查依赖。这样的话，那些使用该软件包的软件在此之后可能就不能正常工作了。
rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm 安装，-ivh安装显示安装进度。
卸载
[user02@hadoop102 ~]$ rpm -qa |grep mysql
[user02@hadoop102 ~]$ sudo rpm -e --nodeps mysql-libs-5.1.73-7.el6.x86_64
error: package mysql-libs-5.1.73-7.el6.x86_64 is not installed
解压
[user02@hadoop102 software]$ ls
apache-hive-3.1.2-bin.tar.gz glibc-2.14.tar.gz hadoop-2.7.2.tar.gz mysql-libs.zip zookeeper-3.4.9.tar.gz
datax.tar.gz glibc-2.17.tar.gz jdk-8u144-linux-x64.tar.gz oracle_LINUX.X64_193000_db_home.zip
[user02@hadoop102 software]$ unzip mysql-libs.zip
Archive: mysql-libs.zip
creating: mysql-libs/
inflating: mysql-libs/MySQL-client-5.6.24-1.el6.x86_64.rpm
inflating: mysql-libs/mysql-connector-java-5.1.27.tar.gz
inflating: mysql-libs/MySQL-server-5.6.24-1.el6.x86_64.rpm
[user02@hadoop102 mysql-libs]$ ll
total 76048
-rw-rw-r--. 1 user02 user02 18509960 Mar 26 2015 MySQL-client-5.6.24-1.el6.x86_64.rpm
-rw-rw-r--. 1 user02 user02 3575135 Dec 1 2013 mysql-connector-java-5.1.27.tar.gz
-rw-rw-r--. 1 user02 user02 55782196 Mar 26 2015 MySQL-server-5.6.24-1.el6.x86_64.rpm
1. 安装Mysql服务端
[root@hadoop102 mysql-libs]# rpm -ivh MySQL-server-5.6.24-1.el6.x86_64.rpm
查看随机密码
[root@hadoop102 mysql-libs]# cat /root/.mysql_secret
# The random password set for the root user at Sun Feb 26 22:37:11 2023 (local time): kc0_hmh9f2iMmDkk
2. 安装MySQL客户端（给hive使用）
[root@hadoop102 mysql-libs]# rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm
连接mysql
[root@hadoop102 mysql-libs]# mysql -uroot -pkc0_hmh9f2iMmDkk
修改密码
mysql> set password=password('123456');
Query OK, 0 rows affected (0.00 sec)
exit退出
3. MySQL中User表中主机配置
# 1. 进入mysql
[root@hadoop102 mysql-libs]# mysql -uroot -p123456
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.24 MySQL Community Server (GPL)
Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
# 2. 显示数据库
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
+--------------------+
4 rows in set (0.00 sec)
# 3. 使用mysql数据库
mysql> use mysql;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
# 4. 查看mysql数据库里的所有表list
mysql> show tables;
+---------------------------+
| Tables_in_mysql |
+---------------------------+
| columns_priv |
| db |
| event |
| func |
| general_log |
| help_category |
| help_keyword |
| help_relation |
| help_topic |
| innodb_index_stats |
| innodb_table_stats |
| ndb_binlog_index |
| plugin |
| proc |
| procs_priv |
| proxies_priv |
| servers |
| slave_master_info |
| slave_relay_log_info |
| slave_worker_info |
| slow_log |
| tables_priv |
| time_zone |
| time_zone_leap_second |
| time_zone_name |
| time_zone_transition |
| time_zone_transition_type |
| user |
+---------------------------+
28 rows in set (0.00 sec)
# 5. 展示user表结构
mysql> desc user;
+------------------------+-----------------------------------+------+-----+-----------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------------+-----------------------------------+------+-----+-----------------------+-------+
| Host | char(60) | NO | PRI | | |
| User | char(16) | NO | PRI | | |
| Password | char(41) | NO | | | |
| Select_priv | enum('N','Y') | NO | | N | |
| Insert_priv | enum('N','Y') | NO | | N | |
| Update_priv | enum('N','Y') | NO | | N | |
| Delete_priv | enum('N','Y') | NO | | N | |
| Create_priv | enum('N','Y') | NO | | N | |
| Drop_priv | enum('N','Y') | NO | | N | |
| Reload_priv | enum('N','Y') | NO | | N | |
| Shutdown_priv | enum('N','Y') | NO | | N | |
| Process_priv | enum('N','Y') | NO | | N | |
| File_priv | enum('N','Y') | NO | | N | |
| Grant_priv | enum('N','Y') | NO | | N | |
| References_priv | enum('N','Y') | NO | | N | |
| Index_priv | enum('N','Y') | NO | | N | |
| Alter_priv | enum('N','Y') | NO | | N | |
| Show_db_priv | enum('N','Y') | NO | | N | |
| Super_priv | enum('N','Y') | NO | | N | |
| Create_tmp_table_priv | enum('N','Y') | NO | | N | |
| Lock_tables_priv | enum('N','Y') | NO | | N | |
| Execute_priv | enum('N','Y') | NO | | N | |
| Repl_slave_priv | enum('N','Y') | NO | | N | |
| Repl_client_priv | enum('N','Y') | NO | | N | |
| Create_view_priv | enum('N','Y') | NO | | N | |
| Show_view_priv | enum('N','Y') | NO | | N | |
| Create_routine_priv | enum('N','Y') | NO | | N | |
| Alter_routine_priv | enum('N','Y') | NO | | N | |
| Create_user_priv | enum('N','Y') | NO | | N | |
| Event_priv | enum('N','Y') | NO | | N | |
| Trigger_priv | enum('N','Y') | NO | | N | |
| Create_tablespace_priv | enum('N','Y') | NO | | N | |
| ssl_type | enum('','ANY','X509','SPECIFIED') | NO | | | |
| ssl_cipher | blob | NO | | NULL | |
| x509_issuer | blob | NO | | NULL | |
| x509_subject | blob | NO | | NULL | |
| max_questions | int(11) unsigned | NO | | 0 | |
| max_updates | int(11) unsigned | NO | | 0 | |
| max_connections | int(11) unsigned | NO | | 0 | |
| max_user_connections | int(11) unsigned | NO | | 0 | |
| plugin | char(64) | YES | | mysql_native_password | |
| authentication_string | text | YES | | NULL | |
| password_expired | enum('N','Y') | NO | | N | |
+------------------------+-----------------------------------+------+-----+-----------------------+-------+
43 rows in set (0.00 sec)
# 6. 查询user表
mysql> update user set host='%' where host='localhost';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
# 7. 修改user表，把Host内容修改为%
mysql> delete from user where Host='hadoop102';
Query OK, 1 row affected (0.00 sec)
# 8. 删除root用户的其他host
mysql> delete from user where Host='127.0.0.1';
Query OK, 1 row affected (0.00 sec)
mysql> delete from user where Host='::1';
Query OK, 1 row affected (0.00 sec)
# 9. 刷新
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
# 10. 退出
mysql> quit;
Bye
[root@hadoop102 mysql-libs]#
4.hive元数据配置到mysql
4.1 驱动拷贝
1）解压驱动包
[root@hadoop102 mysql-libs]# tar -zxvf mysql-connector-java-5.1.27.tar.gz
2） 拷贝mysql-connector-java-5.1.27-bin.jar 到 /opt/module/hive312/lib/
[root@hadoop102 mysql-libs]# cd mysql-connector-java-5.1.27
[root@hadoop102 mysql-connector-java-5.1.27]# cp mysql-connector-java-5.1.27-bin.jar /opt/module/hive312/lib/
4.2 配置metastore到mysql
原文档：https://cwiki.apache.org/confluence/display/Hive/AdminManual+Metastore+3.0+Administration#AdminManualMetastore3.0Administration-GeneralConfiguration
[root@hadoop102 conf]# vim hive-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<!-- jdbc 连接的 URL 前提是在mysql执行一下create database metastore;-->
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://hadoop102:3306/metastore?useSSL=false</value>
</property>
<!-- jdbc 连接的 Driver-->
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<!-- jdbc 连接的 username-->
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<!-- jdbc 连接的 password -->
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value>
</property>
<!-- Hive 元数据存储版本的验证 -->
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
<!--元数据存储授权-->
<property>
<name>hive.metastore.event.db.notification.api.auth</name>
<value>false</value>
</property>
<!-- Hive 默认在 HDFS 的工作目录 这里的value是在hdfs上主动mkdir 上文中有写 -->
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/user/hive/warehouse</value>
</property>
</configuration>
hive连接会失败，重启虚拟机后登录hive ，且show databases正常。
5. 遇到的问题
启动hive的时候报错
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.me
后面发现是mysql里数据库metastore 创建成了matestore
删除重建mysql里的metastore库，进入hive bin目录初始化schematool -dbType mysql -initSchema
重新登录hive正常执行命令。
1703
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
