Hive 安装(meta:mysql)/启动/连接beeline

(一). 前置条件

Linux 中下载安装 mysql.

0. 卸载之前安装的mysql

1.查看当前安装 mysql 情况
	rpm -qa|grep -i mysql
2.停止 mysql 服务
	service mysql stop
3.删除之前安装的 mysql
	rpm -e --nodeps 包名
4.查找旧版本 mysql 的目录及文件
	find / -name mysql
5.删除对应的mysql目录
	rm -rf 文件
6.卸载后my.cnf不会自动删除
	rm -rf /etc/my.cnf
7.查看结果
	rpm -qa|grep -i mysql

1. yum 安装 mysql

配置国内yum源:

1. cd /etc/yum.repos.d/  
2. mv CentOS-Base.repo CentOS-Base.repo.bak 
3. wget http://mirrors.163.com/.help/CentOS7-Base-163.repo 
4. yum clean all 
5. yum makecache

安装 mysql :

1. yum install -y mysql-server
2. service mysqld start
3. 直接输入mysql进入命令行, 修改用户名、密码

use mysql;
delete from user where user='';
update user set host='%' where host='localhost';
update user set password=PASSWORD('123456') where user='root';
flush privileges;

4. 退出 mysql, 重启服务, 测试连接

遇到问题:

在 centos 7 中直接使用yum install -y mysql-server报错:

[root@hadoop03 soft]# yum install -y mysql-server
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
No package mysql-server available.
Error: Nothing to do

解决方式, 运行以下命令即可:

1: wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm
2: rpm -ivh mysql-community-release-el7-5.noarch.rpm

---

(二). 安装 Hive

1. 上传、解压、配置环境变量

tar -zxvf apache-hive-2.3.7-bin.tar.gz -C /opt/

建议使用软链方式,更新软件版本时,则不需要频繁修改 /etc/profile
ln -s apache-hive-2.3.7-bin hive
export HIVE_HOME=/opt/hive
export PATH=$PATH:$HIVE_HOME/bin

source /etc/profile

2. 修改 hive 配置文件, 指定元数据库信息

touch hive-site.xml

<configuration>
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost:3306/hivemetadata?createDatabaseIfNotExist=true</value>
  <description>JDBC connect string for a JDBC metastore</description>
  <!-- hive连接mysql的连接url,同一个服务器连接的主机名最好为localhost hivemetadata hive的元数据库 -->
</property>

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>root</value>
  <description>username to use against metastore database</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>123456</value>
  <description>password to use against metastore database</description>
</property>
</configuration>

3. 将 hadoop 的 hdfs-site、core-site 添加到 hive 的 conf 目录下

cp /opt/hadoop-2.7.7/etc/hadoop/core-site.xml /opt/hadoop-2.7.7/etc/hadoop/hdfs-site.xml /opt/apache-hive-2.3.7-bin/conf/

4. 初始化hive的元数据库

1. 将 mysql 的驱动放在hive安装目录的 lib 下

cp mysql-connector-java-5.1.40-bin.jar /opt/apache-hive-2.3.7-bin/lib/

2. 格式化

schematool -dbType mysql -initSchema

完成输出: schemaTool completed
并且可以在 mysql hivemetadata 库中, 查看到初始化写入许多表

---

(三). 连接 Hive

1. hive

输入命令 hive 即可进入, 一般为了安全操作设置为 hive 本地模式 (该设置仅对当前客户端生效)

set hive.exec.mode.local.auto=true;

2. beeline

1. 前提: 启动 hiveserver2 服务

nohup hiveserver2 1>/home/hadoop/out/hive2hiveserver.log 2>/home/hadoop/out/hive2/hiveserver.err &

2. 连接 beeline 客户端: !connect jdbc:hive2://hadoop03:10000

beeline> !connect jdbc:hive2://hadoop03:10000
Connecting to jdbc:hive2://hadoop03:10000
Enter username for jdbc:hive2://hadoop03:10000: root
Enter password for jdbc:hive2://hadoop03:10000: ******
20/11/20 10:18:49 [main]: WARN jdbc.HiveConnection: Failed to connect to hadoop03:10000
Error: Could not open client transport with JDBC Uri: jdbc:hive2://hadoop03:10000: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not allowed to impersonate root (state=08S01,code=0)

报错原因: 目前 Hive 2.3.3 之后的版本, 都需要对 Hadoop 集群 hdfs-site.xml、core-site.xml 添加配置

hdfs-site.xml 添加启用 webhdfs

<property>
 <name>dfs.webhdfs.enabled</name>
 <value>true</value>
</property>

core-site.xml 添加允许任意访问

<!-- hadoop集群.代理用户.你自己需要登陆hive的用户.hosts -->
<property>
 <name>hadoop.proxyuser.root.hosts</name>
 <value>*</value>
</property>
<!-- hadoop集群.代理用户.你自己需要登陆hive的用户.groups -->
<property>
 <name>hadoop.proxyuser.root.groups</name>
 <value>*</value>
</property>

重新发送 hdfs-site.xml、core-site.xml 至 hive lib 中

cp hdfs-site.xml core-site.xml /opt/apache-hive-2.3.7-bin/conf/

重启后连接即可成功:

[root@hadoop03 hadoop]# nohup hiveserver2 1>/home/hadoop/out/hive2hiveserver.log 2>/home/hadoop/out/hive2/hiveserver.err &
[1] 44524
[root@hadoop03 hadoop]# beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-2.3.7-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 2.3.7 by Apache Hive
beeline> !connect jdbc:hive2://hadoop03:10000
Connecting to jdbc:hive2://hadoop03:10000
Enter username for jdbc:hive2://hadoop03:10000: root
Enter password for jdbc:hive2://hadoop03:10000: ******
Connected to: Apache Hive (version 2.3.7)
Driver: Hive JDBC (version 2.3.7)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://hadoop03:10000> show databases;
+----------------+
| database_name  |
+----------------+
| default        |
| wang           |
+----------------+
2 rows selected (0.568 seconds)
0: jdbc:hive2://hadoop03:10000>