hive 安装步骤以及问题解决方案

1.安装hive

本人前前后后找了很多hive安装文档,按照文档方式安装踩了无数的坑,前人的轮子很多, 因此大家自己实践的时候,主要任务是搜索靠谱有效的文档,如果是mac系统的话参考一下主要连接 https://luckymrwang.github.io/2018/03/14/Install-hive-on-Mac-with-Homebrew/ 就行,注意按照上面方式操作后,需要创建mysql下的hive数据库命令,文章漏掉了这一步.

cd  /usr/local/Cellar/hive/3.1.2_3/libexec/bin

schematool -dbType mysql -initSchema

或者这篇 http://dblab.xmu.edu.cn/blog/2440-2/

2.启动hive

1. 先启动hadoop 

2.在启动hive

 

3.报错

1.比较难解决的问题 是最后运行schematool -dbType mysql -initSchema  初始化hive时候报 Unable to load authentication plugin 'caching_sha2_password'.

schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hive/3.1.2_2/libexec/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/Cellar/hadoop/3.3.0/libexec/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :	 com.mysql.jdbc.Driver
Metastore connection User:	 hadoop
Mon Jan 25 14:51:11 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: java.sql.SQLException : Unable to load authentication plugin 'caching_sha2_password'.
SQL Error code: 0
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

解决方案:https://www.cnblogs.com/feiquan/p/13691980.html 按照这个文档的方案解决了

原因：

hive/lib 下使用的jar[mysql-connector-java-5.1.42-bin.jar] 包和MySQL 版本不统一

主要原因8.x版本的验证模块和之前版本不同：

　　5.x版本是：default_authentication_plugin=mysql_native_password

　　8.x版本就是：default_authentication_plugin=caching_sha2_password

解决：

hive/lib 下换用新jar 包: mysql-connector-java-8.0.15.jar   

cp ~/Downloads/mysql-connector-java-8.0.15/mysql-connector-java-8.0.15.jar libexec/lib

 

2.hive运行时候一直提示

Mon Jan 25 21:22:00 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.

 

Unexpected character '=' (code 61); expected a semi-colon after the reference for entity 'characterE

tips: 

1. Xml文件中不能使用&，要使用他的转义&来代替。

2.mysql 的Java 连接 各个版本下载 https://blog.csdn.net/weixin_41804049/article/details/87719574,可以替换 /usr/local/Cellar/hive/3.1.2_2/libexec/lib  下面的mysql-connector-java-8.0.23  没有和安装的mysql版本一直时候,可以替换

3.mac mysql 常用启安装和卸载操作

1.安装
brew install mysql   -- 默认最新版本

启动mqsql 
brew  services  start  mysql  安装后启动下
brew  services  stop   mysql

2.卸载
brew安装的mysql默认在 usr/local/Celler 目录下面很方便,建议可以brew 方式安装软件

先停止mysql服务
brew  services  stop   mysql

卸载
brew uninstall mysql   

4. 运行hadoop 命令 显示 Invalid HADOOP_COMMON_HOME

WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
ERROR: Invalid HADOOP_COMMON_HOME

解决方案: 这个问题我在百度找了一晚上没有什么解决方案,一度怀疑人生,差点吧hadoop的配置文件都看了一遍, 后来在google 上搜到了答案, 如下第六个问题, 一般打不开贴上问题解决方案. 因为我安装的hadoop3.0以上的版本, 可能低版本的没有这个问题,意思是说hadoop在v3 以上不需要在bash_profile 上配置HADOOP_HOME路径,我在百度上搜的都是3.0以下的配置, 以后的同学再安装hadoop注意啊,看清上下文.

连接:Complete Apache Hadoop Troubleshooting | by Pete Houston | Medium https://medium.com/@petehouston/complete-apache-hadoop-troubleshooting-660122eac6a5

Question 5: After setting up Apache Hadoop, any hadoop command issued shows following error : ERROR: Invalid HADOOP_COMMON_HOME. What happen?
If I’m not mistaken, it looks like you follow wrong or out-of-date Hadoop installation tutorial. Commonly, prior to Hadoop v3, installation often requires to export HADOOP_HOME environment. However, this is a NO-NO for Hadoop v3.

The solution is to drop that HADOOP_HOME, which means:

$ export HADOOP_HOME=
Unset it, then it’s done.

If you’re on a Mac and problem still happens, try to follow my guide on setting up Apache Hadoop on Mac.

5.mysql 运行报错 mysql ERROR 2002 (HY000): '/tmp/mysql.sock'

说明问题:搜了很多博客,都是修改配置文件,都不好使, 只能卸载重新装了,主要是删除 这个目录下面的内容 

sudo rm -rf /usr/local/var/mysql

解决参考链接: https://cloud.tencent.com/developer/article/1468919

多看看官网: https://dev.mysql.com/doc/refman/5.7/en/data-directory-initialization.html

6.Hadoop: connect to host localhost port 22: Connection refused when running start-dfs.sh

在mac环境下打开网络连接,即在偏好设置中-> 共享(share) -> 远程连接勾选

7. hive 启动是报 Access denied for user 'root'@'localhost' (using password: YES) 

问题原因:
根据报错显示密码正确,但是权限不够,因此主要目的是解决权限问题

解决方案:
1.root 方式登录msyql
2.给hive用户增加权限
 GRANT INSERT, SELECT, DELETE, UPDATE ON hive.* TO 'hive'@'localhost' IDENTIFIED BY '123456';
flush privileges; 刷新数据库

参考连接: https://stackoverflow.com/questions/20353402/access-denied-for-user-testlocalhost-using-password-yes-except-root-user

8.ssl 认证问题  https://blog.csdn.net/qq_41785135/article/details/85118329

9.Relative path in absolute URI: 问题  https://blog.csdn.net/fhg12225/article/details/45817477

10. 找不到主类: https://blog.csdn.net/qq_29232943/article/details/103788404

4.总结

   1.安装hive 的过程中很容易出现找不到解决问题的情况,因为放假自己安装没有人可以请教, 经常陷入死胡同, 杀手锏就是 卸载再来一遍. 2.安装过程中发现自己对hive sql还是停留在使用上, 不知道底层原理,对各种报错,查询文档,不知所以然,基本功还是不好,需要把hive的官方文档好好阅读.  3.最重要的遇到问题不要急,学会仔细看报错日志,留意关键错误信息.