Hadoop 之Hive安装

1.下载软件包

[root@master src]# wget http://mirror.bit.edu.cn/apache/hive/hive-1.2.2/apache-hive-1.2.2-bin.tar.gz
--2020-12-13 17:34:29--  http://mirror.bit.edu.cn/apache/hive/hive-1.2.2/apache-hive-1.2.2-bin.tar.gz
正在解析主机 mirror.bit.edu.cn (mirror.bit.edu.cn)... 114.247.56.117, 2001:da8:204:1205::22
正在连接 mirror.bit.edu.cn (mirror.bit.edu.cn)|114.247.56.117|:80... 已连接。
已发出 HTTP 请求,正在等待回应... 200 OK
长度:90859180 (87M) [application/octet-stream]
正在保存至: “apache-hive-1.2.2-bin.tar.gz”

100%[====================================================================================================================================================>] 90,859,180  1.28MB/s 用时 58s    

2020-12-13 17:35:28 (1.49 MB/s) - 已保存 “apache-hive-1.2.2-bin.tar.gz” [90859180/90859180])

[root@master src]# ls
apache-hive-1.2.2-bin.tar.gz  hadoop-2.6.5  hadoop-2.6.5.tar.gz  jdk1.8.0_172  jdk-8u172-linux-x64.tar.gz  test

2.解压

tar zxvf apache-hive-2.3.3-bin.tar.gz

3.创建HDSF文件目录

[root@master src]# hadoop fs -mkdir -p /data/hive/warehouse
[root@master src]# hadoop fs -mkdir -p /data/hive/tmp
[root@master src]# hadoop fs -mkdir -p /data/hive/log
[root@master src]# hadoop fs -ls /
Found 4 items
-rw-r--r--   2 root supergroup       1005 2020-12-07 14:57 /core-site.xml
drwxr-xr-x   - root supergroup          0 2020-12-13 17:41 /data
drwxr-xr-x   - root supergroup          0 2020-12-08 11:30 /dzw
drwxr-xr-x   - root supergroup          0 2020-12-08 11:30 /test
[root@master src]# hadoop fs -ls /data
Found 1 items
drwxr-xr-x   - root supergroup          0 2020-12-13 17:41 /data/hive
[root@master src]# hadoop fs -ls /data/hive
Found 3 items
drwxr-xr-x   - root supergroup          0 2020-12-13 17:41 /data/hive/log
drwxr-xr-x   - root supergroup          0 2020-12-13 17:41 /data/hive/tmp
drwxr-xr-x   - root supergroup          0 2020-12-13 17:41 /data/hive/warehouse
[root@master src]# hadoop fs -chmod -R 777 /data/hive/warehouse
[root@master src]# hadoop fs -ls /data/hive                    
Found 3 items
drwxr-xr-x   - root supergroup          0 2020-12-13 17:41 /data/hive/log
drwxr-xr-x   - root supergroup          0 2020-12-13 17:41 /data/hive/tmp
drwxrwxrwx   - root supergroup          0 2020-12-13 17:41 /data/hive/warehouse
[root@master src]# hadoop fs -chmod -R 777 /data/hive/tmp
[root@master src]# hadoop fs -chmod -R 777 /data/hive/log
[root@master src]# hadoop fs -ls /data/hive              
Found 3 items
drwxrwxrwx   - root supergroup          0 2020-12-13 17:41 /data/hive/log
drwxrwxrwx   - root supergroup          0 2020-12-13 17:41 /data/hive/tmp
drwxrwxrwx   - root supergroup          0 2020-12-13 17:41 /data/hive/warehouse

4.创建文件系统目录

[root@master src]# mkdir -p /data/hive/tmp

5.修改Hive配置文件

[root@master /]# cd /usr/local/src
[root@master src]# ls
apache-hive-1.2.2-bin  apache-hive-1.2.2-bin.tar.gz  hadoop-2.6.5  hadoop-2.6.5.tar.gz  jdk1.8.0_172  jdk-8u172-linux-x64.tar.gz  test
[root@master src]# cd apache-hive-1.2.2-bin/
[root@master apache-hive-1.2.2-bin]# ls
bin  conf  examples  hcatalog  lib  LICENSE  NOTICE  README.txt  RELEASE_NOTES.txt  scripts
[root@master apache-hive-1.2.2-bin]# cd conf
[root@master conf]# ls
beeline-log4j.properties.template  hive-default.xml.template  hive-env.sh.template  hive-exec-log4j.properties.template  hive-log4j.properties.template  ivysettings.xml
[root@master conf]# cp hive-default.xml.template hive-site.xml
[root@master conf]# ls
beeline-log4j.properties.template  hive-default.xml.template  hive-env.sh.template  hive-exec-log4j.properties.template  hive-log4j.properties.template  hive-site.xml  ivysettings.xml
[root@master conf]# 

6,编辑hive-site.xml
在原有代码之前插入

<configuration>
    <property>
        <name>hive.exec.scratchdir</name>
        <value>/data/hive/tmp</value>
    </property>
    <property>
        <name>hive.querylog.location</name>
        <value>/data/hive/log/hadoop</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionURL</name>
        <value>jdbc:mysql://master:3306/hive?createDatabaseIfNotExist=true</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionDriverName</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionUserName</name>
        <value>hive</value>
    </property>
    <property>
        <name>javax.jdo.option.ConnectionPassword</name>
        <value>hive</value>
    </property>
</configuration>

7.配置hive-env.sh

[root@master conf]# cp hive-env.sh.template hive-env.sh
[root@master conf]# vim hive-env.sh

加入配置

HADOOP_HOME=/usr/local/src/hadoop-2.6.5
HIVE_CONF_DIR=/usr/local/src/apache-hive-1.2.2-bin/conf
HIVE_AUX_JARS=/usr/local/src/apache-hive-1.2.2-bin/lib

8.增加环境变量

export PATH=$PATH:$HADOOP_HOME/bin:$HIVE_HOME/bin
export HIVE_HOME=/usr/local/src/apache-hive-1.2.2-bin

#刷新环境变量
source ~/.bashrc

9.安装Mysql/MariaDB数据库(yum方式安装)

[root@master src]# sudo rpm -Uvh http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
获取http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
准备中...                          ################################# [100%]
正在升级/安装...
   1:mysql-community-release-el7-5    ################################# [100%]
[root@master src]# yum install mysql-server
已加载插件:fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirrors.njupt.edu.cn
 * extras: mirrors.cn99.com
 * updates: mirrors.cn99.com
mysql-connectors-community                                                                                                                                             | 2.6 kB  00:00:01     
mysql-tools-community                                                                                                                                                  | 2.6 kB  00:00:00     
mysql56-community                                                                                                                                                      | 2.6 kB  00:00:01     
(1/3): mysql56-community/x86_64/primary_db                                                                                                                             | 288 kB  00:00:01     
(2/3): mysql-tools-community/x86_64/primary_db                                                                                                                         |  83 kB  00:00:02     
(3/3): mysql-connectors-community/x86_64/primary_db                                                                                                                    |  68 kB  00:00:02     
正在解决依赖关系
--> 正在检查事务
---> 软件包 mysql-community-server.x86_64.0.5.6.50-2.el7 将被 安装
--> 正在处理依赖关系 mysql-community-common(x86-64) = 5.6.50-2.el7,它被软件包 mysql-community-server-5.6.50-2.el7.x86_64 需要
--> 正在处理依赖关系 mysql-community-client(x86-64) >= 5.6.10,它被软件包 mysql-community-server-5.6.50-2.el7.x86_64 需要
--> 正在检查事务
---> 软件包 mysql-community-client.x86_64.0.5.6.50-2.el7 将被 安装
--> 正在处理依赖关系 mysql-community-libs(x86-64) >= 5.6.10,它被软件包 mysql-community-client-5.6.50-2.el7.x86_64 需要
---> 软件包 mysql-community-common.x86_64.0.5.6.50-2.el7 将被 安装
--> 正在检查事务
---> 软件包 mariadb-libs.x86_64.1.5.5.56-2.el7 将被 取代
---> 软件包 mysql-community-libs.x86_64.0.5.6.50-2.el7 将被 舍弃
--> 解决依赖关系完成

依赖关系解决

==============================================================================================================================================================================================
 Package                                              架构                                 版本                                         源                                               大小
==============================================================================================================================================================================================
正在安装:
 mysql-community-libs                                 x86_64                               5.6.50-2.el7                                 mysql56-community                               2.2 M
      替换  mariadb-libs.x86_64 1:5.5.56-2.el7
 mysql-community-server                               x86_64                               5.6.50-2.el7                                 mysql56-community                                67 M
为依赖而安装:
 mysql-community-client                               x86_64                               5.6.50-2.el7                                 mysql56-community                                21 M
 mysql-community-common                               x86_64                               5.6.50-2.el7                                 mysql56-community                               287 k

事务概要
==============================================================================================================================================================================================
安装  2 软件包 (+2 依赖软件包)

总下载量:90 M
Is this ok [y/d/N]: y
Downloading packages:
警告:/var/cache/yum/x86_64/7/mysql56-community/packages/mysql-community-common-5.6.50-2.el7.x86_64.rpm: 头V3 DSA/SHA1 Signature, 密钥 ID 5072e1f5: NOKEY   ]  96 kB/s | 549 kB  00:15:49 ETA 
mysql-community-common-5.6.50-2.el7.x86_64.rpm 的公钥尚未安装
(1/4): mysql-community-common-5.6.50-2.el7.x86_64.rpm                                                                                                                  | 287 kB  00:00:04     
(2/4): mysql-community-libs-5.6.50-2.el7.x86_64.rpm                                                                                                                    | 2.2 MB  00:00:17     
(3/4): mysql-community-client-5.6.50-2.el7.x86_64.rpm                                                                                                                  |  21 MB  00:01:45     
(4/4): mysql-community-server-5.6.50-2.el7.x86_64.rpm                                                                                                                  |  67 MB  00:04:54     
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
总计                                                                                                                                                          290 kB/s |  90 MB  00:05:17     
从 file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql 检索密钥
导入 GPG key 0x5072E1F5:
 用户ID     : "MySQL Release Engineering <mysql-build@oss.oracle.com>"
 指纹       : a4a9 4068 76fc bd3c 4567 70c8 8c71 8d3b 5072 e1f5
 软件包     : mysql-community-release-el7-5.noarch (installed)
 来自       : file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql
是否继续?[y/N]:y
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
警告:RPM 数据库已被非 yum 程序修改。
  正在安装    : mysql-community-common-5.6.50-2.el7.x86_64                                                                                                                                1/5 
  正在安装    : mysql-community-libs-5.6.50-2.el7.x86_64                                                                                                                                  2/5 
  正在安装    : mysql-community-client-5.6.50-2.el7.x86_64                                                                                                                                3/5 
  正在安装    : mysql-community-server-5.6.50-2.el7.x86_64                                                                                                                                4/5 
  正在删除    : 1:mariadb-libs-5.5.56-2.el7.x86_64                                                                                                                                        5/5 
  验证中      : mysql-community-common-5.6.50-2.el7.x86_64                                                                                                                                1/5 
  验证中      : mysql-community-libs-5.6.50-2.el7.x86_64                                                                                                                                  2/5 
  验证中      : mysql-community-client-5.6.50-2.el7.x86_64                                                                                                                                3/5 
  验证中      : mysql-community-server-5.6.50-2.el7.x86_64                                                                                                                                4/5 
  验证中      : 1:mariadb-libs-5.5.56-2.el7.x86_64                                                                                                                                        5/5 

已安装:
  mysql-community-libs.x86_64 0:5.6.50-2.el7                                                   mysql-community-server.x86_64 0:5.6.50-2.el7                                                  

作为依赖被安装:
  mysql-community-client.x86_64 0:5.6.50-2.el7                                                  mysql-community-common.x86_64 0:5.6.50-2.el7                                                 

替代:
  mariadb-libs.x86_64 1:5.5.56-2.el7                                                                                                                                                          

完毕!

10.将mysql的jar包放在hive的lib目录下面

11.启动MariaDB

systemctl start mariadb

12.首次登录无密码直接登录

mysql -u root -p

13.修改密码

mysqladmin -u root password

14.登录验证

[root@master ~]# mysql -u root -p
Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 12
Server version: 5.5.68-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.01 sec)

16.拷贝Hive至slave节点

scp -r /usr/local/src/apache-hive-1.2.2-bin root@slave1:/usr/local/src/apache-hive-1.2.2-bin
scp -r /usr/local/src/apache-hive-1.2.2-bin root@slave2:/usr/local/src/apache-hive-1.2.2-bin

17.启动Hive遇见报错

[root@master ~]# $HIVE_HOME/bin/hive

Logging initialized using configuration in jar:file:/usr/local/src/apache-hive-1.2.2-bin/lib/hive-common-1.2.2.jar!/hive-log4j.properties
[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
        at jline.TerminalFactory.create(TerminalFactory.java:101)
        at jline.TerminalFactory.get(TerminalFactory.java:158)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

解决方案
将hadoop下面的jline-0.9.94.jar删除,因为和hive中的jline产生冲突。删除之后启动hive

启动Hive成功

[root@master lib]# $HIVE_HOME/bin/hive

Logging initialized using configuration in jar:file:/usr/local/src/apache-hive-1.2.2-bin/lib/hive-common-1.2.2.jar!/hive-log4j.properties
hive> select 10;
OK
10
Time taken: 2.408 seconds, Fetched: 1 row(s)
  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值