4.Spark大型电商项目-大数据环境搭建之数据仓库hive安装

目录

实验环境

安装hive

安装mysql

配置文件

测试hive是否安装成功

---

本文主要介绍在之前搭建的集群基础上安装hive的详细过程。

实验环境

hive版本：hive 3.1.1

版本可以从hive官网上下载需要的版本：https://hive.apache.org/downloads.html

安装hive

本人安装包是下载到了/下载这个文件夹下

加压缩到/usr/local/文件夹下

sudo tar -zxvf apache-hive-1.2.2-bin.tar.gz -C /usr/local/

hive文件重命名

sudo mv apache-hive-1.2.2-bin hive

文件夹权限修改

sudo chown -R hadoop:hadoop ./hive

配置环境变量

sudo vim ~/.bashrc

export HIVE_HOME=/usr/local/hive

export PATH=$PATH:$HIVE_HOME/bin

export HIVE_CONF_DIR=$HIVE_HOME/conf

配置生效

source ~/.bashrc

安装mysql

在master节点上安装mysql

sudo  apt-get install mysql-server

启动mysql

service mysqld start

chkconfig mysqld on

chkconfig mysql on 是设置开机启动

将mysql 中的connector jar包拷贝到hive中lib文件夹下

cp mysql-connector-java-5.1.46-bin.jar /usr/local/hive/lib/
 

在mysql上创建hive元数据库，创建hive账号，并进行授权

进入mysql

mysql -uroot -p

创建hive数据库

create database if not exists hive;

赋值权限 

grant all privileges on hive.* to 'hive'@'%' identified by 'hive';
grant all privileges on hive.* to 'hive'@'localhost' identified by 'hive';
grant all privileges on hive.* to 'hive'@'Master' identified by 'hive';

刷新权限 

flush privileges;

进入hive数据库

 
use hive;
 

配置文件

配置hive-site.xml

cd /usr/local/hive/conf

mv hive-default.xml.template hive-site.xml

sudo gedit hive-stie.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>username to use against metastore database</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive</value>
    <description>password to use against metastore database</description>
  </property>
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
  </property>
  <property>
   <name>hive.metastore.schema.verification</name>
   <value>false</value>
    <description>
    Enforce metastore schema version consistency.
    True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic
          schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures
          proper metastore schema migration. (Default)
    False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
    </description>
 </property>
</configuration>

配置hive-env.sh

cp hive-env.sh.template hive-env.sh

配置hive-env.sh

sudo vim /usr/local/hive/conf/hive-env.sh

export JAVA_HOME=/usr/lib/jvm/default-java

 export HIVE_HOME=/usr/local/hive

export HADOOP_HOME=/usr/local/hadoop

测试hive是否安装成功

启动hive

hive  --service metastore &

hive

查看数据库

show databases;

创建hive数据库；

create  databases hive;

进入hive数据库

use hive;

创建一张表，并将users.txt中的数据load进去

create table users(id int, name string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' ;

load data local inpath '/usr/local/users.txt' into table users;

users.txt中的数据

1    Jack
2    Mike
3    Tim
4    Hare
5    Nker

 

查看数据表

select  * from users;

select name from users;

退出hive

exit