Hive 3.0.0安装与配置

机器环境是：ubuntu 18.04 LTS、hadoop 3.0.3、hive 3.0.0和mysql 8.0.11。

一、安装JDK和hadoop/mysql

1. 安装hadoop

2. 安装mysql

(1) 安装

ubuntu18.04不能直接用：

sudo apt-get install mysql-server

sudo apt-get install mysql-client

sudo apt-get install libmysqlclient-dev

因为默认安装mysql5.7不支持ubuntu18.04，正确安装如下：

http://dev.mysql.com/downloads/repo/apt/.

sudo dpkg -i mysql-apt-config_0.8.10-1_all.deb

 

 

sudo apt-get update

sudo apt-get install mysql-server

(2）权限管理

ubuntu默认账户不是root，而是debian-sys-maint，需要修改为root

sudo vi /etc/mysql/debian.cnf

mysql -u debian-sys-maint -p 

密码为debian.cnf中password

mysql> update mysql.user set authentication_string=password('root') where user='root‘；

mysql> flush privileges;

退出重启mysql

servcie mysql restart

mysql -uroot -p验证

修改mysql密码：

msyqladmin -uroot -proot password gitroot

关闭远程访问代码：

use mysql;

update user set host="localhost" where user = "root" and host = "%";

flush privileges;

开启远程访问代码：

use mysql;

update user set host = "%" where user = "root";

flush privileges;

修改max_allowed_packet

vi /etc/mysql/my.cnf

若为空，则includedir路径配置文件

彻底卸载mysql

service mysql stop
killall -KILL mysql mysqld_safe mysqld
apt-get --yes purge mysql-server mysql-client
apt-get --yes autoremove --purge
apt-get autoclean
deluser --remove-home mysql
delgroup mysql
rm -rf /etc/apparmor.d/abstractions/mysql /etc/apparmor.d/cache/usr.sbin.mysqld /etc/mysql /var/lib/mysql /var/log/mysql* /var/log/upstart/mysql.log* /var/run/mysqld

sudo apt-get purge mysql-server mysql-client mysql-common mysql-server-core-5.5 mysql-client-core-5.5
sudo rm -rf /etc/mysql /var/lib/mysql
sudo apt-get autoremove
sudo apt-get autoclean

sudo apt-get autoremove --purge mysql-server-5.5 
sudo apt-get autoremove mysql-client 
sudo apt-get autoremove mysql-server 
sudo apt-get autoremove mysql-common 

#清除残留数据和配置信息 ：

dpkg -l |grep ^rc|awk '{print $2}' |sudo xargs dpkg -P

二、配置hive文件

修改hive-env.sh

cp hive-env.sh.template hive-env.sh

vi hive-env.sh

修改内容如下：

# Set HADOOP_HOME to point to a specific hadoop install directory

HADOOP_HOME=/opt/modules/hadoop-3.0.3

# Hive Configuration Directory can be controlled by:

export HIVE_CONF_DIR=/opt/modules/hive-3.0.0/conf

export HIVE_AUX_JARS_PATH=/opt/modules/hive-3.0.0/lib

1. 配置hive-site.xml文件

cp -a conf/hive-default.xml.temple conf/hive-site.xml

vi conf/hive-site.xml

修改如下：

<property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://lee:3306/metastore?createDatabaseIfNotExist=true&amp;useSSL=false<</value>

<description>

JDBC connect string for a JDBC metastore.

To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.

For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.

</description>

</property>

<property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

<description>Driver class name for a JDBC metastore</description>

</property>

<property>

<name>javax.jdo.option.ConnectionUserName</name>

<value>root</value>

<description>Username to use against metastore database</description>

</property>

<property>

<name>javax.jdo.option.ConnectionPassword</name>

<value>root</value>

<description>password to use against metastore database</description>

</property>

备注：&useSSL=false不加会提示SSL警告，设置useSSL=true需要为服务器证书验证提供信任库。

2. hive驱动包

cp mysql-connector-java-8.0.11.jar /opt/modules/hive-3.0.0/lib/

3. 定义hive日志信息

mv hive-log4j2.properties.template hive-log4j2.properties

mkdir logs

vi hive-log4j2.properties

修改信息如下：

property.hive.log.dir = /opt/modules/hive-3.0.0/logs

4. 定义数据库名称和列名

修改hive-site.xml

<property>

<name>hive.cli.print.header</name>

<value>true</value>

<description>Whether to print the names of the columns in query output.</description>

</property>

<property>

<name>hive.cli.print.current.db</name>

<value>true</value>

<description>Whether to include the current database in the Hive prompt.</description>

</property>

5. 修改io.tmpdir

mkdir iotmp

{system:java.io.tmpdir}替换为 iotmp

shift+:命令：

%s#${system:java.io.tmpdir}#/opt/modules/hive-3.0.0/iotmp#g

%s#${system:user.name}#lee#g

6. schema.verification修改

解决报错：caused by : MEtaException(message:Version information not found in metastore)

vi hive-site.xml

<property>

<name>hive.metastore.schema.verification</name>

<value>false</value>

<description>

Enforce metastore schema version consistency.

True: Verify that version information stored in is compatible with one from Hive jars. Also disable automatic

schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures

proper metastore schema migration. (Default)

False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.

</description>

</property>

7. hive仓库目录

bin/hdfs dfs -mkdir -p /user/hive/warehouse

bin/hdfs dfs -chmode g+w /user/hive/warehouse

8. 启动hive

若出现错误：

 Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 

则数据库初始化：

bin/schematool -dbType mysql -initSchema

bin/hive