1、由于hive的元数据存储在关系型数据库中,先安装mysql
解压:tar -zxvf mysql-5.7.24-linux-glibc2.12-x86_64.tar.gz
创建data文件:mkdir data #路径为/usr/local/src/mysql/data
需要创建mysql用户和用户组:
groupadd mysql
useradd -r -g mysql mysql
将安装文件和data文件修改为mysql用户:
chown -R mysql:mysql /usr/local/src/mysql
chmod -R 777 data /usr/local/src/mysql/data
修改配置:
./mysql_ssl_rsa_setup --datadir=/usr/local/src/mysql/data
vi /etc/my.cnf 修改为:(修改datadir的存储位置)
basedir=/usr/local/src/mysql/mysql
datadir=/usr/local/src/mysql/data
port = 3306
socket=/tmp/mysql.sock
pid-file=/tmp/mysqld/mysqld.pid
character-set-server = utf8
log-error=/var/log/mysqld.log
创建/tmp/mysql.sock、/tmp/mysqld/mysqld.pid、/var/log/mysqld.log,并分别修改mysql用户权限:
cd /tmp
touch mysql.sock
chown mysql:mysql mysql.sock
chmod 755 mysql.sock
cd /tmp/mysqld/
touch mysqld.pid
chown -R mysql:mysql mysqld.pid
chmod 755 mysqld.pid
cd /var/log/
touch mysqld.log
chown -R mysql:mysql /var/log
chmod 755 mysqld.log
修改环境变量:vi ~/.bash_profile
修改为:PATH=$PATH:$HOME/bin:/usr/local/src/mysql/mysql/bin
然后source ~/.bash_profile
初始化:./mysqld --initialize --user=root --basedir=/usr/local/src/mysql/mysql --datadir=/usr/local/src/mysql/data --lc_messages_dir=/usr/local/src/mysql/mysql/share --lc_messages=en_US
进入mysql的bin目录下,./mysqld_safe --user=mysql启动mysql,在/var/log/mysqld.log中找生成的临时密码:
然后mysql -uroot -p输入临时密码,即进入mysql命令终端;
设置密码,安装完成:
use mysql
update user set host=‘%‘ where user=‘root‘;
GRANT ALL PRIVILEGES ON *.* TO ‘root‘@‘%‘ IDENTIFIED BY ‘......‘ WITH GRANT OPTION;(或者是:grant all privileges on *.* to root@‘%‘ identified by "新密码";)
FLUSH PRIVILEGES;
设置开机自动启动:
cp /usr/local/src/mysql/mysql/support-files/mysql.server /etc/init.d/mysql
chkconfig --list mysql
service mysql start
service mysql stop
2、安装hive
tar -zxvf apache-hive-3.1.1-bin.tar.gz
ln -s apache-hive-3.1.1-bin hive
修改环境变量:/etc/profile,添加
#hive
export HIVE_HOME=/usr/local/src/hive/hive
export PATH=$PATH:$HIVE_HOME/bin
然后: source /etc/profile
hive --version
进入conf目录,进行设置:cd /usr/local/src/hive/hive/conf
cp hive-default.xml.template hive-site.xml
vi hive-site.xml,增加:
javax.jdo.option.ConnectionURL
jdbc:mysql://localhost:3306/hive
javax.jdo.option.ConnectionDriverName
com.mysql.jdbc.Driver
javax.jdo.option.ConnectionUserName
root
javax.jdo.option.ConnectionPassword
123456
hive.metastore.schema.verification
false
进入mysql数据库,创建hive数据:
create database hive;
进入lib目录,cd ../lib,下载mysql的连接包:
进入hive安装目录bin目录下:执行schematool -dbType mysql -initSchema
初始化成功:
[root@master bin]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/src/hive/apache-hive-3.1.1-bin/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/src/hadoop-3.0.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:mysql://localhost:3306/hive
Metastore Connection Driver : com.mysql.jdbc.Driver
Metastore connection User: root
Thu Dec 13 22:35:42 CST 2018 WARN: Establishing SSL connection without server‘s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false‘. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.mysql.sql
Thu Dec 13 22:35:43 CST 2018 WARN: Establishing SSL connection without server‘s identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to ‘false‘. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
hive自动创建的表:
mysql> show tables;
+-------------------------------+
| Tables_in_hive |
+-------------------------------+
| AUX_TABLE |
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| COMPACTION_QUEUE |
| COMPLETED_COMPACTIONS |
| COMPLETED_TXN_COMPONENTS |
| CTLGS |
| DATABASE_PARAMS |
| DBS |
| DB_PRIVS |
| DELEGATION_TOKENS |
| FUNCS |
| FUNC_RU |
| GLOBAL_PRIVS |
| HIVE_LOCKS |
| IDXS |
| INDEX_PARAMS |
........
由于已经新建了环境变量,直接输入hive进入命令终端:
出现了大量的warn信息,是由于 MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn‘t set,在hive的hive-site.conf中加上useSSL=false,即不启用SSL连接就可以:
再连接就没有了:
create database test;
然后在hdfs 上查看目录:
后续研究下 hive-site的其他配置项的含义。