一、mysql安装配置
离线安装方法:
1.上传mysql安装包及mysql驱动jar包
2.卸载系统自带的mariadb
rpm -qa | grep mariadb | xargs rpm -e --nodeps
3.安装mysql及依赖包(必须按顺序装)
rpm -ivh libaio-0.3.109-13.el7.x86_64.rpm
rpm -ivh numactl-libs-2.0.12-5.el7.x86_64.rpm
rpm -ivh mysql-community-common-5.7.43-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.43-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-compat-5.7.43-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.43-1.el7.x86_64.rpm
rpm -ivh mysql-community-server-5.7.43-1.el7.x86_64.rpm
4.mysql初始化配置
1)启动mysql服务
systemctl start mysqld
2)查看mysql密码
cat /var/log/mysqld.log | grep password(mysql安装之后默认密码在日志文件里)
3)进入mysql,修改mysql密码策略,并修改密码
mysql –uroot –p’password’(password是上面一步查到的密码)
set global validate_password_policy=0;
set global validate_password_length=4;
set password=password(“123456”);
4)开启远程连接,允许所有用户连接mysql
登录到数据库中
use mysql;
update user set host=”%” where user=”root”;
flush privileges;
二、Hive安装配置(本地模式)
1.Hive安装包解压到/opt/module目录下
tar -zxvf apache-hive-3.1.2-bin.tar.gz -C /opt/module
mv apache-hive-3.1.2-bin.tar.gz hive3
2.Hive环境变量配置
vim /etc/profile
添加内容:
export HIVE_HOME=/opt/module/hive3
export PATH=$PATH:$HIVE_HOME/bin
3.hive与hadoop之间guava版本差异,删除hive中的guava,再把hadoop中的拷贝过去
cd /opt/module/hive3/lib
rm -rf guava-19.0.jar
cd /opt/module/hadoop3/share/hadoop/commom/lib
cp guava-27.0-jre.jar /opt/module/hive3/lib
4.添加mysql的jdbc驱动到hive的lib下
cp mysql-connector-java-5.1.49.jar /opt/module/hive3/lib
5.修改hive环境变量文件
cd /opt/module/hive3/conf
vim hive-env.sh
添加内容:
export HADOOP_HOME=/opt/module/hadoop3
export HIVE_CONF_DIR=/opt/module/hive3/conf
export HIVE_JARS_PATH=/opt/module/hive3/lib
6.Hive配置文件修改
新建hive-site.xml并添加内容:
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<!-- jdbc连接的URL -->
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://master:3306/hive?createDatabaseIfNotExist=true&useSSL=false&useUnicode=true&characterEncoding=UTF-8</value>
</property>
<property>
<!-- jdbc连接的Driver -->
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<!-- jdbc连接的username -->
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>
<property>
<!-- jdbc连接的password -->
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value>
</property>
<!-- 关闭元数据授权 -->
<property>
<name>hive.metastore.event.db.notification.api.auth</name>
<value>false</value>
</property>
<!-- 关闭元数据存储版本的验证 -->
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
</configuration>
7.初始化Hive数据库
schematool -initSchema -dbType mysql -verbose
8.启动
hive
测试:
show database;
show tables;
create table test(id int,name string);
insert into test values(1,”ss”);
select * from test;
注意:hadoop的mapred-site.xml中要加上以下内容:
<property>
<name>yarn.app.mapreduce.am.env</name>
<value>HADOOP_MAPRED_HOME=/opt/module/hadoop3</value>
</property>
<property>
<name>mapreduce.map.env</name>
<value>HADOOP_MAPRED_HOME=/opt/module/hadoop3</value>
</property>
<property>
<name>mapreduce.reduce.env</name>
<value>HADOOP_MAPRED_HOME=/opt/module/hadoop3</value>
</property>
Hive安装配置(远程模式)
1.其他与本地模式相同,配置文件有所不同,需加上以下内容:
<!-- H2S运行绑定host -->
<property>
<name>hive.server2.thrift.bind.host</name>
<value>master</value>
</property>
<!-- 远程模式部署metastore服务地址 -->
<property>
<name>hive.metastore.uris</name>
<value>thrift://master:9083</value>
</property>
2.手动启动metastore服务(两种方法)
hive --service metastore(该方法前台启动)
nohup hive --service metastore &(该方法后台挂起)
3.启动hive
hive