Hive 3.1.2安装部署

最新推荐文章于 2024-06-18 18:18:57 发布

只会偷学的小莫

最新推荐文章于 2024-06-18 18:18:57 发布

阅读量431

点赞数 2

文章标签： hive hadoop 数据仓库

本文链接：https://blog.csdn.net/qikuhu/article/details/133014935

版权

1.4 删除hive jar包复制hadoop jar包

1.5 修改hive环境配置

1.6 初始化metadata

1.7启动hive（启动前先启动hadoop）

1 内嵌模式部署

1.1 下载hive安装包

wget https://www.strategylions.com.au/mirror/hive/hive-3.1.2/apache-hive-3.1.2-bin.tar.gz

1.2 解压安装包

tar -xvf apache-hive-3.1.2-bin.tar.gz
mv apache-hive-3.1.2-bin/ /export/servers/hive

1.3 设置环境变量

vim /etc/profile
export HIVE_HOME=/export/servers/hive
export PATH=:$HIVE_HOME/bin:$PATH

source /etc/profile

1.4 删除hive jar包复制hadoop jar包

rm -rf lib/guava-19.0.jar
cp /export/servers/hadoop-3.3.5/share/hadoop/common/lib/guava-27.0-jre.jar ./lib/

1.5 修改hive环境配置

cd /export/servers/hive/conf/
mv hive-env.sh.template hive-env.sh
vim hive-env.sh
export HADOOP_HOME=/export/servers/hadoop-3.3.5
export HIVE_CONF_DIR=/export/servers/hive/conf
export HIVE_AUX_JARS_PATH=/export/servers/hive/lib

1.6 初始化metadata

#初始化metadata
cd /export/server/hive
bin/schematool -dbType derby -initSchema

1.7启动hive（启动前先启动hadoop）

如果提示Hadoop在安全模式就使用以下命令
hadoop dfsadmin -safemode leave
bin/hive

2.本地模式

2.1 mysql安装

Hive使用MySQL作为元数据存储，必然需要连接mysql数据库，所以我们添加一个MySQL的连接驱动包到Hive的安装目录下，然后就可以准备启动Hive了；将我们准备好的mysql-connector-java-5.1.32.jar这个jar包直接上传到/apps/hive/lib 这个目录下即可

#下载mysql

wget https://mvnrepository.com/artifact/mysql/mysql-connector-java/5.1.32
#下载驱动
wget https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.22/mysql-connector-java-8.0.22.jar
把数据库驱动放到$HIVE_HOME/lib目录下
#创建mysql安装包存放点
mkdir /export/software/mysql
#上传mysql-5.7.29安装包到上述文件夹下、解压
tar xvf mysql-5.7.29-1.el7.x86_64.rpm-bundle.tar

#执行安装
yum -y install libaio
rpm -ivh mysql-community-common-5.7.29-1.el7.x86_64.rpm mysql-community-libs-5.7.29-1.el7.x86_64.rpm mysql-community-client-5.7.29-1.el7.x86_64.rpm mysql-community-server-5.7.29-1.el7.x86_64.rpm

2.2 初始化mysql

mysqld --initialize
#更改所属组
chown mysql:mysql /var/lib/mysql -R

#启动mysql
systemctl start mysqld.service

#查看生成的临时root密码
cat /var/log/mysqld.log

#这行日志的最后就是随机生成的临时密码
[Note] A temporary password is generated for root@localhost: o+TU+KDOm004

#修改mysql root密码、授权远程访问
mysql -u root -p
Enter password: #这里输入在日志中生成的临时密码

#更新root密码设置为hadoop
mysql> alter user user() identified by "hadoop";
Query OK, 0 rows affected (0.00 sec)

#授权
mysql> use mysql;
mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'hadoop' WITH GRANT OPTION;
mysql> FLUSH PRIVILEGES;

#mysql的启动和关闭状态查看
systemctl stop mysqld
systemctl status mysqld
systemctl start mysqld

#建议设置为开机自启动服务
systemctl enable mysqld

#查看是否已经设置自启动成功
systemctl list-unit-files | grep mysqld

2.3 hive环境配置

修改hive-site.xml文件

<configuration>

<property>
<name>javax.jdo.option.ConnectionURL</name>
<value> jdbc:mysql://hadoop01（主机名）:3306/hive?createDatabaseIfNotExist=true&useSSL=false&useUnicode=true&characterEncoding=UTF-8</value>
</property>

<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>

<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
</property>

<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>hadoop</value>
</property>


<property>
<name>hive.metastore.event.db.notification.api.auth</name>
<value>false</value>
</property>


<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
</property>
</configuration>

2.4 设置hive hdfs路径

hdfs dfs -mkdir /tmp
hdfs dfs -mkdir -p /user/hive/warehouse
hdfs dfs -chmod g+w /tmp
hdfs dfs -chmod g+w /user/hive/warehouse

2.5 初始化数据库（二选一）

cd $HIVE_HOME/bin/
./metatool -dbType mysql -initSchema

mysql> use hive_metastore
cd /data/soft/apache-hive-3.1.2-bin/scripts/metastore/upgrade/mysql
SOURCE /data/soft/apache-hive-3.1.2-bin/scripts/metastore/upgrade/mysql/hive-schema-3.1.0.mysql.sql;