Hive的三种安装模式(内嵌模式、本地模式、远程模式)
一、内嵌模式
1、准备镜像(apache-hive-2.3.5-bin.tar.gz,下载地址https://mirrors.tuna.tsinghua.edu.cn/apache/hive/stable-2/)
2、配置环境
2.1将镜像(apache-hive-2.3.5-bin.tar.gz)上传至Linux(可以用rz命令上传)
2.2解压到自定义的指定目录module目录下
2.3配置并刷新环境变量文件
命令: vim /etc/profile
添加以下内容(根据自己的实际路径来):
刷新环境变量文件,命令:source /etc/profile
3、配置文件
3.1进入到apache-hive-2.3.5-bin.tar.gz/conf目录下,拷贝文件:
命令:cp hive-default.xml.template hive-default.xml
cp hive-default.xml.template hive-site.xml
3.2修改hive-site.xml
先创建一个临时目录(/opt/zyt/tmp),用于后面的文件配置:
命令(随便再哪个目录下执行):mkdir -p /opt/zyt/tmp
修改配置文件(4处)
用notepad++打开hive-site.xml文件,并修改(是修改,不是添加!!!),直接vim 编辑文件也行,比较麻烦:
小技巧:直接再notepad++中查找<name></name>之间的值,然后修改其对应的value
<property> <name>hive.querylog.location</name> <value>/opt/zyt/tmp.tmpdir</value> <description>Location of Hive run time structured log file</description> </property> <property> <name>hive.server2.logging.operation.log.location</name> <value>/opt/zyt/tmp.tmpdir/operation_logs</value> <description>Top level directory where operation logs are stored if logging functionality is enabled</description> </property> <property> <name>hive.exec.local.scratchdir</name> <value>/opt/zyt/tmp.tmpdir</value> <description>Local scratch space for Hive jobs</description> </property> <property> <name>hive.downloaded.resources.dir</name> <value>/opt/zyt/tmp.tmpdir/${hive.session.id}_resources</value> <description>Temporary local directory for added resources in the remote file system.</description> </property>
3.3初始化数据库
命令(应该随便在那个文件夹都行):schematool -initSchema -dbType derby
至此hive内嵌模式安装完毕,可通过hive --version查看hive版本
4、启动Hive
在此之前先启动hdfs和yarn,命令:start-all.sh
用命令jps查看进程是否正常启动了(包括以下进程NameNode、ResourceManager、DataNode、SecondaryNameNode、NodeManager)
运行hive:
[root@hadoop101 hadoop-2.9.2]# hive Logging initialized using configuration in jar:file:/home/softwares/apache-hive-2.3.4-bin/lib/hive-common-2.3.4.jar!/hive-log4j2.properties Async: true Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. hive> 创建表并测试表结构如下: hive> show tables; OK test Time taken: 26.155 seconds, Fetched: 1 row(s) hive> desc test; OK name string gender string age int Time taken: 1.041 seconds, Fetched: 3 row(s) hive>
二:本地模式
1、安装MySQL软件
命令:yum install –y mysql mysql-server
启动MySQL,命令:service mysqld start
创建MySQL账户(先以root用户登录MySQL:mysql -uroot)
创建hive用户,MySQL语句(开始用的是小写字母的语句,但是总报错,改成大写就没问题了):CREATE USER 'hive' IDENTIFIED BY 'hive';
创建hive数据库,MySQL语句:create database hive;
查看数据库,MySQL语句:show databases;
授权,MySQL语句:grant all privileges on *.* to hive@"%" identified by "hive" with grant option;
#创建hive用户同时设置远程连接(因为后面hive-site.xml中会配置成hive用户)刷新权限,MySQL语句:flush privileges;
2、下载mysql jdbc驱动包(mysql-connector-java-5.1.38-bin.jar),并上传至Linux上,然后把它复制到apache-hive-2.3.5-bin.tar.gz/lib目录下
3、配置apache-hive-2.3.5-bin.tar.gz/conf目录下的相关文件
①修改hive-site.xml文件中的URL、drivername、username、password等配置信息,修改方式同上
<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true&characterEncoding=UTF-8&useSSL=false</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>hive</value> </property>
②将hive-env.sh.template重命名为hive-env.sh,并在该文档的末尾添加环境变量
[root@hadoop101 conf]# cp hive-env.sh.template hive-env.sh 用notepad++打开hive-env.sh文件 #添加环境变量并保存,根据自己的实际路径export JAVA_HOME=/opt/module/jdk1.8.0_151 export HADOOP_HOME=/opt/module/hadoop-2.7.6 export HIVE_HOME=/opt/module/apache-hive-2.3.5-bin export HIVE_CONF_DIR=/opt/module/apache-hive-2.3.5-bin/conf
③重命名hive-log4j2.properties.template文件
[root@hadoop101 conf]# cp hive-log4j2.properties.template hive-log4j2.properties
④重命名 hive-exec-log4j2.properties.template文件
[root@hadoop101 conf]# cp hive-exec-log4j2.properties.template hive-exec-log4j2.properties
4、初始化mysql
进入到apache-hive-2.3.5-bin.tar.gz/bin目录下:
[root@hadoop101 bin]# schematool -initSchema -dbType mysql #初始化mysql
5、查看元数据库的信息
以用户hive登录MySQL(发现一个问题:用以下A方式登录会报错,用B方法可正常登录,为什么?)
A
B
查看数据库:
mysql> use hive; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +---------------------------+ | Tables_in_hive | +---------------------------+ | AUX_TABLE | | BUCKETING_COLS | | CDS | | COLUMNS_V2 | | COMPACTION_QUEUE | | COMPLETED_COMPACTIONS | | COMPLETED_TXN_COMPONENTS | | DATABASE_PARAMS | | DBS | | DB_PRIVS | | DELEGATION_TOKENS | | FUNCS | | FUNC_RU | | GLOBAL_PRIVS | | HIVE_LOCKS | | IDXS | | INDEX_PARAMS | | KEY_CONSTRAINTS | | MASTER_KEYS | | NEXT_COMPACTION_QUEUE_ID | | NEXT_LOCK_ID | | NEXT_TXN_ID | | NOTIFICATION_LOG | | NOTIFICATION_SEQUENCE | | NUCLEUS_TABLES | | PARTITIONS | | PARTITION_EVENTS | | PARTITION_KEYS | | PARTITION_KEY_VALS | | PARTITION_PARAMS | | PART_COL_PRIVS | | PART_COL_STATS | | PART_PRIVS | | ROLES | | ROLE_MAP | | SDS | | SD_PARAMS | | SEQUENCE_TABLE | | SERDES | | SERDE_PARAMS | | SKEWED_COL_NAMES | | SKEWED_COL_VALUE_LOC_MAP | | SKEWED_STRING_LIST | | SKEWED_STRING_LIST_VALUES | | SKEWED_VALUES | | SORT_COLS | | TABLE_PARAMS | | TAB_COL_STATS | | TBLS | | TBL_COL_PRIVS | | TBL_PRIVS | | TXNS | | TXN_COMPONENTS | | TYPES | | TYPE_FIELDS | | VERSION | | WRITE_SET | +---------------------------+ 57 rows in set (0.00 sec) mysql> select * from TBLS; Empty set (0.00 sec) mysql> desc TBLS; +--------------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------------+--------------+------+-----+---------+-------+ | TBL_ID | bigint(20) | NO | PRI | NULL | | | CREATE_TIME | int(11) | NO | | NULL | | | DB_ID | bigint(20) | YES | MUL | NULL | | | LAST_ACCESS_TIME | int(11) | NO | | NULL | | | OWNER | varchar(767) | YES | | NULL | | | RETENTION | int(11) | NO | | NULL | | | SD_ID | bigint(20) | YES | MUL | NULL | | | TBL_NAME | varchar(256) | YES | MUL | NULL | | | TBL_TYPE | varchar(128) | YES | | NULL | | | VIEW_EXPANDED_TEXT | mediumtext | YES | | NULL | | | VIEW_ORIGINAL_TEXT | mediumtext | YES | | NULL | | | IS_REWRITE_ENABLED | bit(1) | NO | | NULL | | +--------------------+--------------+------+-----+---------+-------+ 12 rows in set (0.00 sec) mysql>
三、远程模式