centos 7 hive mysql_Centos7安装hive

所需软件及下载地址

1、mysql或mariadb

mysql下载地址

https://dev.mysql.com/downloads/mysql/5.5.html#downloads

或用mariadb也可以，直接yum安装

yum -y install mariadb-server

2、hive

http://mirrors.hust.edu.cn/apache/hive/

3、mysql-connector-java

https://dev.mysql.com/downloads/connector/j/

一、安装hadoop和mysql，这里略过

注意mysql编码要设置为Latin1，因为hive不支持utf8

在mysql中创建数据库和用户

create database hive;

grant all on hive.* to hive@'%' identified by 'hive';

grant all on hive.* to hive@'localhost' identified by 'hive';

flush privileges;

二、开始安装hive

1、解压

[root@master ~]# tar xf apache-hive-1.2.2-bin.tar.gz -C /opt/

2、修改环境变量

[root@master ]# vi /etc/profile

export HIVE_HOME=/opt/hive

export PATH=$PATH:$HIVE_HOME/bin

[root@master ]# source /etc/profile

3、修改hive-site配置，主要修改以下参数(注，\ 是转义符，因为<>显示不出)

[root@master ]# cd /opt/hive/conf

[root@master conf]# cp hive-default.xml.template hive-site.xml

[root@master conf]# vi hive-site.xml

javax.jdo.option.ConnectionURL

jdbc:mysql://localhost:3306/hive

javax.jdo.option.ConnectionDriverName

com.mysql.jdbc.Driver

javax.jdo.option.ConnectionPassword

hive

hive.hwi.listen.port

9999

This is the port the Hive Web Interface will listen on

datanucleus.autoCreateSchema

true

datanucleus.fixedDatastore

false

javax.jdo.option.ConnectionUserName

hive

Username to use against metastore database

hive.exec.local.scratchdir

/opt/hive/tmp

Local scratch space for Hive jobs

hive.downloaded.resources.dir

/opt/hive/tmp

Temporary local directory for added resources in the remote file system.

hive.querylog.location

/opt/hive/tmp

Location of Hive run time structured log file

4、拷贝mysql-connector-java-5.1.43-bin.jar 到hive 的lib下面

[root@master ~]# mv mysql-connector-java-5.1.43-bin.jar /opt/hive/lib/

5、把jline-2.12.jar拷贝到hadoop相应的目录下,否则启动会报错

[root@master ~]# cd /opt/hive/lib

[root@master lib]# cp jline-2.12.jar /opt/hadoop-2.8.1/share/hadoop/yarn/lib/

6、创建hive临时文件夹

mkdir /opt/hive/tmp

三、启动hive

1、启动hadoop后，执行hive命令

[root@master ~]#hive

2、测试

hive> show databases;

OK

default

Time taken: 0.907 seconds, Fetched: 1 row(s)

hive --service metastore

出现此提示说明安装成功

3、创建表

hive> CREATE TABLE csv_t1(index_code string,name string)

ROW FORMAT SERDE

'org.apache.hadoop.hive.serde2.OpenCSVSerde'

WITH SERDEPROPERTIES (

"separatorChar" = ",", #以逗号为分隔符

"quoteChar" = "'", #以单引号为分隔符

"escapeChar" = "\\") #以双斜杠为分隔符

STORED AS TEXTFILE;

4、导入数据

hive> load data local inpath "/usr/local/test.csv" into csv_t1;

四、报错

1、mysql编码不是latin1

FAILED: Execution Error, return code 1 from

org.apache.Hadoop.hive.ql.exec.DDLTask. MetaException

(message:javax.jdo.JDODataStoreException: An exception was

thrown while adding/validating class(es) : Specified key was too

long; max key length is 767 bytes

com.MySQL.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException:

Specified key was too long; max key length is 767 bytes

解决办法

mysql > alter database hive character set latin1;

2、READ-COMMITTED需要把bin-log以mixed方式来记录

否则进入hive，会如下错误

FAILED: Error in metadata: javax.jdo.JDOException: Couldnt

obtain a new sequence (unique id) : Binary logging not possible.

Message: Transaction level 'READ-COMMITTED' in InnoDB is not

safe for binlog mode 'STATEMENT'

解决办法

mysql >set global binlog_format='MIXED';