mac hive mysql 配置_Mac上Hive环境搭建

本文介绍在Mac上搭建Hive环境。

建议首先配置好Hadoop，搭建与配置可以参考我之前的博文Mac Hadoop的安装与配置。

当然你也可以选择使用Docker搭建环境，本文不作介绍。

安装

对于MacOs，推荐使用HomeBrew安装hive，一步到位。

$ brew install hive

创建元数据库

Hive默认用derby作为元数据库。这里我们用mysql来存储元数据，下面作一些初始化

mysql> create database metastore;

mysql> create user 'hive'@'localhost' identified by '123456';

mysql> grant select,insert,update,delete,alter,create,index,references on metastore.* to 'hive'@'localhost';

mysql> flush privileges;

创建并修改配置

进入Hive的安装目录,创建hive-site.xml文件。(下面命令中hive路径替换为自己实际安装的路径即可)

$ cd /usr/local/Cellar/hive/2.1.1/libexec/conf

$ cp hive-default.xml.template hive-site.xml

在配置文件中找到以下几个property修改value

javax.jdo.option.ConnectionURL

jdbc:mysql://localhost/metastore

javax.jdo.option.ConnectionDriverName

com.mysql.jdbc.Driver

javax.jdo.option.ConnectionUserName

hive(填上述mysql中创建的用户名)

javax.jdo.option.ConnectionPassword

123456(填上述mysql中创建的用户密码)

hive.exec.local.scratchdir

/tmp/hive

hive.querylog.location

/tmp/hive

hive.downloaded.resources.dir

/tmp/hive

hive.server2.logging.operation.log.location

/tmp/hive

拷贝mysql-connector到hive

给Hive的lib目录下拷贝一个mysql-connector

$ curl -L 'http://www.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.42.tar.gz/from/http://mysql.he.net/' | tar xz

$ cp mysql-connector-java-5.1.42/mysql-connector-java-5.1.42-bin.jar /usr/local/Cellar/hive/2.1.1/libexec/lib/

初始化库

初始化一下metastore库

$ schematool -initSchema -dbType mysql

成功之后我们可以看到hive用usr/local/Cellar/hive/2.1.1/libexec/scripts/metastore/upgrade/mysql/hive-schema-2.1.0.mysql.sql的初始化sql脚本在我们配置的元数据库中初始化好了如下表：

mysql> show tables;

+---------------------------+

| Tables_in_metastore |

+---------------------------+

| AUX_TABLE |

| BUCKETING_COLS |

| CDS |

| COLUMNS_V2 |

| COMPACTION_QUEUE |

| COMPLETED_COMPACTIONS |

| COMPLETED_TXN_COMPONENTS |

| DATABASE_PARAMS |

| DBS |

| DB_PRIVS |

| DELEGATION_TOKENS |

| FUNCS |

| FUNC_RU |

| GLOBAL_PRIVS |

| HIVE_LOCKS |

| IDXS |

| INDEX_PARAMS |

| KEY_CONSTRAINTS |

| MASTER_KEYS |

| NEXT_COMPACTION_QUEUE_ID |

| NEXT_LOCK_ID |

| NEXT_TXN_ID |

| NOTIFICATION_LOG |

| NOTIFICATION_SEQUENCE |

| NUCLEUS_TABLES |

| PARTITIONS |

| PARTITION_EVENTS |

| PARTITION_KEYS |

| PARTITION_KEY_VALS |

| PARTITION_PARAMS |

| PART_COL_PRIVS |

| PART_COL_STATS |

| PART_PRIVS |

| ROLES |

| ROLE_MAP |

| SDS |

| SD_PARAMS |

| SEQUENCE_TABLE |

| SERDES |

| SERDE_PARAMS |

| SKEWED_COL_NAMES |

| SKEWED_COL_VALUE_LOC_MAP |

| SKEWED_STRING_LIST |

| SKEWED_STRING_LIST_VALUES |

| SKEWED_VALUES |

| SORT_COLS |

| TABLE_PARAMS |

| TAB_COL_STATS |

| TBLS |

| TBL_COL_PRIVS |

| TBL_PRIVS |

| TXNS |

| TXN_COMPONENTS |

| TYPES |

| TYPE_FIELDS |

| VERSION |

| WRITE_SET |

+---------------------------+

57 rows in set (0.00 sec)

启动hive

接下来，启动hive

$ hive

参考

《Hive编程指南》