HIVE厂牌艺人_【hive】——metastore的三种模式

Hive中metastore(元数据存储)的三种方式：

内嵌Derby方式

Local方式

Remote方式

[一]、内嵌Derby方式

这个是Hive默认的启动模式，一般用于单元测试，这种存储方式有一个缺点：在同一时间只能有一个进程连接使用数据库。

hive-site.xml 中jdbc URL、驱动、用户名、密码等的配置信息如下：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

javax.jdo.option.ConnectionURL

jdbc:derby:;databaseName=metastore_db;create=true

JDBC connect string for a JDBC metastore

javax.jdo.option.ConnectionDriverName

org.apache.derby.jdbc.EmbeddedDriver

Driver class name for a JDBC metastore

javax.jdo.option.ConnectionUserName

APP

username to use against metastore database

javax.jdo.option.ConnectionPassword

mine

password to use against metastore database

hive.metastore.warehouse.dir

file:///Users/micmiu/tmp/hive/warehouse

unit test data goes in here on your local filesystem

执行初始化命令：schematool -dbType derby -initSchema

查看初始化后的信息： schematool -dbType derby -info

配置完成后就可在shell中以CLI的方式访问hive 进行操作验证。

[二]、Local方式

以本地Mysql数据库为例：创建好用户：hive；database：hive。

配置文件 hive-site.xml 中jdbc URL、驱动、用户名、密码等属性值配置如下：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

javax.jdo.option.ConnectionURL

jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true

JDBC connect string for a JDBC metastore

javax.jdo.option.ConnectionDriverName

com.mysql.jdbc.Driver

Driver class name for a JDBC metastore

javax.jdo.option.ConnectionUserName

hive

username to use against metastore database

javax.jdo.option.ConnectionPassword

micmiu

password to use against metastore database

hive.metastore.warehouse.dir

/user/hive/warehouse

location of default database for the warehouse

ps：需要把mysql的驱动包copy到目录 /lib 中

如果是第一次需要执行初始化命令：schematool -dbType mysql -initSchema

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

micmiu-mbp:mysql micmiu$schematool-dbType mysql-initSchema

14/02/1714:46:08INFO Configuration.deprecation:mapred.input.dir.recursive isdeprecated.Instead,usemapreduce.input.fileinputformat.input.dir.recursive

14/02/1714:46:08INFO Configuration.deprecation:mapred.max.split.size isdeprecated.Instead,usemapreduce.input.fileinputformat.split.maxsize

14/02/1714:46:08INFO Configuration.deprecation:mapred.min.split.size isdeprecated.Instead,usemapreduce.input.fileinputformat.split.minsize

14/02/1714:46:08INFO Configuration.deprecation:mapred.min.split.size.per.rack isdeprecated.Instead,usemapreduce.input.fileinputformat.split.minsize.per.rack

14/02/1714:46:08INFO Configuration.deprecation:mapred.min.split.size.per.node isdeprecated.Instead,usemapreduce.input.fileinputformat.split.minsize.per.node

14/02/1714:46:08INFO Configuration.deprecation:mapred.reduce.tasks isdeprecated.Instead,usemapreduce.job.reduces

14/02/1714:46:08INFO Configuration.deprecation:mapred.reduce.tasks.speculative.execution isdeprecated.Instead,usemapreduce.reduce.speculative

Metastore connection URL:jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true

Metastore Connection Driver:com.mysql.jdbc.Driver

Metastore connection User:hive

Starting metastore schema initialization to0.12.0

Initialization script hive-schema-0.12.0.mysql.sql

Initialization script completed

schemaTool completeted

查看初始化后信息 schematool -dbType mysql -info

初始化后查看mysql中表情况：show tables;

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

mysql>show tables;

+---------------------------+

|Tables_in_hive|

+---------------------------+

|BUCKETING_COLS|

|CDS|

|COLUMNS_V2|

|DATABASE_PARAMS|

|DBS|

|DB_PRIVS|

|DELEGATION_TOKENS|

|GLOBAL_PRIVS|

|IDXS|

|INDEX_PARAMS|

|MASTER_KEYS|

|NUCLEUS_TABLES|

|PARTITIONS|

|PARTITION_EVENTS|

|PARTITION_KEYS|

|PARTITION_KEY_VALS|

|PARTITION_PARAMS|

|PART_COL_PRIVS|

|PART_COL_STATS|

|PART_PRIVS|

|ROLES|

|ROLE_MAP|

|SDS|

|SD_PARAMS|

|SEQUENCE_TABLE|

|SERDES|

|SERDE_PARAMS|

|SKEWED_COL_NAMES|

|SKEWED_COL_VALUE_LOC_MAP|

|SKEWED_STRING_LIST|

|SKEWED_STRING_LIST_VALUES|

|SKEWED_VALUES|

|SORT_COLS|

|TABLE_PARAMS|

|TAB_COL_STATS|

|TBLS|

|TBL_COL_PRIVS|

|TBL_PRIVS|

|TYPES|

|TYPE_FIELDS|

|VERSION|

+---------------------------+

41rows inset(0.00sec)

配置完成后就可在shell中以CLI的方式访问hive 进行操作验证。

[三]、Remote方式

以Mysql数据库(192.168.6.77)为例：创建好用户：hive；database：hive_meta。Remote方式需要分别配置服务端和客户端的配置文件：

服务端的 hive-site.xml 中jdbc URL、驱动、用户名、密码等属性值配置如下：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

javax.jdo.option.ConnectionURL

jdbc:mysql://192.168.6.77/hive_meta?createDatabaseIfNotExist=true

JDBC connect string for a JDBC metastore

javax.jdo.option.ConnectionDriverName

com.mysql.jdbc.Driver

Driver class name for a JDBC metastore

javax.jdo.option.ConnectionUserName

hive

username to use against metastore database

javax.jdo.option.ConnectionPassword

micmiu

password to use against metastore database

hive.metastore.warehouse.dir

/user/hive/warehouse

base hdfs path :location of default database for the warehouse

ps：需要把mysql的驱动包copy到目录 /lib 中

如果是第一次需要执行初始化命令：schematool -dbType mysql -initSchema

客户端中配置内容修改如下：

1

2

3

4

5

6

7

8

9

10

11

12

13

hive.metastore.uris

thrift://192.168.6.77:9083

Thrift uri for the remote metastore. Used by metastore client to connect to remote metastore.

hive.metastore.warehouse.dir

/user/hive/warehouse

location of default database for the warehouse

hive metastore 服务端启动命令：

hive --service metastore -p

如果不加端口默认启动：hive --service metastore，则默认监听端口是：9083 ，注意客户端中的端口配置需要和启动监听的端口一致。服务端启动正常后，客户端就可以执行hive操作了。

参考：

https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin