接以前hadoop安装的文章,Hive是什么就不介绍了,网上一大堆。主要介绍在FreeBSD下的安装配置和跟Linux的异同以及几种Metastore的配置方式。
Hive在FreeBSD的安装配置上,跟Linux的差别不是很大,主要要注意一个shell的问题,FreeBSD下默认是没有bash的,所以需要先安装一个bash上去才可以正常的使用Hive。
主要说说配置Metastore的部分。
Hive目前比较流行的有三种数据库当元数据管理的方式,mysql,pgsql,derby。derby属于hive默认安装就带的,无需配置,直接把hive解压缩就可以了。mysql和pgsql是目前使用最广泛的数据库。但是实际上,只要理解了hive对metastore的访问,就可以知道,只要是可以支持JDBC的关系型数据库,都可以当元数据的管理。比如Oracle或者SQL Server也可以用来存储元数据的关系表。只要你有这种JDBC的驱动就可以了。
先说下postgreSQL吧,要使用pgsql的话,当然你需要一个能够正常访问的pgsql,以及正常的访问权限。以下示例数据库名和用户名密码我都用hive,这个自己安装的时候修改一下就好了。
默认的hive从apache下载下来,解压缩之后需要设置一下环境变量HADOOP_HOME和JAVA_HOME,这个是由于hive是hadoop之上封装的应用,相当于一个扩展工具,所以对HDFS和map/reduce访问都需要使用hadoop的接口。
PostgreSQL:
需要先从这个地址下载对应你pgsql版本的jdbc驱动
然后将jar包放到$HIVE_HOME/lib/,就可以了。
将默认的hive-default.xml.template复制一份并改名为hive-site.xml,编辑这个文件,将下列部分是默认的derby访问,需要注释掉:
几种数据库配置形式是一样的。
<!--<
property
>
< name > javax.jdo.option.ConnectionURL </ name >
< value >jdbc:derby:;databaseName=metastore_db;create=true </ value >
< description >JDBC connect string for a JDBC metastore </ description >
</ property >
<!--上下两段可能不在一起,需要查找一下-->
< property >
< name > javax.jdo.option.ConnectionDriverName </ name >
< value >org.apache.derby.jdbc.EmbeddedDriver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name > javax.jdo.option.ConnectionUserName </ name >
< value > </ value >
< description >username to use against metastore database </ description >
</ property >
< name > javax.jdo.option.ConnectionURL </ name >
< value >jdbc:derby:;databaseName=metastore_db;create=true </ value >
< description >JDBC connect string for a JDBC metastore </ description >
</ property >
<!--上下两段可能不在一起,需要查找一下-->
< property >
< name > javax.jdo.option.ConnectionDriverName </ name >
< value >org.apache.derby.jdbc.EmbeddedDriver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name > javax.jdo.option.ConnectionUserName </ name >
< value > </ value >
< description >username to use against metastore database </ description >
</ property >
<
property
>
< name > javax.jdo.option.ConnectionPassword </ name >
< value > </ value >
< description >password to use against metastore database </ description >
</ property >-->
< name > javax.jdo.option.ConnectionPassword </ name >
< value > </ value >
< description >password to use against metastore database </ description >
</ property >-->
然后放入以下内容
<
property
>
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:postgresql://127.0.0.1:5432/hive? </ value >
< description >JDBC connect string for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >org.postgresql.Driver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:postgresql://127.0.0.1:5432/hive? </ value >
< description >JDBC connect string for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >org.postgresql.Driver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
这样就可以了,当你启动一个hive-cli,hive会自动创建相关的表。
然后是mysql的配置,需要注释的部分是一样的。
需要去
下载mysql的connect J,解压缩,并提取出mysql-connector-java-5.1.18-bin.jar,放到$HIVE_HOME/lib/下。
<
property
>
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true </ value >
< description >JDBC connect string for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >com.mysql.jdbc.Driver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:mysql://127.0.0.1:3306/hive?createDatabaseIfNotExist=true </ value >
< description >JDBC connect string for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >com.mysql.jdbc.Driver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
然后运行hive-cli会自动创建相关的表。
--------------------------------------
接下来是Oracle当metastore的配置
<
property
>
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:oracle:thin:@127.0.0.1:1521:hive;create=true </ value >
<!--OCI8连接方式未验证-->
< description > </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >oracle.jdbc.driver.OracleDriver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:oracle:thin:@127.0.0.1:1521:hive;create=true </ value >
<!--OCI8连接方式未验证-->
< description > </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >oracle.jdbc.driver.OracleDriver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
还有就是比较另类的SQL Server当metastore。
<
property
>
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:sqlserver://10.8.0.205:1433; DatabaseName=hive </ value >
< description > </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >com.microsoft.sqlserver.jdbc.SQLServerDriver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
< name >javax.jdo.option.ConnectionURL </ name >
< value >jdbc:sqlserver://10.8.0.205:1433; DatabaseName=hive </ value >
< description > </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionDriverName </ name >
< value >com.microsoft.sqlserver.jdbc.SQLServerDriver </ value >
< description >Driver class name for a JDBC metastore </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionUserName </ name >
< value >hive </ value >
< description >username to use against metastore database </ description >
</ property >
< property >
< name >javax.jdo.option.ConnectionPassword </ name >
< value >hive </ value >
< description >password to use against metastore database </ description >
</ property >
然后赘述一句,hive只要你保存了metastore,就可以正常的使用hive-cli,但是,如果你需要远程其他程序连接hive,就需要启动hiveserver模式,hiveserver其实是启动了一个hive的Thrift server。这是我认为hive跟传统型数据库差异比较大的一点。事实上,严格的说,hive并不能算是一个数据库,他只是一个语言转换器,把sql语言翻译成map/reduce的方式,所以他不需要具备常规数据库所必需的C/S模式。
另外,启动hive不需要hadoop进程的支持,但是需要hadoop的文件,也就是说,你只要把hadoop的文件丢在那里能正常连接hdfs就可以了,然后为hive指定HADOOP_HOME就可以使用了,不需要启动任何hadoop的进程。