hadoop2.2.0的基础上配置hive0.12.0(支持mysql)

前提:已经安装好了hadoop2.2.0,当然也可以是其他版本的hadoop,作者使用的是这个版本,并在vmware上配了一个master和3个slaver。配置方式参考:

http://blog.csdn.net/glgl2424/article/details/24868273

 

一、下载hive-0.12.0,并解压到/usr/hive中

二、将hive-default.xml.template拷贝一份并重命名为hive-site.xml,同样,将hive-env.sh.template拷贝一份并重命名为hive-env.sh

三、修改2个配置文件,env,sh主要是修改环境变量,而-site.xml主要是修改hive的元数据库配置(本文以配置mysql作为元数据)

关键配置如下:(-site.xml)

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>


<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
  <description>username to use against metastore database</description>
</property>


<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>hive</value>
  <description>password to use against metastore database</description>
</property>

<property>
  <name>hive.stats.dbclass</name>
  <value>jdbc:mysql</value>
  <description>The default database that stores temporary hive statistics.</description>
</property>

<property>
  <name>hive.stats.jdbcdriver</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>The JDBC driver for the database that stores temporary hive statistics.</description>
</property>


<property>
  <name>hive.stats.dbconnectionstring</name>
  <value>jdbc:mysql://localhost:3306/hive_stats?useUnicode=true&amp;characterEncoding=latin1&amp;user=mysql_username&amp;password=mysql_password&amp;createDatabaseIfNotExist=true</value>
  <description>The default connection string for the database that stores temporary hive statistics.</description>
</property>

<property>
  <name>hive.server2.thrift.sasl.qop</name>
  <value>auth</value>
  <description>Sasl QOP value; Set it to one of following values to enable higher levels of
     protection for hive server2 communication with clients.
      "auth" - authentication only (default)
      "auth-int" - authentication plus integrity protection
      "auth-conf" - authentication plus integrity and confidentiality protection
     This is applicable only hive server2 is configured to use kerberos authentication.
 </description>
</property>

<property>
  <name>hive.metastore.schema.verification</name>
  <value>false</value>
   <description>
   Enforce metastore schema version consistency.
   True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic
         schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures
         proper metastore schema migration. (Default)
   False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
   </description>
</property>

ps:hive在hadoop中的路径可以默认不修改,env中java-home类似的配置可以省略,因为hadoop时已经配置了

     HADOOP_HOME=/usr/hadoop
    export HIVE_CONF_DIR=/usr/hive/conf



四:安装mysql并添加hive账号(账号密码都是hive)

1、执行yum install mysql-connector-java命令,自动安装mysql

2、将/usr/share/java/mysql-connector-java-5.1.17.jar拷贝到hivelib目录既可

3、添加用户:

•create database hive 
•创建hive用户,并授权
•grant all on hive.* to hive@'%'  identified by 'hive';  
•flush privileges;  

五、启动hive,第一次启动会报错,后续就正确了

六、验证:
1、hive是否可以创建数据库和表
hive> create table myhive(id int ,name string);
OK
Time taken: 3.32 seconds
hive> show tables;
OK
myhive
Time taken: 0.261 seconds, Fetched: 1 row(s)

2、加载hadoop中的文件
//待补充

3、select和insert
//待补充


安装Hive参考地址:
http://blog.csdn.net/vah101/article/details/18653543
http://blog.chinaunix.net/uid-451-id-3143781.html
http://www.cnblogs.com/ggjucheng/archive/2012/04/26/2471341.html
http://www.throwexcept.com/5/32/2026.html

常见问题:
1、hive启动报错:mysql驱动jar没有放入hive/lib中 http://bbs.csdn.net/topics/390787105?page=1#post-397387341
2、debug: hive --hiveconf  hive.root.logger=DEBUG,console

3、hadoop用户无法启动hive,启动报错如下:

14/05/22 07:10:34 INFO hive.metastore: Trying to connect to metastore with URI thrift://localhost:9083

14/05/22 07:10:34 WARN hive.metastore: Failed to connect to the MetaStore Server...

org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused

解决办法:进入/usr/hive/bin目录下,执行./hive

[hadoop@Master bin]$ pwd
/usr/hive/bin
[hadoop@Master bin]$ ./hive

 而不是直接在根目录下执行hive

[hadoop@Master ~]$ pwd
/home/hadoop
[hadoop@Master ~]$ hive






学习文章:

http://www.open-open.com/doc/view/8e07f484a8db411cb7964f52263811f8

http://www.blogjava.net/hengheng123456789/archive/2011/01/05/342354.html

 http://www.iteblog.com/archives/tag/hive的那些事

http://www.iteblog.com/archives/886 cj96 

http://www.iteblog.com/archives/851

 

配置参考文章:

http://www.throwexcept.com/5/32/2026.html

http://www.cnblogs.com/ggjucheng/archive/2012/04/26/2471341.html

 http://www.ibm.com/developerworks/cn/data/library/bd-hivelibrary/

 

 

注意点:

1、目前是把hive的目录放到跟hadoop同级,即/usr/hive

2、hive包中conf目录下有个默认的文件名叫hive-default.xml.template,我们要做的是拷贝一份,且命名为hive-site.xml,这是很坑爹的,同时里面有个<value>auth</auth>的错误

3、hive.metastore.warehouse.dir hive的表其实是存在hdfs的一个数据文件,并不是真正的表,表自身的信息称为元数据,表里面的数据其实就是hdfs中的数据,所以需要指定hive在hdfs中的目录

4、hive启动报错:参见:http://www.micmiu.com/bigdata/hive/hive-default-setup/

 

实战:

http://www.open-open.com/lib/view/open1386742941845.html

 

数据:

http://www.ibm.com/developerworks/cn/data/library/bd-hivelibrary/


常见的mysql命令:

http://www.360doc.com/content/10/1125/16/4506941_72353321.shtml


hive命令:

http://blog.csdn.net/xiaoping8411/article/details/7605039 


hive数据导入:

http://www.iteblog.com/archives/949


  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值