Hive-1.1.0-cdh5.7.0安装部署

前言：
    考虑到Apache组件的架包兼容性问题，生产环境统一使用CDH、HDP
    选择统一的cdh5.7.0尾号
        hadoop-2.6.0-cdh5.7.0.tar.gz
        hive-1.1.0-cdh5.7.0.tar.gz

    CDH组件下载地址：http://archive.cloudera.com/cdh5/cdh/5/

风格介绍:
使用Apache组件,统一使用Hadoop用户
[hadoop@hadoop001 ~]$ pwd
/home/hadoop
[hadoop@hadoop001 ~]$ ll
total 28
drwxr-xr-x. 5 hadoop hadoop 4096 Jun  1 22:32 app        -- 安装路径(/app/tmp/HDFS Kafka ZK 数据目录)
drwxr-xr-x. 2 hadoop hadoop 4096 Jun  1 16:47 data        -- 测试数据存放路径 
drwxr-xr-x. 2 hadoop hadoop 4096 Jun  1 16:48 lib            -- 存放相关开发jar
drwxr-xr-x. 2 hadoop hadoop 4096 Jun  1 16:48 maven_repo    -- maven 本地仓库
drwxr-xr-x. 2 hadoop hadoop 4096 Jun  1 16:48 shell              -- shell脚本
drwxr-xr-x. 2 hadoop hadoop 4096 Jun  1 22:30 software       -- 下载安装包
drwxr-xr-x. 2 hadoop hadoop 4096 Jun  1 16:48 source         -- 存放源代码

一、下载二进制文件包
[hadoop@hadoop001 software]$ wget http://archive.cloudera.com/cdh5/cdh/5/hive-1.1.0-cdh5.6.1.tar.gz
[hadoop@hadoop001 software]$ tar -xzvf hive-1.1.0-cdh5.6.1.tar.gz 
[hadoop@hadoop001 software]$ mv hive-1.1.0-cdh5.6.1.tar.gz  ../app/hive-1.1.0

二、配置环境变量

1. export HIVE_HOME=/home/hadoop/app/hive-1.1.0
2. export PATH=$HIVE_HOME/bin:$PATH

三、修改配置参数
[hadoop@hadoop001 conf]$ pwd
/home/hadoop/app/hive-1.1.0/conf
[hadoop@hadoop001 conf]$ cp hive-env.sh.template hive-env.sh
[hadoop@hadoop001 conf]$ vim hive-env.sh

1.指向Hadoop路径

1. HADOOP_HOME=/home/hadoop/app/hadoop-2.8.1
   

2.创建hive-site.xml(配置元数据的存放路径)
[hadoop@hadoop001 conf]$ vim hive-site.xml

1. <configuration>
   
2.         <property>
   
3.                 <name>javax.jdo.option.ConnectionURL</name>
   
4.                 <value>jdbc:mysql://localhost:3306/hive_data?createDatabaseIfNotExist=true&amp;characterEncoding=latin1</value>
   
5.         </property>
   
6. 
   
7.         <property>
   
8.                 <name>javax.jdo.option.ConnectionDriverName</name>
   
9.                 <value>com.mysql.jdbc.Driver</value>
   
10.         </property>
    
11. 
    
12.         <property>
    
13.                 <name>javax.jdo.option.ConnectionUserName</name>
    
14.                 <value>root</value>
    
15.         </property>
    
16. 
    
17.         <property>
    
18.                 <name>javax.jdo.option.ConnectionPassword</name>
    
19.                 <value>root</value>
    
20.         </property>
    
21. </configuration>

3.导入MySQLDriver到 $HIVE_HOME/lib
cp MySQLDriver $HIVE_HOME/lib


四、启动Hive
[hadoop@hadoop001 bin]$ pwd
/home/hadoop/app/hive-1.1.0/bin
[hadoop@hadoop001 bin]$ hive  或者 ./hive
[hadoop@hadoop001 bin]$ hive
which: no hbase in (/home/hadoop/app/hive-1.1.0/bin:/home/hadoop/app/hadoop-2.8.1/bin:/usr/local/protobuf/bin:/opt/software/findbugs-
Logging initialized using configuration in jar:file:/home/hadoop/app/hive-1.1.0/lib/hive-common-1.1.0-cdh5.7.0.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.
hive> 


五、检查是否能创建数据库及表
hive> create database hive_data;
OK
Time taken: 0.682 seconds
hive> create table test(in int,name string);
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections, we don't support retries at the client level.)

MySQL进入hive_data数据库字符集：

1. ALTER DATABASE hive_data CHARACTER SET latin1

hive> create table test(in int,name string);
OK
Time taken: 0.81 seconds
hive> show tables;
OK
test
Time taken: 0.053 seconds, Fetched: 1 row(s)

六、Hive指向路径修改
Hive中的数据存放在HDFS之上,相当于顶层文件夹;创建数据库默认的路径为/user/hive/warehouse下;
修改Hive默认存放路径  $HIVE_HOME/conf/hive-site-xml

1. -- 设定数据目录
2. <property>
3. <name>hive.metastore.warehouse.dir</name>
4. <value>/usr/local/hive/warehouse</value>
5. </property>
7. -- 设定临时文件目录
8. <poprerty>
9. <name>hive.exec.scratdir</name>
10. <value>/usr/local/hive/tmp</value>
11. </property>
13. -- hive相关日志的目录
14. <property>
15. <name>hive.querylog.location</name>
16. <value>/usr/local/hive/log</value>
17. </property>

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/31441024/viewspace-2155546/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/31441024/viewspace-2155546/