hadoop-hive安装配置
下载hive-0.8.1.tar.gz ---------在ahdoop1.0.0这个版本上要使用这个版本以上
把这个文件解压在/hadoop/app目录下
1.添加bin/hive-config.sh,添加jdk支持
exportJAVA_HOME=/usr/java/jdk1.7.0_02
exportHIVE_HOME=/hadoop/app/hive-0.8.1
exportHADOOP_HOME=/hadoop/app/hadoop-1.0.0
Java代码
root@master:/data/soft#hadoop fs –mkdir /tmp
root@master:/data/soft#hadoop fs –mkdir /usr/hive/warehouse
root@master:/data/soft#hadoop fs –chmod g+w /tmp
root@master:/data/soft# hadoop fs –chmod g+w /usr/hive/warehouse ------------最好把这个目录创建
通过我的试验,以上创建目录的步骤是可以省略的,Hive会自动创建需要的目录
3.修改conf/hive-default.xml,这个是hive的关键配置,所以一般不要直接修改,新建hive-site.xml文件,将修改的内容在这个里面配置。
[root@hadoop-m conf]# cat /hadoop/app/hive-0.8.1/conf/hive-site.xml
<configuration>
<property>
<name>hive.metastore.local</name>
<value>true</value>
</property>
<property>
<name>hive.exec.scratchdir</name>
<value>/hadoop/app/hive-0.8.1/tmp</value>
<description>Scratch space for Hive jobs</description>
</property>
<property>
<name>hive.querylog.location</name>
<value>/hadoop/app/hive-0.8.1/querylog</value>
</property>
<property>
<name>hive.hwi.listen.host</name>
<value>0.0.0.0</value>
<description>This is the host address the Hive Web Interface will listen on</description>
</property>
<property>
<name>hive.hwi.listen.port</name>
<value>9999</value>
<description>This is the port the Hive Web Interface will listen on</description>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.10.33:3306/hive?characterEncoding=UTF-8</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>hdfs://master:9000/cucrz/hive/warehouse</value>
<value>/cucrz/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>hive</value> #mysql登陆名
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>111</value> #mysql密码
<description>password to use against metastore database</description>
</property>
</configuration>
进入/home/hadoop/hive-0.8.1目录
用命令:bin/hive开启hive
Hive使用的语法与SQL差不多
检查:
c. 检查
$ hive
hive> use dw2;
OK
Time taken: 3.43 seconds
hive> create table hwz2(id int, name string ) row format delimited FIELDS TERMINATED BY ',';
OK
Time taken: 2.519 seconds
hive> show tables;
hwz2
Time taken: 0.419 seconds
hive> load data local inpath 'demo.txt' overwrite into table hwz2;
Copying data from file:/home/hadoop/demo.txt
Copying file: file:/home/hadoop/demo.txt
Loading data to table dw2.hwz2
Deleted hdfs://master:9000/user/hive/warehouse/dw2.db/hwz2
OK
Time taken: 0.557 seconds
hive> select * from hwz2;
OK
12 jack
12 jack
12 jack
12 jack
12 jack
12 jack
12 jack
12 jack
$ hadoop dfs -lsr /user/hive
Warning: $HADOOP_HOME is deprecated.
drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:36 /user/hive/warehouse
drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:48 /user/hive/warehouse/dw2.db
drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:48 /user/hive/warehouse/dw2.db/hwz2
-rw-r--r-- 2 hadoop supergroup 1201 2012-03-22 15:48 /user/hive/warehouse/dw2.db/hwz2/demo.txt
drwxr-xr-x - hadoop supergroup 0 2012-03-22 12:36 /user/hive/warehouse/hwz
drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:36 /user/hive/warehouse/hwz2
-rw-r--r-- 2 hadoop supergroup 1201 2012-03-22 15:36 /user/hive/warehouse/hwz2/demo.txt
$ hadoop dfs -cat /user/hive/warehouse/dw2.db/hwz2/demo.txt |head
Warning: $HADOOP_HOME is deprecated.
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
d. 在mysql中验证建立的新表
mysql> use hive;
Database changed
mysql> show tables;
+-----------------+
| Tables_in_hive |
+-----------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| DATABASE_PARAMS |
| DBS |
| PARTITION_KEYS |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SORT_COLS |
| TABLE_PARAMS |
| TBLS |
+-----------------+
14 rows in set (0.00 sec)
错误:
[hadoop@master data]$ hive
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.Logging initialized using configuration in jar:file:/app/hive/lib/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history file=/tmp/hadoop/hive_job_log_hadoop_201209251016_560764333.txt
当登录hive的时候 老是有个警告:
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
根据警告的提示 只需要将 一个参数值从 org.apache.hadoop.metrics.jvm.EventCounter 修改为 org.apache.hadoop.log.metrics.EventCounter 即可。
解决的办法:
-- 将conf下面所有的配置文件都cp 一份
cd $HIVE_HOME/conf
cp hive-default.xml.template hive-default.xml
cp hive-default.xml.template hive-site.xml
cp hive-env.sh.template hive-env.sh
cp hive-log4j.properties.template hive-log4j.properties
--修改参数
vi hive-log4j.properties
找到 参数项 log4j.appender.EventCounter
将这一项的值修改为: org.apache.hadoop.log.metrics.EventCounter
修改之后就好了 再次登录hive 警告就消失了。
[hadoop@master conf]$ hive
Logging initialized using configuration in file:/app/hive/conf/hive-log4j.properties
Hive history file=/tmp/hadoop/hive_job_log_hadoop_201209251022_944441939.txt
hive> exit