hadoop-hive安装配置

hadoop-hive安装配置

下载hive-0.8.1.tar.gz   ---------在ahdoop1.0.0这个版本上要使用这个版本以上

    把这个文件解压在/hadoop/app目录下

1.添加bin/hive-config.sh，添加jdk支持

exportJAVA_HOME=/usr/java/jdk1.7.0_02

exportHIVE_HOME=/hadoop/app/hive-0.8.1 

exportHADOOP_HOME=/hadoop/app/hadoop-1.0.0

Java代码

root@master:/data/soft#hadoop fs –mkdir /tmp 

root@master:/data/soft#hadoop fs –mkdir /usr/hive/warehouse 

root@master:/data/soft#hadoop fs –chmod g+w /tmp 

root@master:/data/soft#            hadoop fs –chmod g+w /usr/hive/warehouse  ------------最好把这个目录创建

通过我的试验，以上创建目录的步骤是可以省略的，Hive会自动创建需要的目录

3.修改conf/hive-default.xml，这个是hive的关键配置，所以一般不要直接修改，新建hive-site.xml文件，将修改的内容在这个里面配置。

[root@hadoop-m conf]# cat /hadoop/app/hive-0.8.1/conf/hive-site.xml 
<configuration>
<property>
  <name>hive.metastore.local</name>
    <value>true</value>
    </property>


<property>  
  <name>hive.exec.scratchdir</name>  
    <value>/hadoop/app/hive-0.8.1/tmp</value>  
      <description>Scratch space for Hive jobs</description>  
      </property>  
      <property>  
        <name>hive.querylog.location</name>  
          <value>/hadoop/app/hive-0.8.1/querylog</value>  
          </property>  
          <property>  
            <name>hive.hwi.listen.host</name>  
              <value>0.0.0.0</value>  
                <description>This is the host address the Hive Web Interface will listen on</description>  
                </property>  
                <property>  
                  <name>hive.hwi.listen.port</name>  
                    <value>9999</value>  
                      <description>This is the port the Hive Web Interface will listen on</description>  
                      </property>  
<property>
  <name>javax.jdo.option.ConnectionURL</name>
     <value>jdbc:mysql://192.168.10.33:3306/hive?characterEncoding=UTF-8</value>
      <description>JDBC connect string for a JDBC metastore</description>
      </property>


<property>
  <name>hive.metastore.warehouse.dir</name>
     <value>hdfs://master:9000/cucrz/hive/warehouse</value>
        <value>/cucrz/hive/warehouse</value>
          <description>location of default database for the warehouse</description>
          </property>


<property>
<name>javax.jdo.option.ConnectionDriverName</name>                     
<value>com.mysql.jdbc.Driver</value>
</property>


<property>     
<name>javax.jdo.option.ConnectionUserName</name>     
<value>hive</value>           #mysql登陆名
<description>username to use against metastore database</description>  
</property>
  
<property>     
<name>javax.jdo.option.ConnectionPassword</name>     
<value>111</value>             #mysql密码
<description>password to use against metastore database</description>  
</property>


 </configuration>

进入/home/hadoop/hive-0.8.1目录

用命令：bin/hive开启hive

Hive使用的语法与SQL差不多

检查：

c. 检查
$ hive
hive> use dw2;
OK

Time taken: 3.43 seconds

hive> create table hwz2(id int, name string ) row format delimited FIELDS TERMINATED BY ',';
OK
Time taken: 2.519 seconds

hive> show tables;

OK
hwz2
Time taken: 0.419 seconds
hive> load data local inpath 'demo.txt' overwrite into table hwz2;
Copying data from file:/home/hadoop/demo.txt
Copying file: file:/home/hadoop/demo.txt
Loading data to table dw2.hwz2
Deleted hdfs://master:9000/user/hive/warehouse/dw2.db/hwz2
OK
Time taken: 0.557 seconds
hive> select * from hwz2;
OK
12 jack
12 jack
12 jack
12 jack
12 jack
12 jack
12 jack
12 jack

$ hadoop dfs -lsr /user/hive
Warning: $HADOOP_HOME is deprecated.

drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:36 /user/hive/warehouse
drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:48 /user/hive/warehouse/dw2.db
drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:48 /user/hive/warehouse/dw2.db/hwz2
-rw-r--r-- 2 hadoop supergroup 1201 2012-03-22 15:48 /user/hive/warehouse/dw2.db/hwz2/demo.txt
drwxr-xr-x - hadoop supergroup 0 2012-03-22 12:36 /user/hive/warehouse/hwz
drwxr-xr-x - hadoop supergroup 0 2012-03-22 15:36 /user/hive/warehouse/hwz2
-rw-r--r-- 2 hadoop supergroup 1201 2012-03-22 15:36 /user/hive/warehouse/hwz2/demo.txt
$ hadoop dfs -cat /user/hive/warehouse/dw2.db/hwz2/demo.txt |head
Warning: $HADOOP_HOME is deprecated.

12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack
12,jack



d. 在mysql中验证建立的新表
mysql> use hive;
Database changed

mysql> show tables;
+-----------------+
| Tables_in_hive |
+-----------------+
| BUCKETING_COLS |
| CDS |
| COLUMNS_V2 |
| DATABASE_PARAMS |
| DBS |
| PARTITION_KEYS |
| SDS |
| SD_PARAMS |
| SEQUENCE_TABLE |
| SERDES |
| SERDE_PARAMS |
| SORT_COLS |
| TABLE_PARAMS |
| TBLS |
+-----------------+
14 rows in set (0.00 sec)

错误：

[hadoop@master data]$ hive

WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in jar:file:/app/hive/lib/hive-common-0.9.0.jar!/hive-log4j.properties

Hive history file=/tmp/hadoop/hive_job_log_hadoop_201209251016_560764333.txt

当登录hive的时候 老是有个警告：

WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.

根据警告的提示  只需要将 一个参数值从 org.apache.hadoop.metrics.jvm.EventCounter 修改为 org.apache.hadoop.log.metrics.EventCounter 即可。

解决的办法：

-- 将conf下面所有的配置文件都cp 一份

cd $HIVE_HOME/conf

cp hive-default.xml.template hive-default.xml  

cp hive-default.xml.template hive-site.xml  

cp hive-env.sh.template hive-env.sh  

cp hive-log4j.properties.template hive-log4j.properties  

--修改参数 

vi  hive-log4j.properties  

找到  参数项 log4j.appender.EventCounter

将这一项的值修改为： org.apache.hadoop.log.metrics.EventCounter

修改之后就好了 再次登录hive  警告就消失了。

[hadoop@master conf]$ hive
Logging initialized using configuration in file:/app/hive/conf/hive-log4j.properties
Hive history file=/tmp/hadoop/hive_job_log_hadoop_201209251022_944441939.txt
hive> exit