Hive的安装部署

Hive环境的搭建
/home/hadoop/      （目录）
             software  存放安装软件
                  hadoop-2.6.0-cdh5.7.0.tar.gz
                  hive-1.1.0-cdh5.7.0.tar.gz

             data 存放测试数据

             source 存放源代码
                  hadoop-2.6.0-cdh5.7.0-src.tar.gz 

             lib  存放相关开发的jar

            app  软件安装目录
                  apache-flume-1.6.0-cdh5.7.0-bin
                  apache-maven-3.3.9
                  ...

            tmp 存放HDFS/Kafka/ZK数据目录

            maven_repo  maven本地仓库

            shell  存放上课相关的脚本 

 mysql：root/root

      mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('root');

             mysql>  FLUSH PRIVILEGES;

CDH版本选择：
    选择统一的cdh5.7.0尾号
     hadoop-2.6.0-cdh5.7.0.tar.gz
     hive-1.1.0-cdh5.7.0.tar.gz

    wget http://archive.cloudera.com/cdh5/cdh/5/hive-1.1.0-cdh5.7.0.tar.gz
    tar -zxvf hive-1.1.0-cdh5.7.0.tar.gz -C ~/app

1）创建目录：
 mkdir

2）上传tar.gz文件：

[hadoop@ruozehadoop000 software]$ rz

total 459260
-rw-r--r--. 1 hadoop hadoop  42610549 Jun  2  2018 hadoop-2.6.0-cdh5.7.0-src.tar.gz
-rw-r--r--. 1 hadoop hadoop 311585484 Jun  2  2018 hadoop-2.6.0-cdh5.7.0.tar.gz
-rw-r--r--. 1 hadoop hadoop 116082695 Jun  2  2018 hive-1.1.0-cdh5.7.0.tar.gz

3）解压：
[hadoop@hadoop-01 software]$ tar -zxvf hadoop-2.6.0-cdh5.7.0.tar.gz -C ~/app/
[hadoop@hadoop-01 software]$ tar -zxvf hive-1.1.0-cdh5.7.0.tar.gz -C  ~/app/

4）添加HIVE_HOME到系统环境变量：
[hadoop@ruozehadoop000 app]$ vi ~/.bash_profile
export HIVE_HOME=/home/hadoop/app/hive-1.1.0-cdh5.7.0
export PATH=$HIVE_HOME/bin:$PATH
[hadoop@ruozehadoop000 app]$ source ~/.bash_profile

5）Hive配置修改
[hadoop@ruozehadoop000 conf]$ cp hive-env.sh.template hive-env.sh

vi hive-env.sh
    HADOOP_HOME=/home/hadoop/app/hadoop-2.6.0-cdh5.7.0

hive-site.xml   统一元数据管理
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
    <configuration>
        <property>
          <name>javax.jdo.option.ConnectionURL</name>
          <value>jdbc:mysql://localhost:3306/ruozedata_basic03(数据库名)?createDatabaseIfNotExist=true（当数据库不存在的时候，自动创建数据库）</value>
        </property>

        <property>
          <name>javax.jdo.option.ConnectionDriverName</name>
          <value>com.mysql.jdbc.Driver</value>
        </property>

        <property>
          <name>javax.jdo.option.ConnectionUserName</name>
          <value>root</value>
        </property>

        <property>
          <name>javax.jdo.option.ConnectionPassword</name>
          <value>root</value>
        </property>
    </configuration>

6）拷贝MySQL驱动包到$HIVE_HOME/lib

拷贝 mysql-connector-java-5.1.27-bin.jar
[hadoop@ruozehadoop000 ~]$ cd $HIVE_HOME
[hadoop@ruozehadoop000 hive-1.1.0-cdh5.7.0]$ cd lib
[hadoop@ruozehadoop000 lib]$ rz mysql-connector-java-5.1.27-bin.jar

如果没有拷贝MySQL驱动包，启动hive会报错：
    The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH.
    Please check your CLASSPATH specification,
    and the name of the driver.

7）权限问题
创建表失败：
    FAILED: Execution Error,
    return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask.
    MetaException(message:For direct MetaStore DB
    connections, we don't support retries at the client
    level.)

    思路：找日志
    日志在哪里：$HIVE_HOME/conf/hive-log4j.properties.template
                hive.log.dir=${java.io.tmpdir}/${user.name}
                hive.log.file=hive.log
                问题：能不能改，如何改？
                答：修改环境变量中java.io.tmpdir的设定路径

    日志错误：
        ERROR [main]: Datastore.Schema (Log4JLogger.java:error(115)) - An exception was thrown while adding/validating class(es) :
        Specified key was too long; max key length is 767 bytes
        com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes

    解决：
        alter database ruozedata_basic02 character set latin1;
   
8）启动Hive