【Hive】Hive0.13.1的安装及Hive运行RuntimeException Cannot make directory错误的解决

最新推荐文章于 2024-07-30 19:46:58 发布

cy_dream

最新推荐文章于 2024-07-30 19:46:58 发布

阅读量3.2k

点赞数 1

分类专栏： Hive

本文链接：https://blog.csdn.net/qq_33304418/article/details/52419360

版权

Hive 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

安装Hive之前先安装Mysql，详情请看另一篇文章
http://blog.csdn.net/qq_33304418/article/details/52373054

FAILED: RuntimeException Cannot make directory: hdfs://mycluster/tmp/hive-hadoop/hive_2016-09-01_19-29-23_277_522751999966567152-3 (state=42000,code=40000)

解决方法1：关闭hadoop安全模式 (首先查看hadoop是否处在安全模式)
hadoop dfsadmin -safemode leave 

解决方法2：hive在运行时要读取写入目录里的内容，所以把权限放开，设置权限(详情往下看，是自己学习时遇到的问题及解决方法)
hadoop fs -chmod -R 777 /tmp

安装hive

解压
[hadoop@Master ruanJian]$ tar zxf  apache-hive-0.13.1-bin.tar.gz
[hadoop@Master ruanJian]$ mv apache-hive-0.13.1-bin hive013
[hadoop@Master ruanJian]$ cd hive013/conf
[hadoop@Master conf]$ cp hive-default.xml.template hive-site.xml
[hadoop@Master conf]$ cp hive-env.sh.template hive-env.sh
[hadoop@Master conf]$ vi hive-env.sh
HADOOP_HOME=/usr/hadoop-2.6.4    //添加hadoop的安装目录

[hadoop@Master conf]$ vi hive-site.xml 

<property> 
  <name>javax.jdo.option.ConnectionURL</name> 
  <value>jdbc:mysql://192.168.1.110:3306/hive?=createDatabaseIfNotExist=true</value> 
  <description>JDBC connect string for a JDBC metastore</description> 
</property> 
<property> 
  <name>javax.jdo.option.ConnectionDriverName</name> 
  <value>com.mysql.jdbc.Driver</value> //修改jdbc的驱动
  <description>Driver class name for a JDBC metastore</description> 
</property> 
<property> 
  <name>javax.jdo.option.ConnectionUserName</name> 
  <value>root</value>   //mysql的用户名
  <description>username to use against metastore database</description> 
</property> 
<property> 
  <name>javax.jdo.option.ConnectionPassword</name> 
  <value>123456</value>   //mysql的密码，尽量不要为root， 我的为root启动Hive连接不上
  <description>password to use against metastore database</description> 
</property>

hive创建表(为后续做准备)

[hadoop@Master bin]$ ./hive

create table t_emp(
id int,
name string,
age int,
dept_name string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ',';

新建user.txt并添加以下内容
1,张三,32,销售部
2,李四,23,开发部
3,王五,19,测试部
4,赵六,28,产品经理 

[hadoop@Master bin]$ ./hiveload data local inpath '/home/hadoop/data/user.txt' into table user;        //根据自己目录选择

hive执行方式三种

    1.hive -e 'hql'
    2.hive -f 'hql.file'
    3.hive jdbc 代码执行脚本

在此介绍用eclipse 使用jdbc连接hive

[hadoop@Master bin]$ ./hive --service hiveserver2  //必须先启动服务

//启动之前查看修改此处
[hadoop@Master conf]$ vi hive-site.xml
<property>
   <name>hive.server2.long.polling.timeout</name>  
   <value>5000L</value>    此处5000L修改为5000 不然后边会报错
   <description>Time in milliseconds that HiveServer2 will wait, before resp onding to asynchronous calls that use long polling</description>
</property>

重新启动一个窗口
[hadoop@Master bin]$ ./beeline //启动客户端 hive/bin 目录下提供

Beeline version 0.13.1 by Apache Hive

beeline> !connect jdbc:hive2://192.168.1.110/default    //执行  default默认的数据库

scan complete in 6ms
Connecting to jdbc:hive2://192.168.1.110/default
Enter username for jdbc:hive2://192.168.1.110/default: root
Enter password for jdbc:hive2://192.168.1.110/default:      //密码默认为空
Connected to: Apache Hive (version 0.13.1)
Driver: Hive JDBC (version 0.13.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://192.168.1.110/default> show tables;
+-----------+
| tab_name  |
+-----------+
+-----------+
No rows selected (1.991 seconds)        //至此发现可以显示表，但是不能执行查询等语句
0: jdbc:hive2://192.168.1.110/default>select count(*) from user;

FAILED: RuntimeException Cannot make directory: hdfs://mycluster/tmp/hive-hadoop/hive_2016-09-01_19-29-23_277_522751999966567152-3 (state=42000,code=40000)

这个错误是以下原因：
浏览器访问会看到以下结果http://192.168.1.110:50070/explorer.html#
Browse Directory
Permission  Owner   Group   Size    Replication Block Size  Name
drwxr-xr-x  hadoop  supergroup  0 B 0   0 B home
drwx------  hadoop  supergroup  0 B 0   0 B tmp
drwxr-xr-x  hadoop  supergroup  0 B 0   0 B user

hive在运行时要读取写入目录里的内容，所以把权限放开，设置权限：

hadoop fs -chmod -R 777 /tmp

修改后：
Permission  Owner   Group   Size    Replication Block Size  Name
drwxr-xr-x  hadoop  supergroup  0 B 0   0 B home
drwxrwxrwx  hadoop  supergroup  0 B 0   0 B tmp
drwxr-xr-x  hadoop  supergroup  0 B 0   0 B user

重新运行：
0: jdbc:hive2://192.168.1.110/default>select count(*) from t_emp;
+------+
| _c0  |
+------+
| 4    |
+------+
1 row selected (71.704 seconds)
0: jdbc:hive2://192.168.1.110/default>

java jdbc连接hive执行查询等语句

public class HiveTest {

    public static void main(String[] args) throws Exception {

        Class.forName("org.apache.hive.jdbc.HiveDriver");
        Connection conn = DriverManager.getConnection("jdbc:hive2://192.168.1.110/default", "root", "");

        try {

            Statement st = conn.createStatement();
            ResultSet ret = st.executeQuery("select count(*) from user");

            if(ret.next()){
                System.out.println(ret.getInt(1));
            }
        } catch (Exception e) {
            e.printStackTrace();
        }finally{
            conn.close();
        }
    }
}