安装Hive之前先安装Mysql,详情请看另一篇文章
http://blog.csdn.net/qq_33304418/article/details/52373054
FAILED: RuntimeException Cannot make directory: hdfs://mycluster/tmp/hive-hadoop/hive_2016-09-01_19-29-23_277_522751999966567152-3 (state=42000,code=40000)
解决方法1:关闭hadoop安全模式 (首先查看hadoop是否处在安全模式)
hadoop dfsadmin -safemode leave
解决方法2:hive在运行时要读取写入目录里的内容,所以把权限放开,设置权限(详情往下看,是自己学习时遇到的问题及解决方法)
hadoop fs -chmod -R 777 /tmp
安装hive
解压
[hadoop@Master ruanJian]$ tar zxf apache-hive-0.13.1-bin.tar.gz
[hadoop@Master ruanJian]$ mv apache-hive-0.13.1-bin hive013
[hadoop@Master ruanJian]$ cd hive013/conf
[hadoop@Master conf]$ cp hive-default.xml.template hive-site.xml
[hadoop@Master conf]$ cp hive-env.sh.template hive-env.sh
[hadoop@Master conf]$ vi hive-env.sh
HADOOP_HOME=/usr/hadoop-2.6.4 //添加hadoop的安装目录
[hadoop@Master conf]$ vi hive-site.xml
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.1.110:3306/hive?=createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value> //修改jdbc的驱动
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value> //mysql的用户名
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value> //mysql的密码,尽量不要为root, 我的为root启动Hive连接不上
<description>password to use against metastore database</description>
</property>
hive创建表(为后续做准备)
[hadoop@Master bin]$ ./hive
create table t_emp(
id int,
name string,
age int,
dept_name string
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ',';
新建user.txt并添加以下内容
1,张三,32,销售部
2,李四,23,开发部
3,王五,19,测试部
4,赵六,28,产品经理
[hadoop@Master bin]$ ./hiveload data local inpath '/home/hadoop/data/user.txt' into table user; //根据自己目录选择
hive执行方式三种
1.hive -e 'hql'
2.hive -f 'hql.file'
3.hive jdbc 代码执行脚本
在此介绍用eclipse 使用jdbc连接hive
[hadoop@Master bin]$ ./hive --service hiveserver2 //必须先启动服务
//启动之前查看修改此处
[hadoop@Master conf]$ vi hive-site.xml
<property>
<name>hive.server2.long.polling.timeout</name>
<value>5000L</value> 此处5000L修改为5000 不然后边会报错
<description>Time in milliseconds that HiveServer2 will wait, before resp onding to asynchronous calls that use long polling</description>
</property>
重新启动一个窗口
[hadoop@Master bin]$ ./beeline //启动客户端 hive/bin 目录下提供
Beeline version 0.13.1 by Apache Hive
beeline> !connect jdbc:hive2://192.168.1.110/default //执行 default默认的数据库
scan complete in 6ms
Connecting to jdbc:hive2://192.168.1.110/default
Enter username for jdbc:hive2://192.168.1.110/default: root
Enter password for jdbc:hive2://192.168.1.110/default: //密码默认为空
Connected to: Apache Hive (version 0.13.1)
Driver: Hive JDBC (version 0.13.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://192.168.1.110/default> show tables;
+-----------+
| tab_name |
+-----------+
+-----------+
No rows selected (1.991 seconds) //至此发现可以显示表,但是不能执行查询等语句
0: jdbc:hive2://192.168.1.110/default>select count(*) from user;
FAILED: RuntimeException Cannot make directory: hdfs://mycluster/tmp/hive-hadoop/hive_2016-09-01_19-29-23_277_522751999966567152-3 (state=42000,code=40000)
这个错误是以下原因:
浏览器访问会看到以下结果http://192.168.1.110:50070/explorer.html#
Browse Directory
Permission Owner Group Size Replication Block Size Name
drwxr-xr-x hadoop supergroup 0 B 0 0 B home
drwx------ hadoop supergroup 0 B 0 0 B tmp
drwxr-xr-x hadoop supergroup 0 B 0 0 B user
hive在运行时要读取写入目录里的内容,所以把权限放开,设置权限:
hadoop fs -chmod -R 777 /tmp
修改后:
Permission Owner Group Size Replication Block Size Name
drwxr-xr-x hadoop supergroup 0 B 0 0 B home
drwxrwxrwx hadoop supergroup 0 B 0 0 B tmp
drwxr-xr-x hadoop supergroup 0 B 0 0 B user
重新运行:
0: jdbc:hive2://192.168.1.110/default>select count(*) from t_emp;
+------+
| _c0 |
+------+
| 4 |
+------+
1 row selected (71.704 seconds)
0: jdbc:hive2://192.168.1.110/default>
java jdbc连接hive执行查询等语句
public class HiveTest {
public static void main(String[] args) throws Exception {
Class.forName("org.apache.hive.jdbc.HiveDriver");
Connection conn = DriverManager.getConnection("jdbc:hive2://192.168.1.110/default", "root", "");
try {
Statement st = conn.createStatement();
ResultSet ret = st.executeQuery("select count(*) from user");
if(ret.next()){
System.out.println(ret.getInt(1));
}
} catch (Exception e) {
e.printStackTrace();
}finally{
conn.close();
}
}
}