原文地址: http://blog.csdn.net/nsrainbow/article/details/43678237 最新课程请关注原作者博客,获得更好的显示体验
声明
- 本文基于Centos 6.x + CDH 5.x
HttpFs 有啥用
HttpFs可以干这两件事情
- 通过HttpFs你可以在浏览器里面管理HDFS上的文件
- HttpFs还提供了一套REST 风格的API可以用来管理HDFS
其实很简单的一个东西嘛,但是很实用
安装HttpFs
在集群里面找一台可以访问hdfs的机器安装HttpFs
$ sudo yum install hadoop-httpfs
配置
编辑/etc/hadoop/conf/core-site.xml
<property>
<name>hadoop.proxyuser.httpfs.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.httpfs.groups</name>
<value>*</value>
</property>
这边是定义可以使用httpfs的用户组和host,写*就是不限制
配置好之后重启hadoop
启动HttpFs
$ sudo service hadoop-httpfs start
使用HttpFs
{
"FileStatuses": {
"FileStatus": [{
"pathSuffix": "hbase",
"type": "DIRECTORY",
"length": 0,
"owner": "hbase",
"group": "hadoop",
"permission": "755",
"accessTime": 0,
"modificationTime": 1423446940595,
"blockSize": 0,
"replication": 0
},
{
"pathSuffix": "tmp",
"type": "DIRECTORY",
"length": 0,
"owner": "hdfs",
"group": "hadoop",
"permission": "1777",
"accessTime": 0,
"modificationTime": 1423122488037,
"blockSize": 0,
"replication": 0
},
{
"pathSuffix": "user",
"type": "DIRECTORY",
"length": 0,
"owner": "hdfs",
"group": "hadoop",
"permission": "755",
"accessTime": 0,
"modificationTime": 1423529997937,
"blockSize": 0,
"replication": 0
},
{
"pathSuffix": "var",
"type": "DIRECTORY",
"length": 0,
"owner": "hdfs",
"group": "hadoop",
"permission": "755",
"accessTime": 0,
"modificationTime": 1422945036465,
"blockSize": 0,
"replication": 0
}]
}
}
这个 &user.name=httpfs 表示用默认用户 httpfs 访问,默认用户是没有密码的。
webhdfs/v1 这是HttpFs的根目录
访问 http://host2:14000/webhdfs/v1/user?op=LISTSTATUS&user.name=httpfs 可以看到
{
"FileStatuses": {
"FileStatus": [{
"pathSuffix": "cloudera",
"type": "DIRECTORY",
"length": 0,
"owner": "root",
"group": "hadoop",
"permission": "755",
"accessTime": 0,
"modificationTime": 1423472508868,
"blockSize": 0,
"replication": 0
},
{
"pathSuffix": "hdfs",
"type": "DIRECTORY",
"length": 0,
"owner": "hdfs",
"group": "hadoop",
"permission": "700",
"accessTime": 0,
"modificationTime": 1422947019504,
"blockSize": 0,
"replication": 0
},
{
"pathSuffix": "history",
"type": "DIRECTORY",
"length": 0,
"owner": "mapred",
"group": "hadoop",
"permission": "1777",
"accessTime": 0,
"modificationTime": 1422945692887,
"blockSize": 0,
"replication": 0
},
{
"pathSuffix": "hive",
"type"