HUE整合HDFS MR

HUE(HadoopUser Experience)管理工具HUE是一个开源的HadoopUl系统,它基于PythonWEB框架实现,通过使用HUE我们可以在浏览器端的Web控制台上与Hadoop集群进行交互来分析处理数据。
官网下载页面 http://gethue.com/category/release/

环境与软件
系统:CentOS 6.5 三台 搭建hadoop集群
软件:hue-3.7.0-cdh5.3.6.tar.gz

mini01	mini02	mini03
NameNode		SecondaryNameNode
DataNode	DataNode	DataNode
ResourceManager	JobHistoryServer
NodeManager	NnodeManager	NodeManager
HUE

1.准备环境依赖

这里整理好了,可以直接yum安装

[hadoop@mini01 ~]$  sudo yum install -y ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite-devel gmp-devel

2.解压HUE

[hadoop@mini01 tools]$ tar -zxvf hue-3.7.0-cdh5.3.6.tar.gz -C ../install

3.编译HUE

[hadoop@mini01 tools]$ cd ../install/hue-3.7.0-cdh5.3.6/

[hadoop@mini01 hue-3.7.0-cdh5.3.6]$ make apps

4.配置HUE

修改Hue.ini文件
路径:/home/hadoop/install/hue-3.7.0-cdh5.3.6/desktop/conf/hue.ini
修改内容参照如下

    secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
    http_host=mini01
    http_port=8888
    time_zone=Asia/Shanghai
# Webserver runs as this user
    server_user=hue
    server_group=hue

与HDFS集成按照如下配置

[[hdfs_clusters]]
    # HA support by using HttpFs
    #hdfs如果配置了高可用,则需要使用hffpFs

    [[[default]]]     #我没有配置高可用所以端口是9000,如果高可用则是8020
      # Enter the filesystem uri
      fs_defaultfs=hdfs://mini01:9000

      # NameNode logical name.
      ## logical_name=

      # Use WebHdfs/HttpFs as the communication mechanism.
      # Domain should be the NameNode or HttpFs host.
      # Default port is 14000 for HttpFs.
      ## webhdfs_url=http://localhost:50070/webhdfs/v1
      #这里如果配置了高可用,那么端口就是14000
       webhdfs_url=http://mini01:50070/webhdfs/v1
      # Change this if your HDFS cluster is Kerberos-secured
      ##security_enabled=false

      # Default u