Hue集成HDFS、YARN、Hive、MySql、HBase的相关配置

官网教程

与HDFS集成

• 修改hadoop中core-site.xml配置文件(集群中所有机器都需要配置)

  #core-site.xml
  <property>
      <name>hadoop.proxyuser.hue.hosts</name>
      <value>*</value>
  </property>
  <property>
      <name>hadoop.proxyuser.hue.groups</name>
      <value>*</value>
  </property>
  

• 修改hue.ini配置文件

  #在hue.ini文件中找到[hadoop]->[[hdfs_clusters]]->[[[default]]]
  fs_defaultfs=hdfs://ns
  webhdfs_url=http://bigdata-pro01.bigDAta.com:50070/webhdfs/v1
  hadoop_hdfs_home=/opt/modules/hadoop-2.6.0
  hadoop_bin=/opt/modules/hadoop-2.6.0/bin
  hadoop_conf_dir=/opt/modules/hadoop-2.6.0/etc/hadoop
  

• 启动hue

  先启动zookeeper，hdfs，再启动hue
  

与YARN集成

• 修改hue.ini配置文件(需要区分集群是否是HA)

  #在hue.ini文件中找到[hadoop]->[[yarn_clusters]]
  #HA集群
  [[[default]]]
  	  # Enter the host on which you are running the ResourceManager
        resourcemanager_host=rs
        # The port where the ResourceManager IPC listens on
        resourcemanager_port=8032
        submit_to=True
        #Resource Manager logical name (required for HA)
        logical_name=rm1
        resourcemanager_api_url=http://bigdata-pro01.bigDAta.com:8088
        proxy_api_url=http://bigdata-pro01.bigDAta.com:8088
        history_server_api_url=http://bigdata-pro01.bigDAta.com:19888
  [[[ha]]]
        logical_name=rm2
        submit_to=True
        resourcemanager_api_url=http://bigdata-pro02.bigDAta.com:8088
  	  history_server_api_url=http://bigdata-pro01.bigDAta.com:19888
  ------------------------------------------------------------------------------
  #非HA集群
  [[[default]]]
        resourcemanager_host=rs
        resourcemanager_port=8032
        submit_to=True
        resourcemanager_api_url=http://bigdata-pro01.bigDAta.com:8088
        proxy_api_url=http://bigdata-pro01.bigDAta.com:8088
        history_server_api_url=http://bigdata-pro01.bigDAta.com:19888
  

  • <!--resourcemanager_host对应于yarn-site.xml中配置的-->
    <property>
        <name>yarn.resourcemanager.cluster-id</name>
        <value>rs</value>
    </property>
    <!--logical_name名字对应于yarn-site.xml中配置的--> 
    <property> 
        <name>yarn.resourcemanager.ha.rm-ids</name>  
        <value>rm1,rm2</value> 
    </property> 
    <!--resourcemanager_api_url配置资源管理的地址和端口，对应于yarn-site.xml中配置的-->
    <property>
    	<name>yarn.resourcemanager.webapp.address</name>
    	<value>bigdata-pro01.bigDAta.com:8088</value>
    </property>
    <!--history_server_api_url配置历史记录资源管理的地址和端口，对应mapred-site.xml-->
    <property>
    	<name>mapreduce.jobhistory.webapp.address</name>
    	<value>bigdata-pro01.bigDAta.com:19888</value>
    </property>
    

与Hive集成

• 修改hue.ini配置文件

  [beeswax]
  	  # Host where HiveServer2 is running.
        hive_server_host=bigdata-pro03.bigDAta.com
        # Port where HiveServer2 Thrift server runs on.
        hive_server_port=10000
        # Hive configuration directory, where hive-site.xml is located
        hive_conf_dir=/opt/modules/hive-2.1.0/conf
  

与MySql集成

• 修改hue.ini配置文件

  [[[mysql]]]
        nice_name="My SQL DB"
        # For MySQL and PostgreSQL, name is the name of the database.
        # For Oracle, Name is instance of the Oracle server. For express edition
        name=metastore
        engine=mysql
        host=bigdata-pro01.bigDAta.com
        port=3306
        user=root
        password=123456
  

  <!--出现了database is locked异常-->
  1.Hue的元数据库是sqlite，容易产生database is locked的异常,那就是mysql没有配置成功，查看上述配置，果然engine前的##忘记去掉了。
  <!--"Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)")-->
  2.不知怎么回事，host前面的##也没去掉
  mysql有两种连接方式
    a.通过socket连接，该方式支持本机连接，需要在my.cnf文件中配置正确的socket，配置项：socket 配置	内容：/tmp/mysql.sock
  	执行方式:mysql -u [用户名] -p [密码]
  	b.通过tcp连接，该方式支持远程连接
  	执行方式:mysql -h [数据库地址] -u [用户名] -p [密码]
  

  

与Hbase集成

• 修改hue.ini配置文件

  [hbase]
     hbase_clusters=(Cluster|bigdata-pro01.bigDAta.com:9090)
     # HBase configuration directory, where hbase-site.xml is located.
     hbase_conf_dir=/opt/modules/hbase-1.0.0-cdh5.4.0/conf
     thrift_transport=buffered
  

• 测试

  先启动hbase,再启动HBase中启动thrift服务 /opt/modules/hbase-1.0.0-cdh5.4.0/bin/hbase-daemon.sh start thrift 然后启动hue