大数据分析工具Hue安装及集成各种组件

最新推荐文章于 2024-09-06 10:52:14 发布

终有一刻

最新推荐文章于 2024-09-06 10:52:14 发布

阅读量792

点赞数 8

分类专栏：大数据文章标签：数据分析数据挖掘

本文链接：https://blog.csdn.net/weixin_42732268/article/details/140800999

版权

大数据专栏收录该内容

5 篇文章 0 订阅

订阅专栏

文章目录

1、Hue安装

1.1、下载安装包
Hue的安装支持多种方式，包括rpm包的方式进行安装、tar.gz包的方式进行安装以及cloudera manager的方式来进行安装等，我们这里使用tar.gz包的方式来进行安装
Hue的压缩包的下载地址：
http://archive.cloudera.com/cdh5/cdh/5/
我们这里使用的是CDH5.14.0这个对应的版本，具体下载地址为
http://archive.cloudera.com/cdh5/cdh/5/hue-3.9.0-cdh5.14.0.tar.gz
通过百度网盘分享的文件：hue-3.9.0-cdh5.14.0.tar.gz
链接：https://pan.baidu.com/s/1GDNVLAvF_cs7vwLhmRKL7w
提取码：1i4y

1.2、使用WINSC 上传文件至/opt/tools,使用CP命令复制到/opt/sever
再使用tar -zxf hue-3.9.0-cdh5.14.0.tar.gz 进行解压

[root@zhang server]# ls
flume  hadoop-2.8.0  hive-3.1.2  hue-3.9.0-cdh5.14.0

1.3、安装各种依赖包

yum install -y asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make openldap-devel python-devel sqlite-devel gmp-devel

1.4、Hue初始化配置文件

[root@zhang conf]# pwd
/opt/server/hue-3.9.0-cdh5.14.0/desktop/conf
[root@zhang conf]# vi hue.ini 
[desktop]

  # Set this to a random string, the longer the better.
  # This is used for secure hashing in the session store.
  secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o

  # Execute this script to produce the Django secret key. This will be used when
  # 'secret_key' is not set.
  ## secret_key_script=

  # Webserver listens on this address and port
  http_host=zhang
  http_port=8888

  # Choose whether to enable the new Hue 4 interface.
  ## is_hue_4=true

  # Choose whether to still allow users to enable the old Hue 3 interface.
  ## disable_hue_3=false

  # A comma-separated list of available Hue load balancers
  ## hue_load_balancer=

  # Time zone name
  time_zone=America/Los_Angeles

  # Enable or disable Django debug mode.
  django_debug_mode=false

  # Enable development mode, where notably static files are not cached.
  ## dev=false

  # Enable or disable database debug mode.
  ## database_logging=false
# Whether to send debug messages from JavaScript to the server logs.
  ## send_dbug_messages=false
  # Enable or disable backtrace for server error
  http_500_debug_mode=false
  # Enable or disable memory profiling.
  ## memory_profiler=false
  # Server email for internal error messages
  ## django_server_email='hue@localhost.localdomain'
  # Email backend
  ## django_email_backend=django.core.mail.backends.smtp.EmailBackend
  # Webserver runs as this user
 server_user=root
 server_group=root
  # This should be the Hue admin and proxy user
 default_user=root
  # This should be the hadoop cluster admin
  ## default_hdfs_superuser=root


[[database]]
    # Database engine is typically one of:
    # postgresql_psycopg2, mysql, sqlite3 or oracle.
    #
    # Note that for sqlite3, 'name', below is a path to the filename. For other backends, it is the database name
    # Note for Oracle, options={"threaded":true} must be set in order to avoid crashes.
    # Note for Oracle, you can use the Oracle Service Name by setting "host=" and "port=" and then "name=<host>:<port>/<service_name>".
    # Note for MariaDB use the 'mysql' engine.
     engine=mysql
     host=zhang
     port=3306
    user=root
    password=root
    # conn_max_age option to make database connection persistent value in seconds
    # https://docs.djangoproject.com/en/1.9/ref/databases/#persistent-connections
    ## conn_max_age=0
    # Execute this script to produce the database password. This will be used when 'password' is not set.
    ## password_script=/path/script
    name=mysql
    #options={"threaded":true}
    # Database schema, to be used only when public schema is revoked in postgres
    ## schema=public
  # Configuration options for specifying the Desktop session.
  # For more info, see https://docs.djangoproject.com/en/1.4/topics/http/sessions/
  # ------------------------------------------------------------------------

1.5、创建mysql中Hue使用的DB

create database hue default character set utf8 default collate utf8_general_ci;
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| sys                |
+--------------------+
4 rows in set (0.01 sec)

mysql>

1.6、编译Hue

[root@zhang hue-3.9.0-cdh5.14.0]# make apps

编译成功之后，会在hue数据库中创建许多初始化表

2、启动Hue

前台启动：
./build/env/bin/supervisor

[root@zhang hue-3.9.0-cdh5.14.0]# cd /opt/server/hue-3.9.0-cdh5.14.0/
root@zhang hue-3.9.0-cdh5.14.0]# ./build/env/bin/supervisor
Traceback (most recent call last):
  File "./build/env/bin/supervisor", line 9, in <module>
    load_entry_point('desktop==3.9.0', 'console_scripts', 'supervisor')()
  File "/opt/server/hue-3.9.0-cdh5.14.0/desktop/core/src/desktop/supervisor.py", line 319, in main
    setup_user_info()
  File "/opt/server/hue-3.9.0-cdh5.14.0/desktop/core/src/desktop/supervisor.py", line 257, in setup_user_info
    desktop.lib.daemon_utils.get_uid_gid(SETUID_USER, SETGID_GROUP)
  File "/opt/server/hue-3.9.0-cdh5.14.0/desktop/core/src/desktop/lib/daemon_utils.py", line 45, in get_uid_gid
    raise KeyError("Couldn't get user id for user %s" % (username,))
KeyError: "Couldn't get user id for user hue"
[root@zhang hue-3.9.0-cdh5.14.0]# 
[root@zhang hue-3.9.0-cdh5.14.0]# su zhang
[zhang@zhang hue-3.9.0-cdh5.14.0]$ ./build/env/bin/supervisor
[INFO] Not running as root, skipping privilege drop
starting server with options:
{'daemonize': False,
 'host': 'zhang',
 'pidfile': None,
 'port': 8888,
 'server_group': 'root',
 'server_name': 'localhost',
 'server_user': 'root',
 'ssl_certificate': None,
 'ssl_certificate_chain': None,
 'ssl_cipher_list': 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA',
 'ssl_private_key': None,
 'threads': 50,
 'workdir': None}
WARNING: Couldn't write lextab module 'pycparser.lextab'. [Errno 13] Permission denied: 'lextab.py'
WARNING: yacc table file version is out of date
WARNING: Couldn't create 'pycparser.yacctab'. [Errno 13] Permission denied: 'yacctab.py'

3、WEB界面访问

地址：http://192.168.66.128:8888/hue/editor?editor=185
在这里插入图片描述

4、集成Hive

4.1、如果需要配置hue与hive的集成，我们需要启动hive的metastore服务以及hiveserver2服务（impala需要hive的metastore服务，hue需要hive的hiveserver2服务）

zhang@zhang bin]$ ./hive --service hiveserver2 &
[1] 5411
[zhang@zhang bin]$ which: no hbase in (/opt/server/hive-3.1.2/bin:/opt/server/flume/bin:/usr/lib/jvm/jdk1.8.0_65/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/lib/jvm/jdk1.8.0_65/bin:/opt/server/hadoop-2.8.0/bin:/opt/server/hadoop-2.8.0/sbin:/home/zhang/.local/bin:/home/zhang/bin)
2024-07-30 02:09:15: Starting HiveServer2
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/server/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/server/hadoop-2.8.0/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 8fe5cc1a-82a5-4227-9b6b-6f85d3c2c0ed
Hive Session ID = f77969f0-78fa-4de1-a881-abc12454bc5d
Hive Session ID = 2eea9353-057d-48cd-9fe6-04880cd8b298
Hive Session ID = 99227d48-0ebe-4d32-8514-a30894cd9012
OK
OK
OK
OK
OK
OK
OK
OK

具体可查看安装Hadoop组件Hive,配置Oracle存储元数据

4.2、修改Hue.ini

997 [beeswax]
    998 
    999   # Host where HiveServer2 is running.
   1000   # If Kerberos security is enabled, use fully-qualified domain name (FQDN).
   1001   hive_server_host=zhang
   1002 
   1003   # Port where HiveServer2 Thrift server runs on.
   1004   hive_server_port=10000
   1005 
   1006   # Hive configuration directory, where hive-site.xml is located
   1007    hive_conf_dir=/opt/server/hive-3.1.2/conf/
   1008 
   1009   # Timeout in seconds for thrift calls to Hive service
   1010   server_conn_timeout=120
   1011 
   1012   # Choose whether to use the old GetLog() thrift call from before Hive 0.14 to retrieve the logs.
   1013   # If false, use the FetchResults() thrift call from Hive 1.0 or more instead.
   1014   ## use_get_log_api=false
   1015 
   1016   # Limit the number of partitions that can be listed.
   1017   ## list_partitions_limit=10000
   1018 
   1019   # The maximum number of partitions that will be included in the SELECT * LIMIT sample query for partitioned tables.
   1020   ## query_partitions_limit=10
   1021 
   1022   # A limit to the number of rows that can be downloaded from a query before it is truncated.
   1023   # A value of -1 means there will be no limit.
   1024   ## download_row_limit=100000
1040   # Override the default desktop username and password of the hue user used for authentications with other services.
   1041   # e.g. Used for LDAP/PAM pass-through authentication.
   1042   auth_username=root
   1043   auth_password=root

5、启动Hue访问Hive

./build/env/bin/supervisor

Web访问：http://192.168.66.128:8888/hue/jobbrowser#!jobs
在这里插入图片描述

终有一刻

关注

8
点赞
踩
18

收藏

觉得还不错? 一键收藏
打赏
0
评论
大数据分析工具Hue安装及集成各种组件

4.1、如果需要配置hue与hive的集成，我们需要启动hive的metastore服务以及hiveserver2服务（impala需要hive的metastore服务，hue需要hive的hiveserver2服务）Hue的安装支持多种方式，包括rpm包的方式进行安装、tar.gz包的方式进行安装以及cloudera manager的方式来进行安装等，我们这里使用tar.gz包的方式来进行安装。地址：http://192.168.66.128:8888/hue/editor?4.2、修改Hue.ini。
复制链接

扫一扫