HDP 2.4 离线安装

本文介绍在CentOS 6环境下离线安装HDP 2.4(两个节点)。主要步骤如下，

1 安装前配置

(1)配置host name，同步集群每个节点

[root@esggy-del-n001 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.10.22.151    esggy-del-n001.novalocal        esggy-del-n001
10.10.22.152    esggy-del-n002.novalocal        esggy-del-n002
[root@esggy-del-n001 ~]# scp /etc/hosts esggy-del-n002:/etc/

(2)配置集群节点间免密码，在此不作详细说明

(3)关闭防火墙，每个节点相同

[root@esggy-del-n001 ~]# service iptables stop
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Unloading modules:                               [  OK  ]
[root@esggy-del-n001 ~]# chkconfig iptables off

(4)关闭selinux，每个节点相同

[root@esggy-del-n001 ~]# setenforce 0
[root@esggy-del-n001 ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

(5) 安装并打开ntp服务，每个节点相同

[root@esggy-del-n001 ~]# yum install -y ntp
[root@esggy-del-n001 ~]# service ntpd start
Starting ntpd:                                             [  OK  ]
[root@esggy-del-n001 ~]# chkconfig ntpd on

(6)安装JDK，每个节点相同

[root@esggy-del-n001 opt]# rpm -ivh jdk-8u11-linux-x64.rpm
Preparing...                ########################################### [100%]
   1:jdk                    ########################################### [100%]
Unpacking JAR files...
        rt.jar...
        jsse.jar...
        charsets.jar...
        tools.jar...
        localedata.jar...
        jfxrt.jar...

2 准备HDP 离线安装包

[root@esggy-del-n001 hdp]# ll
total 8
drwxr-xr-x. 3 root root 4096 Jun 30 18:13 ambari2.4
drwxr-xr-x. 3 root root 4096 Jun 30 17:24 hdp2.4
[root@esggy-del-n001 hdp]# cd ambari2.4/
[root@esggy-del-n001 ambari2.4]# ls
ambari-agent-2.4.1.0-22.x86_64.rpm              ambari-metrics-hadoop-sink-2.4.1.0-22.x86_64.rpm
ambari-metrics-collector-2.4.1.0-22.x86_64.rpm  ambari-metrics-monitor-2.4.1.0-22.x86_64.rpm
ambari-metrics-common-2.4.1.0-22.noarch.rpm     ambari-server-2.4.1.0-22.x86_64.rpm
ambari-metrics-grafana-2.4.1.0-22.x86_64.rpm

3 createpo创建yum本地源

[root@esggy-del-n001 hdp]# yum install -y createrepo
[root@esggy-del-n001 hdp]# createrepo .
Spawning worker 0 with 186 pkgs
Workers Finished
Gathering worker results

Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

4 生成临时HTTP服务并准备本地ambari repo文件

[root@esggy-del-n001 hdp]# python -m SimpleHTTPServer 8910 &
[1] 2154
[root@esggy-del-n001 hdp]# Serving HTTP on 0.0.0.0 port 8910 ...
[root@esggy-del-n001 yum.repos.d]# pwd
/etc/yum.repos.d
[root@esggy-del-n001 yum.repos.d]# cat ambari.repo
[ambari]
baseurl=http://10.10.22.151:8910
enabled=1
gpgcheck=0

5 从本地yum源安装ambari安装包

[root@esggy-del-n001 ambari2.4]# yum install -y ambari-server
...
Installed:
  ambari-server.x86_64 0:2.4.1.0-22

Dependency Installed:
  postgresql.x86_64 0:8.4.20-7.el6      postgresql-libs.x86_64 0:8.4.20-7.el6      postgresql-server.x86_64 0:8.4.20-7.el6

Complete!

6 配置ambari服务(ambari-server setup)

[root@esggy-del-n001 ambari2.4]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'enabled'
SELinux mode is 'permissive'
WARNING: SELinux is set to 'permissive' mode and temporarily disabled.
OK to continue [y/n] (y)? y
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
==============================================================================
Enter choice (1): 3
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_11
Validating JDK on Ambari Server...done.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? n
Configuring database...
Default properties detected. Using built-in database.
Configuring ambari database...
Checking PostgreSQL...
Running initdb: This may take up to a minute.
Initializing database: [  OK  ]

About to start PostgreSQL
Configuring local database...
Connecting to local database...done.
Configuring PostgreSQL...
Restarting PostgreSQL
Extracting system views...
......ambari-admin-2.4.1.0.22.jar
.....
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

7 启动ambari服务，并检查8080端口是否成功监听(ambari-server start)

[root@esggy-del-n001 ambari2.4]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
No errors were found.
Ambari database consistency check finished
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start....................
Ambari Server 'start' completed successfully.

[root@esggy-del-n001 ambari2.4]# netstat -nlp | grep 8080
tcp        0      0 :::8080                     :::*                        LISTEN      3352/java

8 打开ambari网页部署Hadoop集群，登录默认用户 admin/admin

9 执行Hadoop安装

10 至此，HDP hadoop集群安装完毕，最后需要关闭本地源

[root@esggy-del-n001 centos6]# ps -ef | grep 8910
root      2154  1513  0 18:16 pts/0    00:00:06 python -m SimpleHTTPServer 8910
root      7486  1513  0 18:40 pts/0    00:00:00 grep 8910
[root@esggy-del-n001 centos6]# kill -9 2154
[root@esggy-del-n001 centos6]# ps -ef | grep 8910
root      7495  1513  0 18:40 pts/0    00:00:00 grep 8910
[root@esggy-del-n001 centos6]# rm -rf /etc/yum.repos.d/ambari.repo