Ambari集成Hadoop安装

1、安装准备

1.1 操作系统

CentOS6.5

1.2. 文件准备

ambari安装有两种方式：
1.在线安装
2.离线安装
官网提供的在线安装的repository地址不正确， 需要修改， 另外在线安装很慢， 所以成功机率很低， 本文介绍离线安装，需要下载3个安装包：HDP-2.0.6.1-centos6-rpm.tar.gz、HDP-UTILS-1.1.0.16-centos6.tar.gz、ambari-1.4.4.23-centos6.tar.gz，下载地址如下
http://s3.amazonaws.com/public-repo-1.hortonworks.com/HDP/centos6/HDP-2.0.6.1-centos6-rpm.tar.gz
http://s3.amazonaws.com/public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.16/repos/centos6/HDP-UTILS-1.1.0.16-centos6.tar.gz
http://s3.amazonaws.com/public-repo-1.hortonworks.com/ambari/centos6/ambari-1.4.4.23-centos6.tar.gz

2.服务器配置

(与《基础准备（Hadoop/Spark/Mahout安装准备）》一致)

2.1. 关闭防火墙

[root@master ~]# service iptables stop
#永久关闭
[root@master ~]# chkconfig iptables on/off

2.2. 关闭selinux

[root@master ~]# setenforce 0
[root@master ~]# vi /etc/selinux/config
SELINUX=enforcing
#改为
SELINUX= disabled

2.3. 时间同步

[root@master ~]# cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
[root@master ~]# ntpdate us.pool.ntp.org
#如示安装ntpdate，可以使用yum install ntpdate

[root@master ~]# yum install ntpdate

2.4.配置主机名

[root@master ~]# vi /etc/sysconfig/network

HOSTNAME=localhost

#改为
HOSTNAME=master

2.5. 配置域名(hosts)

[root@master ~]# vi /etc/hosts
#增加如下信息，此处根据自己实际IP进行配置，其它slave节点也需要配置该信息
192.168.2.140 master.hadoop
192.168.2.141 slave1.hadoop
192.168.2.142 slave2.hadoop
192.168.2.143 slave3.hadoop

2.6. SSH配置

[root@master ~]# ssh-keygen -t rsa
#下面直接回车既可
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
5b:59:bf:2e:20:bf:d6:81:a9:ae:8d:3e:66:78:38:03 root@master
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|            .    |
|           o .   |
|        S oo  .  |
|  E     .o+ .  . |
|   . o  .+ o ..  |
|    = =o. o o.   |
|     B++o... ..  |
+-----------------+
[root@master ~]# chmod 700 .ssh
[root@master ~]# cat ~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys
[root@master ~]# chmod 600 ~/.ssh/authorized_keys
#修改无密码登陆配置文件
[root@master ~]# vi /etc/ssh/sshd_config
#默认前面都有个 "#"， 去掉既可
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
#测试ssh是否可用
[root@master ~]# ssh localhost
#如此处不需要输入密码， 说明配置成功。 重复以上步骤完成其它节点上SSH无密码登陆
#将生成的公钥复制到其它节点上
[root@master ~]# scp ~/.ssh/id_rsa.pub slave1.hadoop:/root/
#将公钥追加到authorized_keys中
[root@master ~]#cat ~/id_rsa.pub>>~/.ssh/authorized_keys

2.7.启动httpd

[root@master ~]#service httpd start

3.Ambari安装配置

3.1. 上传安装文件至master

上传下载的3个.tar.gz文件到master的/var/www/html文件夹，然后解压缩
[root@master ambari]# cd /var/www/html/
[root@master html]# ls -l
total 1859620
-rw-r--r--. 1 root root   42346541 Mar 26 18:17 ambari-1.4.4.23-centos6.tar.gz
-rw-r--r--. 1 root root 1835314916 Mar 27 09:44 HDP-2.0.6.1-centos6-rpm.tar.tar
-rw-r--r--. 1 root root   26581600 Mar 26 18:07 HDP-UTILS-1.1.0.16-centos6.tar.gz
[root@master html]# gzip -d ambari-1.4.4.23-centos6.tar.gz 
[root@master html]# gzip -d HDP-UTILS-1.1.0.16-centos6.tar.gz 
[root@master html]# tar -xf HDP-2.0.6.1-centos6-rpm.tar.tar
#解压出所有文件
[root@master html]# ls -l
total 1861116
drwxr-xr-x  3 root   root         4096 Mar 27 16:03 ambari
-rw-r--r--  1 root   root     42987520 Mar 26 18:17 ambari-1.4.4.23-centos6.tar
drwxr-xr-x  3 root   root         4096 Mar 27 15:51 HDP
-rw-r--r--. 1 root   root   1835314916 Mar 27 09:44 HDP-2.0.6.1-centos6-rpm.tar.tar
drwxr-xr-x  3 hadoop hadoop       4096 Aug 21  2013 HDP-UTILS-1.1.0.16
-rw-r--r--  1 root   root     27463680 Mar 26 18:07 HDP-UTILS-1.1.0.16-centos6.tar

3.2.配置repo

1.将：
ambari\centos6\1.x\updates\1.4.4.23\ambari.repo
HDP\centos6\2.x\updates\2.0.6.1\hdp.repo
HDP-UTILS-1.1.0.16\repos\centos6\hdp-util.repo
文件到/etc/yum.repo.d/文件夹下面
[root@master html]# cp ambari/centos6/1.x/updates/1.4.4.23/ambari.repo /etc/yum.repos.d/
[root@master html]# cp HDP-UTILS-1.1.0.16/repos/centos6/hdp-util.repo /etc/yum.repos.d/
[root@master html]# cp HDP/centos6/2.x/updates/2.0.6.1/hdp.repo /etc/yum.repos.d/
[root@master html]# cd /etc/yum.repos.d/
#修改ambari.repo配置如下：
[Updates-ambari-1.4.4.23]
name=ambari-1.4.4.23 - Updates
baseurl=http://master.hadoop/ambari/centos6/1.x/updates/1.4.4.23
gpgcheck=0
gpgkey=http://master.hadoop/ambari/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
#修改hdp.repo配置如下：
[Updates-HDP-2.x]
name=Hortonworks Data Platform HDP-2.x - Updates
baseurl=http://master.hadoop/HDP/centos6/2.x/updates/2.0.6.1/
gpgcheck=0
gpgkey=http://master.hadoop/HDP/centos6/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
#修改hdp-util.repo配置如下：
[HDP-UTILS-1.1.0.16]
name=Hortonworks Data Platform Version - HDP-UTILS-1.1.0.16
baseurl= http://master.hadoop/HDP-UTILS-1.1.0.16/repos/centos6
gpgcheck=0
enabled=1
priority=1

3.3. 安装ambari-server

[root@master 1.4.4.23]# yum install ambari-server

3.4.配置ambari-server

[root@master 1.4.4.23]# ambari-server setup

[root@master ~]# ambari-server setup
Using python  /usr/bin/python2.6
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Ambari-server daemon is configured to run under user 'root'. Change this setting [y/n] (n)? 
Adjusting ambari-server permissions and ownership...
Checking iptables...
Checking JDK...
JDK already exists, using /var/lib/ambari-server/resources/jdk-6u31-linux-x64.bin
Installing JDK to /usr/jdk64
Successfully installed JDK to /usr/jdk64/jdk1.6.0_31
Downloading JCE Policy archive from http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-6.zip to /var/lib/ambari-server/resources/jce_policy-6.zip
JCE Policy files are required for secure HDP setup. Please ensure  all hosts have the JCE unlimited strength policy 6, files.
ERROR: Failed to download JCE policy files:
ERROR: Reason:  Failed to download JCE Policy archive : Request header doesn't contain Content-Length. Please check that JCE Policy archive is available at http://public-repo-1.hortonworks.com/ARTIFACTS/jce_policy-6.zip . Also you may install JCE Policy archive manually using --jce-policy command line argument.
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
==============================================================================
Enter choice (1): 1
Database Name (ambari): 
Username (ambari): 
Enter Database Password (bigdata): 
Default properties detected. Using built-in database.
Checking PostgreSQL...
Running initdb: This may take upto a minute.
Initializing database: [  OK  ]


About to start PostgreSQL
Configuring local database...
Connecting to the database. Attempt 1...
Configuring PostgreSQL...
Restarting PostgreSQL
Ambari Server 'setup' completed successfully.

1、给ambari配置本地hadoo源地址：
[root@ambari ~]# cd /var/lib/ambari-server/resources/stacks/HDPLocal/2.0.6/repos/
[root@ambari repos]# vim repoinfo.xml  修改红色部分为本地源地址
<reposinfo>
  <os type="centos6">
    <repo>
      <baseurl>http://172.16.235.164/hdp/</baseurl>
      <repoid>HDP-2.0.6</repoid>
      <reponame>HDP</reponame>
    </repo>
  </os>

2、拷贝http源到vm2、vm3；
scp /etc/yum.repos.d/* root@vm2:/etc/yum.repos.d/
scp /etc/yum.repos.d/* root@vm3:/etc/yum.repos.d/

3.5.页面配置过程

启动ambari-server
[root@master 1.4.4.23]# ambari-server sart
通过浏览器访问
master.hadoop:8080
用户名/密码：admin/admin
界面配置

3.5.1. welcome

 

3.5.2. Select Stack

 

3.5.3. Install Options

 

3.5.4. Comfirm Hosts

 

3.5.5. Choose Services

 

3.5.6. Assign Masters

 

3.5.7. Assign Slaves and Clients

 

3.5.8. Customize Services

Customize Services(1)

 

Customize Service(2)

 

3.5.9. Review

 

 

3.5.10.Install Start and Test

 

 

3.6. Ambari集群界面

 

图1 Ambari Hadoop(主界面)

 

图2 Ambari HDFS

 

图3 Ambari HBase

 

图4 Ambari Hosts(集群)

转载请注明出处：

http://blog.csdn.net/sunbow0/article/details/41961657