Ambari安装与集群配置

参考网址：
http://www.ibm.com/developerworks/cn/opensource/os-cn-bigdata-ambari/index.html
http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/index.html

第一部分：基本环境准备

    准备3个节点，下面操作需要在每个节点上执行。
    参考网址http://docs.hortonworks.com/HDPDocuments/Ambari-2.4.2.0/bk_ambari-installation/content/prepare_the_environment.html

1、配置IP和hostname
    [root@node1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eno16777736
    [root@node1 ~]# vi /etc/hosts
    注意：不要从您的hosts文件中删除以下两行。 删除或编辑以下行可能会导致需要网络功能的各种程序失败。
    127.0.0.1 localhost.localdomain localhost
    ::1 localhost6.localdomain6 localhost6
    [root@node1 ~]# vi /etc/hostname
    [root@node1 ~]# service network restart
    
2、更改yum源
2.1 下载国内镜像源repo文件
    如果已经安装wget工具，则可以通过下面命令下，自动载配置
    [root@node1 ~]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
    --2017-01-02 21:44:57--  http://mirrors.aliyun.com/repo/Centos-7.repo
    Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 115.28.122.210, 112.124.140.210
    Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|115.28.122.210|:80... connected.
    HTTP request sent, awaiting response... 200 OK
    Length: 2573 (2.5K) [application/octet-stream]
    Saving to: ‘/etc/yum.repos.d/CentOS-Base.repo’
    100%[========================================================================>] 2,573       --.-K/s   in 0s      
    2017-01-02 21:44:58 (231 MB/s) - ‘/etc/yum.repos.d/CentOS-Base.repo’ saved [2573/2573]
    如果没有安装wget工具，直接下载http://mirrors.aliyun.com/repo/Centos-7.repo
    然后将下载的阿里镜像源文件移到/etc/yum.repos.d下即可（重命名CentOS-Base.repo）
    [root@node1 ~]# mv Centos-7.repo /etc/yum.repos.d/CentOS-Base.repo
2.2 更新yum缓存
    [root@node1 ~]# yum clean all
    Loaded plugins: fastestmirror
    Cleaning repos: base c7-media extras updates
    Cleaning up everything
    Cleaning up list of fastest mirrors
    [root@node1 ~]# yum makecache
    ......
    Determining fastest mirrors
     * base: mirrors.aliyun.com
     * extras: mirrors.aliyun.com
     * updates: mirrors.aliyun.com
    Metadata Cache Created

3、关闭防火墙和SELINUX
    [root@node1 ~]# systemctl disable firewalld
     Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
     Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
    [root@node1 ~]# service firewalld stop
     Redirecting to /bin/systemctl stop  firewalld.service
    [root@node1 ~]# setenforce 0
    [root@node1 ~]# vi /etc/selinux/config
     SELINUX=disabled
    设置umask值
    [root@node1 ~]echo umask 0022 >> /etc/profile
    
4、设置SSH无密钥登录

4.1 每个节点生成无密钥SSH

      [root@node1 ~]#yum install openssh-server

    [root@node1 ~]# ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
    [root@node1 ~]# cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
    [root@node1 ~]# chmod 0600 ~/.ssh/authorized_keys

    

4.2    相互访问
    [root@anode1 ~]#vi /etc/hosts
    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
    172.16.54.201 anode1
    172.16.54.202 anode2
    172.16.54.203 anode3
    [root@anode1 ~]#scp /etc/hosts 172.16.54.202:/etc/

    [root@anode1 ~]#scp /etc/hosts 172.16.54.203:/etc/

    然后在每个节点执行sh sshKeyGen.sh

    
5、设置时间同步
    [root@node1 ~]# yum install -y ntp
    [root@node1 ~]# systemctl enable ntpd
     Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.
    [root@node1 ~]# systemctl start ntpd

6、安装配置JDK 1.8
6.1 卸载预装的JDK
    rpm -qa|grep jdk
    yum -y remove java java-1.7.0-openjdk-1.7.0.79-2.5.5.4.el6.x86_64
    rpm -qa|grep java
    yum -y remove java tzdata-java-2015e-1.el6.noarch
6.2 配置新的JDK
    注意，每个节点的jdk安装到相同的目录下面，比如都安装到/opt目录下。
    [root@node1 ~]#vi /etc/profile
    #set Java Path
    export JAVA_HOME=/opt/java/jdk1.8.0_101
    export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib
    export PATH=$PATH:$JAVA_HOME/bin
    [root@node1 ~]# source /etc/profile
    [root@node1 ~]# java -version

7、拾遗

7.1 设置buildNumber

    这一部分是针对于ambari源码编译出现buildNumber缺失一种解决办法。

    [root@node1 ~]# vi .bash_profile
    export buildNumber=9999
    [root@anode1 ~]# source .bash_profile
    [root@node1 ~]# echo $buildNumber
    9999
7.2 确认是否安装了wget
    [root@anode1 ~]# yum install -y wget

    
第二部分 Ambari-server安装与配置

1、下载Ambari 的公共库文件（public repository）
    http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repo
    [root@anode2 ~]# wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repo -O /etc/yum.repos.d/ambari.repo
    2017-01-03 06:41:08 URL:http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari.repo [287/287] -> "/etc/yum.repos.d/ambari.repo" [1]
   
2、修改ambari.repo文件
    如果访问hortonworks.com速度顺畅，则不要修改ambari.repo，可以跳过这一步。
    当访问hortonworks.com速度较慢时，可以建立本地repository。
    [root@vnode0 yum.repos.d]# cat ambari.repo
    #VERSION_NUMBER=2.4.2.0-136
    [Updates-ambari-2.4.2.0]
    name=ambari-2.4.2.0 - Updates
    baseurl=http://192.168.1.180/ambari/centos7/2.4.2.0-136/
    gpgcheck=0
    gpgkey=http://192.168.1.180/ambari/centos7/2.4.2.0-136//RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    enabled=1

    priority=1

    然后将ambari.repo复制到其他节点
    [root@node1 yum.repos.d]# scp ambari.repo 172.16.54.202:/etc/yum.repos.d/

3、安装Ambari-server
    [root@east81 yum.repos.d]# yum install ambari-server
    Loaded plugins: fastestmirror, langpacks
    Loading mirror speeds from cached hostfile
    Resolving Dependencies

    ==================================================================================================================
     Package                                    Arch                            Version                                    Repository                                       Size
   =====================================================================================================================
    Installing:
     ambari-server                              x86_64                          2.4.2.0-136                                Updates-ambari-2.4.2.0                          645 M
    Installing for dependencies:
     postgresql                                 x86_64                          9.2.13-1.el7_1                             os                                              3.0 M
     postgresql-libs                            x86_64                          9.2.13-1.el7_1                             os                                              230 k
     postgresql-server                          x86_64                          9.2.13-1.el7_1                             os                                              3.8 M

    Transaction Summary
    ======================================================================================================================
    Install  1 Package (+3 Dependent packages)

    Total download size: 652 M
    Installed size: 732 M
    Is this ok [y/d/N]: y
    Downloading packages:
    (1/4): postgresql-libs-9.2.13-1.el7_1.x86_64.rpm                                                                                                      | 230 kB  00:00:00     
    (2/4): postgresql-9.2.13-1.el7_1.x86_64.rpm                                                                                                           | 3.0 MB  00:00:00     
    (3/4): postgresql-server-9.2.13-1.el7_1.x86_64.rpm                                                                                                    | 3.8 MB  00:00:00     
    ambari-server-2.4.2.0-136.x86_ FAILED                                                                                                     ]  159 B/s | 7.8 MB 1180:43:26 ETA
    http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: [Errno 12] Timeout on http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
    Trying other mirror.
    ambari-server-2.4.2.0-136.x86_ FAILED                                          -                                                           ]  437 B/s |  31 MB 414:04:07 ETA
    http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: [Errno 12] Timeout on http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.2.0/ambari/ambari-server-2.4.2.0-136.x86_64.rpm: (28, 'Operation too slow. Less than 1000 bytes/sec transferred the last 30 seconds')
    Trying other mirror.
    warning: /var/cache/yum/x86_64/7/Updates-ambari-2.4.2.0/packages/ambari-server-2.4.2.0-136.x86_64.rpm: Header V4 RSA/SHA1 Signature, key ID 07513cad: NOKEY MB  00:00:00 ETA
    Public key for ambari-server-2.4.2.0-136.x86_64.rpm is not installed
    (4/4): ambari-server-2.4.2.0-136.x86_64.rpm                                                                                                           | 645 MB  00:17:12     
    -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                        295 kB/s | 652 MB  00:37:44     
    Retrieving key from http://public-repo-1.hortonworks.com/ambari/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    Importing GPG key 0x07513CAD:
     Userid     : "Jenkins (HDP Builds) <jenkin@hortonworks.com>"
     Fingerprint: df52 ed4f 7a3a 5882 c099 4c66 b973 3a7a 0751 3cad
     From       : http://public-repo-1.hortonworks.com/ambari/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
    Is this ok [y/N]: y
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : postgresql-libs-9.2.13-1.el7_1.x86_64                                                                                                           1/4
      Installing : postgresql-9.2.13-1.el7_1.x86_64                                                                                                                2/4
      Installing : postgresql-server-9.2.13-1.el7_1.x86_64                                                                                                         3/4
      Installing : ambari-server-2.4.2.0-136.x86_64                                                                                                                4/4
      Verifying  : postgresql-9.2.13-1.el7_1.x86_64                                                                                                                1/4
      Verifying  : ambari-server-2.4.2.0-136.x86_64                                                                                                                2/4
      Verifying  : postgresql-libs-9.2.13-1.el7_1.x86_64                                                                                                           3/4
      Verifying  : postgresql-server-9.2.13-1.el7_1.x86_64                                                                                                         4/4
    Installed:
      ambari-server.x86_64 0:2.4.2.0-136                                                                                                                                         
    Dependency Installed:
      postgresql.x86_64 0:9.2.13-1.el7_1              postgresql-libs.x86_64 0:9.2.13-1.el7_1               postgresql-server.x86_64 0:9.2.13-1.el7_1                  
    Complete!
    [root@east81 yum.repos.d]#
4、设置Ambari-server
    [root@anode1 opt]# ambari-server setup
    Using python  /usr/bin/python
    Setup ambari-server
    Checking SELinux...
    SELinux status is 'disabled'
    Customize user account for ambari-server daemon [y/n] (n)? y    #输入y
    Enter user account for ambari-server daemon (root):             #回车，默认值
    Adjusting ambari-server permissions and ownership...
    Checking firewall status...
    Checking JDK...
    [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
    [2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
    [3] Custom JDK
    ==============================================================================
    Enter choice (1): 3                                             #输入3
    WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
    WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
    Path to JAVA_HOME: /opt/jdk1.8.0_101                            #输入JDK根目录
    Validating JDK on Ambari Server...done.
    Completing setup...
    Configuring database...
    Enter advanced database configuration [y/n] (n)?                #回车，默认PostgreSQL
    Configuring database...
    Default properties detected. Using built-in database.
    Configuring ambari database...
    Checking PostgreSQL...
    Running initdb: This may take up to a minute.
    Initializing database ... OK
    About to start PostgreSQL
    Configuring local database...
    Connecting to local database...done.
    Configuring PostgreSQL...
    Restarting PostgreSQL
    Extracting system views...
    ambari-admin-2.4.2.0.0.jar
    Adjusting ambari-server permissions and ownership...
    Ambari Server 'setup' completed successfully.
    [root@anode1 opt]#

5、启动Ambari-server
    [root@anode1 opt]# ambari-server start
    Using python  /usr/bin/python
    Starting ambari-server
    Ambari Server running with administrator privileges.
    Organizing resource files at /var/lib/ambari-server/resources...
    Ambari database consistency check started...
    No errors were found.
    Ambari database consistency check finished
    Server PID at: /var/run/ambari-server/ambari-server.pid
    Server out at: /var/log/ambari-server/ambari-server.out
    Server log at: /var/log/ambari-server/ambari-server.log
    Waiting for server start....................
    Ambari Server 'start' completed successfully.
    
第三部分 集群配置

1、登录Ambari-server
    比如，Ambari-server所在节点node1的IP是172.16.54.201
    在浏览器中打开http://172.16.54.201:8080
    用户名和密码都是admin,即可登录Ambari-server
2、单击“Launch Install Wizard”按钮，启动Ambari安装向导
    (1) 在“Name your cluster”输入框输入集群名称，比如cc
    (2) Select Version选择版本,选择HDP2.4
        如果“使用公共存储库”，保留“redhat7”即可，其他系统删除。
        如果“使用本地存储库”，同样保留“redhat7”即可，删除其他系统。修改：
        HDP-2.4输入：http://192.168.1.180/hdp/HDP/centos7/2.x/updates/2.4.3.0/

        HDP-UTILS-1.1.0.20输入：http://192.168.1.180/hdp/HDP-UTILS-1.1.0.20/repos/centos7/

        提示：会在各个节点生成hdp repo文件

       [root@a1 yum.repos.d]# ls
       ambari.repo  bak  CentOS-Base.repo  HDP.repo  HDP-UTILS.repo 

       [root@a1 yum.repos.d]# more HDP.repo
       [HDP-2.4]
       name=HDP-2.4
       baseurl=http://192.168.1.60/hdp/HDP/centos7/2.x/updates/2.4.3.0/
       path=/
       enabled=1
       gpgcheck=0
       [root@a1 yum.repos.d]# more HDP-UTILS.repo
       [HDP-UTILS-1.1.0.20]
       name=HDP-UTILS-1.1.0.20
       baseurl=http://192.168.1.60/hdp/HDP-UTILS-1.1.0.20/repos/centos7/
       path=/
       enabled=1
       gpgcheck=0

    (3)Install Options安装选项
        在Target Hosts：输入节点的hostname值，不是IP。比如：
        anode1
        anode2
        anode3
    (4) Host Registration Information主机注册信息
        将Ambari-server主机的SSH私钥复制到输入框
        [root@anode1 ~]# cat /root/.ssh/id_rsa
    (5) 确认主机，进行anode1、anode2、anode3节点注册
    (6)    选择服务,先选择最小服务安装,后续再安装其他服务。
        建议选择ZooKeeper和Ambari Metrics
    (7)设置Grafana Admin Password
        输入123456即可
    (8)分配从属机和客户端,所有节点选择安装Client

    
第四部分：遇到问题

1、警告 Ambari Metrics     hbase_master_heapsize     512     
    Value is less than the recommended default of 640
    HBase Master Heap Size. In embedded mode, total heap size is sum of master and regionserver heap sizes.
    解决办法：
    Ambari Metrics配置，检索hbase_master_heapsize，然后将值修改为640，单击保存按钮
    最后重启组件。

 2、ambari 报Heartbeat Lost问题

     重启ambari-agent 

     [root@anode1 ~]# ambari-server start