centos7安装ambari2.7.4

  • 起三台centos7虚拟机

  • 安装好yum源

  • 配置hosts文件

    172.28.128.16 master.ambari3.0.com master
    172.28.128.17 node01.ambari3.0.com node01
    172.28.128.18 node02.ambari3.0.com node02
    
  • 关闭防火墙

    systemctl stop firewalld
    systemctl disable firewalld
    
  • 配置master与其他两个节点免密登录

    # master:
    ssh-keygen
    cd ~/.ssh/
    cat id_rsa.pub >> authorized_keys
    chmod 600 ~/.ssh
    chmod 600 ~/.ssh/authorized_keys
    # node01
    mkdir ~/.ssh/
    # node02
    mkdir ~/.ssh/
    # master
    scp /root/.ssh/authorized_keys root@master:/root/.ssh/authorized_keys
    scp /root/.ssh/authorized_keys root@node01:/root/.ssh/authorized_keys
    scp /root/.ssh/authorized_keys root@node02:/root/.ssh/authorized_keys
    
  • 免密登录测试

    ssh master
    ssh node01
    ssh node02
    
  • 配置时间同步

    yum -y install ntp
    
    ntpdate cn.pool.ntp.org
    
  • 配置阿里云的镜像地址

    mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
    curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-8.repo
    yum makecache
    
  • 关闭selinux

    vim /etc/sysconfig/selinux
    SELINUX=disabled
    
  • 重启机器即可

  • 关闭THP

    • 什么是THP?

      透明大页面(THP)是一种Linux内存管理系统,通过使用更大的内存页面,可以减少具有大量内存的计算机上的Translation Lookaside Buffer(TLB)查找的开销。但是,数据库工作负载通常在THP上表现不佳,因为它们往往具有稀疏而不是连续的内存访问模式。 您应该在Linux机器上禁用THP以确保使用MongoDB获得最佳性能。
      
    • 查询THP状态

      cat /sys/kernel/mm/transparent_hugepage/defrag
      cat /sys/kernel/mm/transparent_hugepage/enabled
      
    • 若结果为 [always] 则表示启动了THP

    • 永久关闭THP

      vim /etc/rc.d/rc.local
      
    • 在文件末尾添加

      if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
              echo never > /sys/kernel/mm/transparent_hugepage/enabled
      fi
      if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
              echo never > /sys/kernel/mm/transparent_hugepage/defrag
      fi
      
    • 并赋予rc.local文件执行权限

      chmod +x /etc/rc.d/rc.local
      
    • 重启机器

    • 再看下THP状态

      cat /sys/kernel/mm/transparent_hugepage/defrag
      cat /sys/kernel/mm/transparent_hugepage/enabled
      
    • 若结果显示为 [never] 则表明关闭成功

  • 最大打开文件设置

    • 检查最大文件描述符数的当前值

      ulimit -Sn
      ulimit -Hn
      
    • 如若输出小于10000,则将其设置为10000

      ulimit -n 10000
      
  • [master]安装mysql

    • [master]安装Mysql数据库

      yum install mariadb-server -y
      systemctl start mariadb
      systemctl enable mariadb
      #数据库初始化设置
      mysql_secure_installation
      #首先是设置密码,会提示先输入密码
      Enter current password for root (enter for none):<–初次运行直接回车
      #设置密码
      Set root password? [Y/n] <– 是否设置root用户密码,输入y并回车或直接回车
      New password: <– 设置root用户的密码
      Re-enter new password: <– 再输入一次你设置的密码
      #其他配置
      Remove anonymous users? [Y/n] <– 是否删除匿名用户,回车
      Disallow root login remotely? [Y/n] <–是否禁止root远程登录,回车,
      Remove test database and access to it? [Y/n] <– 是否删除test数据库,回车
      Reload privilege tables now? [Y/n] <– 是否重新加载权限表,回车
      #初始化MariaDB完成,接下来测试登录,输入密码能正常登陆就完成了
      mysql -uroot -p
      
    • 登录数据库执行如下操作(看是否需要hive和oozie)

      mysql -uroot -p
      create database ambari character set utf8 ;  
      CREATE USER 'ambari'@'%'IDENTIFIED BY '123456';
      GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
      FLUSH PRIVILEGES;
      create database hive character set utf8 ;  
      CREATE USER 'hive'@'%'IDENTIFIED BY '123456';
      GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
      FLUSH PRIVILEGES;
      create database oozie character set utf8 ;  
      CREATE USER 'oozie'@'%'IDENTIFIED BY '123456';
      GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';
      FLUSH PRIVILEGES;
      
    • [master]安装mysql jdbc 驱动

      yum install mysql-connector-java -y
      
    • 拷贝本地RPM-GPG-KEY-mysql文件到集群,并且执行如下命令

      rpm --import RPM-GPG-KEY-mysql
      
  • 下载ambari及HDP的安装包

    http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.4.0/ambari-2.7.4.0-centos7.tar.gz
    http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.1.4.0/HDP-3.1.4.0-centos7-rpm.tar.gz
    http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
    
  • 制作本地yum源

    • 在httpd网站根目录,默认是即/var/www/html/,创建目录ambari,

    • 并且将下载的压缩包解压到/var/www/html/ambari目录

    • HDP-UTILS需要先建立一个HDP-UTILS文件夹并将HDP-UTILS-1.1.0.21-centos7.tar.gz放到HDP-UTILS目录下再解压

      yum install httpd -y
      firewall-cmd --add-service=http 
      firewall-cmd --permanent --add-service=http
      systemctl start httpd.service
      systemctl enable httpd.service
      yum install yum-utils createrepo -y
      cd /var/www/html/
      mkdir ambari
      cd /var/www/html/ambari/
      tar -zxvf /opt/install/ambari-2.7.4.0-centos7.tar.gz -C ./
      tar -zxvf /opt/install/HDP-3.1.4.0-centos7-rpm.tar.gz -C ./
      mkdir HDP-UTILS
      cd HDP-UTILS
      tar -zxvf /opt/install/HDP-UTILS-1.1.0.22-centos7.tar.gz -C ./
      
    • 验证httpd网站是否可用,用浏览器直接访问下面的地址,如果能看到目录列表就表示成功:

      http://172.28.128.16/ambari/
      
    • 配置ambari等本地源

      cd /etc/yum.repos.d/
      vim ambari.repo 
      #VERSION_NUMBER=2.7.4.0-118
      [ambari-2.7.4.0]
      name=ambari Version - ambari-2.7.4.0
      baseurl=http://172.28.128.16/ambari/ambari/centos7/2.7.4.0-118
      gpgcheck=1
      gpgkey=http://172.28.128.16/ambari/ambari/centos7/2.7.4.0-118/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
      enabled=1
      priority=1
      
      #VERSION_NUMBER=3.1.4.0-315
      [HDP-3.1.4.0]
      name=HDP Version - HDP-3.1.4.0
      baseurl=http://172.28.128.16/ambari/HDP/centos7/3.1.4.0-315
      gpgcheck=1
      gpgkey=http://172.28.128.16/ambari/HDP/centos7/3.1.4.0-315/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
      enabled=1
      priority=1
       
      [HDP-UTILS-1.1.0.22]
      name=HDP Utils Version - HDP-UTILS-1.1.0.22
      baseurl=http://172.28.128.16/ambari/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22
      gpgcheck=1
      gpgkey=http://172.28.128.16/ambari/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
      enabled=1
      priority=1
      
    • 将ambari.repo 拷贝到其他节点上

    • 在所有节点上执行如下命令

      yum clean all
      yum list update
      yum makecache
      yum repolist
      
  • 安装jdk

    mkdir /opt/java/
    cd /opt/install/
    tar -zxvf jdk-8u231-linux-x64.tar.gz -C /opt/java/
    vim /etc/profile
    export JAVA_HOME=/opt/java/jdk1.8.0_231
    export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
    PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
    source /etc/profile
    ==========================
    scp -r java node01:/opt
    scp -r java node02:/opt
    scp /etc/profile node01:/etc/
    scp /etc/profile node02:/etc/
    
  • 安装Ambari

    • [master]安装yum的相关工具

      yum install yum-utils -y
      yum repolist
      yum install createrepo -y
      
    • [master]安装ambari-sever

      yum install ambari-server -y
      
    • [master]执行如下命令

      ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar 
      
    • [master]安装ambari-server

      ambari-server setup
      
    • 出现如下安装提示

      Using python  /usr/bin/python
      Setup ambari-server
      Checking SELinux...
      SELinux status is 'disabled'
      Customize user account for ambari-server daemon [y/n] (n)?  回车
      Adjusting ambari-server permissions and ownership...
      Checking firewall status...
      Checking JDK...
      [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
      [2] Custom JDK
      ==============================================================================
      Enter choice (1): 2  选择2
      WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
      WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
      Path to JAVA_HOME:/opt/java/jdk1.8.0_231   输入java_home路径
      Validating JDK on Ambari Server...done.
      Check JDK version for Ambari Server...
      JDK version found: 8
      Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
      Checking GPL software agreement...
      GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
      Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? 回车
      Completing setup...
      Configuring database...
      Enter advanced database configuration [y/n] (n)? y 输入y 回车
      Configuring database...
      ==============================================================================
      Choose one of the following options:
      [1] - PostgreSQL (Embedded)
      [2] - Oracle
      [3] - MySQL / MariaDB
      [4] - PostgreSQL
      [5] - Microsoft SQL Server (Tech Preview)
      [6] - SQL Anywhere
      [7] - BDB
      ==============================================================================
      Enter choice (1):3  输入3 回车
      Hostname (localhost):   回车
      Port (3306):   回车
      Database name (ambari):  回车
      Username (ambari):   回车
      Enter Database Password (bigdata):  输入上面配置的密码123456 回车
      Re-enter password:  输入上面配置的密码123456 回车
      Configuring ambari database...
      Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? 回车
      Configuring remote database connection properties...
      WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
      Proceed with configuring remote database connection properties [y/n] (y)?回车
      
      
    • 将Ambari数据库脚本导入到数据库

      #用Ambari用户(上面设置的用户)登录mysql
      mysql -u ambari -p
      use ambari;
      source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;
      commit;
      
  • 所有节点安装ambari-agent

    yum install ambari-agent -y
    
  • [master]执行如下命令启动ambari-server

    ambari-server start
    
  • 每台机器做如下操作,禁用ssl检查

    # 禁用ssl检查
    vim /etc/python/cert-verification.cfg 
    [https]
    verify=disable
    # 修改/etc/ambari-agent/conf/ambari-agent.ini,在[security]添加下面内容
    force_https_protocol=PROTOCOL_TLSv1_2
    
  • 重启agent

    ambari-agent restart 
    
  • 执行如下命令,确认ambari服务起来了

    netstat -nlp|grep 8080
    
  • 浏览器访问如下地址: 用admin/admin登陆

    http://172.28.128.16:8080/#/login
    
  • 配置集群名称

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-PlCftIQe-1611884380909)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105140816089.png)]

  • 只留redhat7的yum源,并且配置本地HDP和HDP-UTILS地址

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BnGQIpRn-1611884380912)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105141105888.png)]

  • 拷贝master机器上id_rsa到本地,作为SSH Private Key 导入,填写集群主机名列表

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-0KVPoSbm-1611884380912)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105141405044.png)]

  • 等待各个节点初始化完成

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-GlB58I6G-1611884380914)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105141452154.png)]

  • 初始化完成界面如下

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-inSIPeYw-1611884380914)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105141715882.png)]

  • 选择服务安装,根据需要选择,这里选择了hdfs、YARN + MapReduce2、Tez、 Hive、ZooKeeper、Kafka、Spark2

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Dq01l1zy-1611884380915)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105142029682.png)]

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-zhr1VsDx-1611884380917)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105142041808.png)]

  • 服务分布图如下

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-te1nUX4e-1611884380917)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105142234035.png)]

  • 角色分布图如下

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-dkn1fhu9-1611884380918)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105142343741.png)]

  • 配置各个服务的账号密码

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-UwnwEfkI-1611884380918)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105142525532.png)]

  • 将需要填写的配置填写好

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-FQ3L5MP0-1611884380919)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105142614507.png)]

  • 根据需要填写修改各个服务的配置

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-UAVeT5Yj-1611884380919)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105142735100.png)]

  • 根据需要修改各个服务的用户

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-EX0PDlo8-1611884380920)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105143053035.png)]

  • 如果有哪些配置不对,会有提示

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-b7JwIVcG-1611884380920)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105143327235.png)]

  • 查看最终安装的视图,建议保存,后面可以参考这个查找

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-IALEGrpP-1611884380921)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105143604199.png)]

  • 点击deploy 等待安装完成,这一步要的时间会很久

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-E9wMIEB0-1611884380921)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105143730049.png)]

  • 安装结束后,如下

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-f3b26Dpk-1611884380922)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105145627935.png)]

  • 点击下一步进入ambari界面

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-vRYB3bOh-1611884380922)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105145659716.png)]

  • 手动启一下没有启动的服务

  • 发现range-admin会安装包错

    • 报错如下

      SQL state: 28000 java.sql.SQLException: Access denied for user 'root'@'node01.ambari3.0.com' (using password: YES) ErrorCode: 1045
      
    • 解决办法:

      use mysql
      select host,user,password,Create_routine_priv,Super_priv from user;
      grant all privileges on *.* to 'root'@'172.28.128.17' identified by '123456';
      grant all privileges on *.* to 'root'@'172.28.128.18' identified by '123456';
      grant all privileges on *.* to 'root'@'node01.ambari3.0.com' identified by '123456';
      grant all privileges on *.* to 'root'@'node02.ambari3.0.com' identified by '123456';
      grant all privileges on ranger.* to 'root'@'172.28.128.17' identified by '123456' with grant option;
      grant all privileges on ranger.* to 'root'@'172.28.128.18' identified by '123456' with grant option;
      grant all privileges on ranger.* to 'root'@'node01.ambari3.0.com' identified by '123456' with grant option;
      grant all privileges on ranger.* to 'root'@'node02.ambari3.0.com' identified by '123456' with grant option;
      FLUSH PRIVILEGES;
      
  • 最终ambari界面如下

    [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-dRdBc5B7-1611884380923)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105162231547.png)]

HDP3.X测试发现的问题

  • 用root用户访问hive,执行sql会抱没有权限

    • 报错如下

      Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied: user [root] does not have [USE] privilege on [Unknown resource!!] (state=42000,code=40000)
      
    • 经查发现是由ranger来统一控制权限的

    • 如何在ranger中添加root对hive的访问权限?

    • 在ranger中添加{USER}用户,如下图

      [外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-mNjLarwJ-1611884380923)(C:\Users\15996\AppData\Roaming\Typora\typora-user-images\image-20210105171530061.png)]

    • 再次用root用户访问就OK了

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值