Ambari安装 (Ambari-2.7.4.0 +HDP-3.1.4.0+HDP-UTILS-1.1.0.22 基于CentOS7)

1. Ambari简介

Apache Ambari是一种基于Web的工具，支持Apache Hadoop集群的供应、管理和监控。Ambari已支持大多数Hadoop组件，包括HDFS、MapReduce、Hive、Pig、 Hbase、Zookeeper、Sqoop和Hcatalog等。
Apache Ambari支持HDFS、MapReduce、Hive、Pig、Hbase、Zookeepr、Sqoop和Hcatalog等的集中管理。也是5个顶级hadoop管理工具之一。

2.Ambari下载

官方网站 : https://docs.cloudera.com/HDPDocuments/index.html

由于Cloudera公司在官网宣布:

从2021年1月31日开始，所有Cloudera软件都需要有效的订阅进行访问。

HDP-3.1.4.0是最后一个可以下载到的开源版本
安装包下载链接 :
Ambari https://archive.cloudera.com/p/ambari/2.x/2.7.4.14/centos7/ambari-2.7.4.14-1-centos7.tar.gz
HDP https://archive.cloudera.com/p/HDP/3.x/3.1.4.0/centos7/HDP-3.1.4.0-centos7-rpm.tar.gz
HDP-UTILS https://archive.cloudera.com/p/HDP-UTILS/1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz

推荐使用迅雷进行下载(直接复制链接)，使用浏览器直接下载会遇到403错误(或需要输入订阅的用户名和密码)

3.安装环境

虚拟机	网络(根据虚拟机配置)	内存	磁盘
master	192.168.182.134	4G	50G
node	192.168.182.135	2G	50G

4.环境配置

4.1修改主机名和hosts

# 分别修改主机名(master node)
hostnamectl set-hostname master
hostnamectl set-hostname node
# 添加主机名到IP的映射(master node)
vi /etc/hosts
# 根据主机IP进行添加(master node)
192.168.182.134 master
192.168.182.135 node

4.2关闭防火墙和关闭SELINUX

# 关闭防火墙(master node)
systemctl stop firewalld && systemctl disable firewalld
# 关闭SELINUX(master node)
setenforce 0
vi /etc/selinux/config
# SELINUX=enforcing修改为
SELINUX=disabled

4.3配置免密登录

# master
ssh-keygen
ssh-copy-id master
ssh-copy-id node
# node
ssh-keygen
ssh-copy-id master
ssh-copy-id node

4.4安装时间同步服务ntp

# master
yum -y install ntp
vi /etc/ntp.conf
--------------------------------------------------
将下列内容前加'#'进行注释
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
--------------------------------------------------
# 添加
server 127.127.1.0
fudge 127.127.1.0 stratum 10
# 启动服务
systemctl start ntpd && systemctl enable ntpd
# node
yum -y install ntpdate
ntpdate master

4.5安装http服务(master)

# 安装服务
yum -y install httpd
# 启动服务
systemctl start httpd && systemctl enable httpd

4.6配置yum源(master)

通过Xftp将Ambari、HDP、HDP-UTILS文件上传到master的根目录即可

# 创建文件夹
mkdir /var/www/html/{ambari,hdp,hdp-utils}
# 解压文件
tar -zxvf ambari-2.7.4.0-centos7.tar.gz -C /var/www/html/ambari/
tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz -C /var/www/html/hdp/
tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp-utils/
# 配置ambari的yum源
vi /etc/yum.repos.d/ambari.repo
# 添加以下内容
[ambari]
name=ambari
baseurl=http://192.168.182.134/ambari/ambari/centos7/2.7.4.0-118/
gpgcheck=0
# 配置hdp和hdp-utils的yum源
vi /etc/yum.repos.d/hdp.repo
# 添加以下内容
[HDP-3.1]
name=HDP-3.1
baseurl=http://192.168.182.134/hdp/HDP/centos7/3.1.4.0-315/
gpgcheck=0
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS-1.1.0.22
baseurl=http://192.168.182.134/hdp-utils/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=0
# 验证
yum repolist

将yum文件使用scp发送到node节点

scp /etc/yum.repos.d/ambari.repo node:/etc/yum.repos.d/
scp /etc/yum.repos.d/hdp.repo node:/etc/yum.repos.d/

4.7安装以及配置JDK(master)

将jdk安装包上传到master

# 创建文件夹
mkdir /usr/local/java
# 解压文件
tar -zxvf jdk-8u281-linux-x64.tar.gz -C /usr/local/java/
# 配置环境变量
vi /etc/profile
# 添加
export JAVA_HOME=/usr/local/java/jdk1.8.0_281/
export PATH=$PATH:$JAVA_HOME/bin
# 刷新环境变量
source /etc/profile
# 验证
java -version
# 将jdk和环境变量发送到node
scp -r /usr/local/java/  node:/usr/local/
scp /etc/profile node:/etc/
# node刷新环境变量以及验证
source /etc/profile
java -version

5.安装以及配置数据库(master)

数据库使用MariaDB(MySQL的一个分支)，安装到master上

# master
yum -y install mariadb-server
# 启动
systemctl start mariadb && systemctl enable mariadb
# 初始化
mysql_secure_installation

<!--没有输入[Y/n]的，直接回车-->
[root@master ~]# mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we'll need the current
password for the root user.  If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none): 
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.

Set root password? [Y/n] 
New password:  <!--设置密码，例:1234-->
Re-enter new password: 
Password updated successfully!
Reloading privilege tables..
 ... Success!


By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] 
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] n
 ... skipping.

By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] 
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] 
 ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!

6.安装和配置ambari-server(master)

6.1安装

# 安装
yum -y install ambari-server

6.2配置数据库

# 登录数据库
mysql -uroot -p1234
# 创建amabri数据库
create database ambari;
#授权
grant all on ambari.* to ambari@'%' identified by '1234';
grant all on ambari.* to ambari@localhost identified by '1234';
# 使用ambari-server提供的sql脚本创建相关的表
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

6.3配置JDBC

通过Xftp上传至 /usr/share/java/ 中

# 创建文件夹
mkdir -p /usr/share/java
cd /usr/share/java
# jdbc重命名
mv mysql-connector-java-8.0.23.jar /usr/share/java/mysql-connector-java.jar

6.4配置ambari-server

# 配置
ambari-server setup

<!--没有输入[Y/n]的，直接回车-->
[root@master ~]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2 <!--此处一定要选择2，否则就会下载JDK-->
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/local/java/jdk1.8.0_281/
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? 
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3 <!--选择安装的MariaDB-->
Hostname (localhost): master <!--这里是主机名-->
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (bigdata): <!--输入前面设置的数据库密码，例:1234-->
Re-enter password: 
Configuring ambari database...
Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? 
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? 
Extracting system views...
ambari-admin-2.7.4.0.118.jar
....
Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

6.5启动ambari-server

# 启动
ambari-server start

[root@master ~]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start............................................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.
<!--最后显示successfully表示成功启动-->

7.安装和配置ambari-agent(master node)

# 安装ambari-agent
yum -y install ambari-agent
# 配置
vi /etc/ambari-agent/conf/ambari-agent.ini
# 修改
[server]
hostname=master
#启动
ambari-agent start

8.HDP部署

8.1登录

浏览器输入 http://192.168.182.134:8080
默认用户名和密码为 admin

8.2配置集群

接下来全部 NEXT，直到显示如图

等待安装成功

如果结果显示有黄色警告，配置已经成功了，只是由于某个服务没有正常启动导致的。

安装成功