【Ambari】Ambari搭建Hadoop集群

配置说明

ip	host	os	配置
172.**.***.86	master	CentOS 7.4 64位	2vCpu 4GiB
172.**.***.42	slave1	CentOS 7.4 64位	1vCpu 2GiB
172.**.***.73	slave2	CentOS 7.4 64位	1vCpu 2GiB
172.**.***.240	slave3	CentOS 7.4 64位	1vCpu 2GiB
172.**.***.214	slave4	CentOS 7.4 64位	2vCpu 4GiB

• [注]
  • 需要注意一下服务器的架构,x86和arm有极大的区别,arm搭建大数据环境的坑和包会比较多,混搭很麻烦,之前尝试用2台鲲鹏(arm)和1台x86的搭过,中途坑太多了最后放弃了,这次所有服务器都是x86

基础环境

修改主机名和hosts文件

• 每台主机

• 修改主机名

# 使用hostnamectl命令修改主机名，执行该命令后立即生效，只不过需要重启Xshell连接
# 以其中一台为例，代码如下
hostnamectl set-hostname master

• 修改hosts文件

# 添加机器ip与主机名映射
vim /etc/hosts
121.36.74.82 master
121.36.57.173 slave1
121.36.111.110 slave2

关闭防火墙和selinux

• 云服务器防火墙和selinux默认是关闭的

• 防火墙

# 查看防火墙状态
systemctl status firewalld
# 查看开机是否启动防火墙服务
systemctl is-enabled firewalld
# 关闭防火墙
systemctl stop firewalld
systemctl disable firewalld
# 再次查看防火墙状态和开机防火墙是否启动
systemctl status firewalld
systemctl is-enabled firewalld

• 禁用selinux

# 永久性关闭selinux（重启服务器生效）
sed -i 's/SELINUX=enforcing/SELINUX =disabled/' /etc/selinux/config
# 临时关闭selinux（立即生效，重启服务器失效）
setenforce 0
# 查看selinux状态
getenforce
# disabled为永久关闭，permissive为临时关闭，enforcing为开启

jdk下载配置

# 创建目录
mkdir /usr/java 

# 解压压缩包 
tar zxvf /root/software/jdk-8u241-linux-x64.tar.gz -C /usr/java/

# 配置jdk环境变量
# 编辑/etc/profile,文末插入以下内容：
vim /etc/profile
# set java
export JAVA_HOME=/usr/java/jdk1.8.0_241
export PATH=${JAVA_HOME}/bin:$PATH

#使环境变量生效
source /etc/profile #这个是全局变量，永久的

#安装验证
java -version

yum 离线安装 mysql

• 检查本地rpm包

# 检查本地资源库中是否有mysql的rpm包
rpm -qa | grep mysql
# 删除相关rpm包
rpm -ev <rpm包名> --nodeps
# 检查本地资源库中是否有mariadb的rpm包
rpm -qa|grep mariadb
# 删除相关rpm包
rpm -ev <rpm包名> --nodeps

• 安装mysql
  参考：https://blog.csdn.net/wohiusdashi/article/details/89358071

# 依次安装
rpm -ivh mysql-community-common
rpm -ivh mysql-community-libs
rpm -ivh mysql-community-client
rpm -ivh mysql-community-server
rpm -ivh mysql-community-libs-compat

# 后面加上--nodeps可以避免依赖问题，但缺少的依赖需要安装

# 启动mysql
service mysqld start
# 查看mysql
systemctl status mysqld 
# 查看随机密码
grep 'temporary password' /var/log/mysqld.log
# 登陆mysql
mysql -u root -p
Enter password: 

# 为了可以设置简单密码
set global validate_password_policy=LOW;
set global validate_password_length=4;
# 立即修改密码为root
SET PASSWORD FOR 'root'@'localhost' = PASSWORD('root');
# 查看密码策略
SHOW VARIABLES LIKE 'validate_password%';

• 允许用户远程访问权限（这里是说明,直接执行下面的就可以了）

# 允许用户root远程登录：
grant all on *.* to 'root'@'%' identified by 'root' with grant option;
flush privileges;
# 部分服务器此时仍然连接不上，可能是安全组没设置的原因

• 修改防火墙配置允许3306端口访问

# 前面直接关了防火墙,所以这一步不用
# 编辑 
vim /etc/sysconfig/iptables
增加 -A INPUT -m state --state NEW -m tcp -p tcp --dport 3306 -j ACCEPT
# 重启 防火墙  
service iptables restart

免密操作：使 ambari-server 节点免密操作 ambari-agent 节点

• 各个主机均执行操作

## 生成密钥对
ssh-keygen -t rsa   ## 一路回车即可
## 进入.ssh目录，如果目录不存在则创建
cd ~/.ssh
## 将公钥导入至authorized_keys
cat id_rsa.pub >> authorized_keys
## 修改文件权限
chmod 700 ~/.ssh
chmod 600 authorized_keys

• 在master操作

# 配置主从免密登陆
# 这里只配置了单向免密（master->slave*） 
[root@master ~]# cat ~/.ssh/id_rsa.pub | ssh root@slave1 'cat - >> ~/.ssh/authorized_keys'
[root@master ~]# cat ~/.ssh/id_rsa.pub | ssh root@slave2 'cat - >> ~/.ssh/authorized_keys'
# 验证主机点是否可以免密登陆从节点，执行exit命令退出即可
ssh slave1 
ssh slave2 

hadoop 各节点主机保持始终同步

# 安装ntp服务
yum install ntp

主节点

vim /etc/ntpd.conf
# 将下列字段注释：
# server 0.centos.pool.ntp.org iburst
# server 1.centos.pool.ntp.org iburst
# server 2.centos.pool.ntp.org iburst
# server 3.centos.pool.ntp.org iburst

# 添加：
server 127.127.1.0   # 表示NTP主服务器是与自身的系统时钟同步
fudge 127.127.1.0 stratum 10   # 指定阶层编号为10，降低其优先度

Service ntpd start
Service ntpd status

[注]阿里云ntp没有ntpd.conf文件，而是ntp.conf，上面需要添加
的内容是默认的

从节点

# 将下列字段注释：
# server 0.centos.pool.ntp.org iburst
# server 1.centos.pool.ntp.org iburst
# server 2.centos.pool.ntp.org iburst
# server 3.centos.pool.ntp.org iburst

# 添加：
server 127.127.1.0   # 表示NTP主服务器是与自身的系统时钟同步
fudge 127.127.1.0 stratum 10   # 指定阶层编号为10，降低其优先度
server master   # 配置时间服务器为本地的时间服务器
restrict master nomodify notrap noquery   # 允许上层时间服务器主动修改本机时间

service ntpd start
service ntpd status

# 硬件时间
hwclock --systohc

# 设置开机自启动
vim /etc/rc.d/rc.local
添加 service ntpd restart

chmod +x /etc/rc.d/rc.local

安装配置 ambari-server

搭建 yum 本地源

# 安装httpd
yum install httpd
systemctl enable httpd.service
systemctl start httpd.service

# 安装wget
yum install wget

• 下载ambari,hdp,hdp-gpl,hdp-util
  https://docs.cloudera.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/ambari_repositories.html

# 将ambari,hdp,hdp-gpl,hdp-util移动到/var/www/html
[root@master software]# mv ambari HDP HDP-GPL HDP-UTILS /var/www/html/ 
[root@master software]# cd /var/www/html/
[root@master html]# ls
ambari  HDP  HDP-GPL  HDP-UTILS

# 创建ambari.repo
cd /etc/yum.repos.d
[root@master yum.repos.d]# vim ambari.repo
# 写入
[ambari]
name=ambari
baseurl=http://39.99.193.128/ambari/centos7/2.7.3.0-139/
enabled=1
gpgcheck=0

# 将ambari拷贝到其他主机上
scp -r ambari.repo root@slave1:/etc/yum.repos.d/

mysql 用户设置

# 新增ambari用户并增加权限
mysql -uroot -proot
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari';
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambari';
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';
CREATE USER 'ambari'@'master' IDENTIFIED BY 'ambari';
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'master';  
# 本地主机名
FLUSH PRIVILEGES;

# 登陆ambari用户
mysql -uambari -pambari
# 新建ambari数据库
CREATE DATABASE ambari;
exit

使用 yum 安装 ambari-server

# 安装ambari-server
yum install ambari-server

• 将mysql-connector-java.jar拷贝到/usr/share/java目录下

执行 ambari-server 配置命令，用户交互

[root@master ~]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_241
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): master
Port (3306): 
Database name (ambari): ambari
Username (ambari): ambari
Enter Database Password (bigdata): 
Re-enter password: 
Configuring ambari database...
Enter full path to custom jdbc driver: /usr/share/java/mysql-connector-java.jar
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
....ambari-admin-2.7.3.0.139.jar

Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

初始化 ambari 数据库

# 这个sql建表语句是ambari自带的
[root@master ~]# mysql -uambari -pambari
mysql> use ambari;
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

启动 ambari-server

[root@master ~]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.......................................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.

登陆浏览器访问http://39.99.193.128:8080/，利用界面部署集群
默认登陆账号/密码为：admin/admin。

如果不能访问的话，查看一下服务器的安全组规则

部署 hadoop 集群

移除 Smartsense 服务

# 每一台都要移除
[root@master ~]# cd /etc/yum.repos.d/
[root@master yum.repos.d]# find / -name SMARTSENSE | xargs
/var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SMARTSENSE /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/SMARTSENSE /var/lib/ambari-server/resources/stacks/HDP/3.0/services/SMARTSENSE /var/lib/ambari-server/resources/stacks/HDP/2.1/services/SMARTSENSE
[root@master yum.repos.d]# rm -rf /var/lib/ambari-agent/cache/stacks/HDP/3.0/services/SMARTSENSE /var/lib/ambari-agent/cache/stacks/HDP/2.1/services/SMARTSENSE /var/lib/ambari-server/resources/stacks/HDP/3.0/services/SMARTSENSE /var/lib/ambari-server/resources/stacks/HDP/2.1/services/SMARTSENSE
[root@master yum.repos.d]# 
[root@master yum.repos.d]# find / -name SMARTSENSE | xargs

# 移除完之后master
ambari-server restart

设置集群名称，选择 hdp 版本，并设置 yum 仓库地址

• 设置集群名称
  

• 选择hdp版本,设置yum仓库地址

[注]yum仓库是我们之前上传到 /var/www/html发布之后的仓库地址

确认安装主机列表，并填写 ambari-server 节点的用户私钥

查看密钥
[root@master yum.repos.d]# cat ~/.ssh/id_rsa

注册并检查主机：每台节点上都会安装 ambari-agent ,并检查各节点环境

第一次的时候这里出现了failed,猜测可能是安全组没有配置的原因，去每台服务器上配置了一下安全组，可以参考最后的链接Ambari服务端口

选择要安装的 hadoop 服务， 分配master,slave,clients

• 选择服务

• 分配

服务安装、启动并测试

[注]：由于服务器内存比较小，通过逐渐增加服务并监控内存的方式来安装，相关命令如下

# 查看内存信息
free -h
# total 总计物理内存大小
# used 已使用内存大小
# free 可用内存大小
# shared 多个进程共享的内存大小
# buff/cache  磁盘缓存大小
# available

# 手动释放内存（buff/cache）
echo 3 > /proc/sys/vm/drop_caches

安装完成之后

链接

ambari版本与支持软件版本

• https://supportmatrix.hortonworks.com/

mysql下载链接

• https://dev.mysql.com/downloads/mysql/

ambari下载链接

• ambari,hdp,hdp-gpl,hdp-util
  https://docs.cloudera.com/HDPDocuments/Ambari-2.7.3.0/bk_ambari-installation/content/ambari_repositories.html

参考资料

1. create17, Ambari2.7安装配置, https://841809077.github.io/2018/11/03/Ambari%20v2.7.1%E5%AE%89%E8%A3%85.html
2. CSDN, 不甘于平凡的溃败, CentOS7下安装mysql5.7, https://blog.csdn.net/wohiusdashi/article/details/89358071
3. 简书, 粮忆雨, 【ambari-部署】记录一些Ambari 服务端口, https://www.jianshu.com/p/3a95e4fb311a

同时感谢博主create17在我混搭的情况下遇到坑时提供的帮助