Ambari 2.7.3_HDP 3.1.0_Centos 7.4.1708 安装

最新推荐文章于 2022-09-28 15:38:15 发布

CesarChoy

最新推荐文章于 2022-09-28 15:38:15 发布

阅读量1k

点赞数

分类专栏： HDP

本文链接：https://blog.csdn.net/weixin_42687074/article/details/106771995

版权

HDP 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

前文：

19年自己搭建的HDP，现在把我搭建的过程分享给大家。后面图片本来是高清无码，但是由于图片太多，所以没有一张一张上传所以不清晰了，请谅解。

第一章环境准备

注意：以下操作三台机器都需要进行

1.1虚拟机准备

vim /etc/hosts 配置三台主机地址

192.168.1.112 hadoop112    

192.168.1.113 hadoop113    

192.168.1.114 hadoop114

1.2防火墙装备

1.2.1临时关闭防火墙命令

systemctl stop firewalld

1.2.2永久关闭防火墙命令

systemctl disable firewalld

1.2.3关闭SELINUX

vim /etc/sysconfig/selinux
将SELINUX=enforcing改为SELINUX=disabled

1.3 安装JDK

1.3.1创建module和software文件夹用于存放软件

mkdir /opt/module

Mkdir /opt/software

1.3.2 上传JDK安装包

用SecureCRT将jdk-8u144-linux-x64.tar.gz导入到hadoop102的/opt/software目录下

tar -zxvf /opt/software/jdk-8u144-linux-x64.tar.gz -C /opt/module/

1.3.3 配置JDK路径

在profile文件末尾添加JDK路径

 vi /etc/profile

#JAVA_HOME

export JAVA_HOME=/opt/module/jdk1.8.0_144

export PATH=$PATH:$JAVA_HOME/bin

让修改后的文件生效并测试JDK是否安装成功

source /etc/profile

java -version

1.4 SSH免密登录

1.4.1生成公钥和私钥

ssh-keygen -t rsa

1.4.2 将公钥拷贝到要免密登录的目标机器上

ssh-copy-id hadoop112

ssh-copy-id hadoop113

ssh-copy-id hadoop114

1.5 修改yum源为阿里云镜像

1.5.1备份原生yum源

mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.bk

1.5.2下载阿里云的资源文件并命名

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo

1.5.3把服务器的包下载到本地缓存

yum makecache

1.6安装ntp同步时间服务器

1.6.1安装ntp

yum install -y ntp

1.6.2查看ntp服务状态

systemctl status ntpd

1.6.3临时启动ntp服务

systemctl start ntpd

1.6.4永久启动ntp服务

systemctl enable ntpd

1.7 关闭Linux的THP服务

1.7.1编辑rc.local启动文件

vim /etc/rc.local



添加：

if test -f /sys/kernel/mm/transparent_hugepage/defrag; then

  echo never > /sys/kernel/mm/transparent_hugepage/defrag

fi

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then

  echo never > /sys/kernel/mm/transparent_hugepage/enabled

fi

exit 0

1.7.2给rc.local添加可执行权限

chmod +x /etc/rc.d/rc.local

1.7.3验证

cat /sys/kernel/mm/transparent_hugepage/enabled

always madvise [never]

1.8 配置UMASK

1.8.1设定用户所创建目录的初始权限

umask 0022

1.9 禁止离线更新

1.9.1 禁止离线更新

vim /etc/yum/pluginconf.d/fastestmirror.conf

修改：enabled=0

安装Ambari集群

注意：以下操作只在主节点上进行

2.1 安装配置HTTPD 服务

2.1.1安装

yum install httpd -y

2.1.2查看状态

systemctl status httpd

2.1.3临时启动

systemctl start httpd

2.1.4永久启动

systemctl enable httpd

2.2 安装本地源制作相关工具

2.2.1安装工具

yum install yum-utils createrepo yum-plugin-priorities -y

2.2.2 开启工具

vim /etc/yum/pluginconf.d/priorities.conf

添加gpgcheck=0

2.2.3将下载的4个tar包解压

ambari安装包解压


tar -zxvf /opt/software/ambari-2.7.3.0-centos7.tar.gz -C /var/www/html/

HDP及其工具安装包解压

mkdir /var/www/html/hdp

tar -zxvf /opt/software/HDP-3.1.0.0-centos7-rpm.tar.gz -C /var/www/html/hdp

tar -zxvf /opt/software/HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp

tar -xzvf /opt/software/HDP-GPL-3.1.0.0-centos7-gpl.tar.gz -C /var/www/html/hdp

2.2.4 创建本地源

createrepo  /var/www/html/

2.2.5将Ambari存储库文件下载到安装主机上的目录中

wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.3.0/ambari.repo

2.2.6 修改配置文件，配置为本地源

2.2.6.1修改ambari.repo，配置为本地源

vim /etc/yum.repos.d/ambari.repo



#VERSION_NUMBER=2.7.3.0-139

[ambari-2.7.3.0]

#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json

name=ambari Version - ambari-2.7.3.0

baseurl=http://hadoop112/ambari/centos7/2.7.3.0-139/

gpgcheck=0

gpgkey=http://hadoop112/ambari/centos7/2.7.3.0-139/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

2.2.6.2修改hdp.repo，配置为本地源

vim /var/www/html/hdp/HDP/centos7/3.1.0.0-78/hdp.repo





#VERSION_NUMBER=3.1.0.0-78

[HDP-3.1.0.0]

name=HDP Version - HDP-3.1.0.0

baseurl=http://hadoop112/hdp/HDP/centos7/3.1.0.0-78/

gpgcheck=0

gpgkey=http://hadoop112/hdp/HDP/centos7/3.1.0.0-78/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1





[HDP-UTILS-1.1.0.22]

name=HDP-UTILS Version - HDP-UTILS-1.1.0.22

baseurl=http://hadoop112/hdp/HDP-UTILS/centos7/1.1.0.22/

gpgcheck=0

gpgkey=http://hadoop112/hdp/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1



添加

#VERSION_NUMBER=3.1.0.0-78

[HDP-GPL-3.1.0.0]

name=HDP-GPL Version - HDP-GPL-3.1.0.0

baseurl=http://hadoop112/hdp/HDP-GPL/centos7/3.1.0.0-78/

gpgcheck=0

gpgkey=http://hadoop112/hdp/HDP-GPL/centos7/3.1.0.0-78/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

2.2.6.3修改hdp-util.repo，配置为本地源

vim /var/www/html/hdp/HDP-UTILS/centos7/1.1.0.22/hdp-utils.repo



[HDP-UTILS-1.1.0.22]

name=Hortonworks Data Platform Utils Version - HDP-UTILS-1.1.0.22

baseurl=http://hadoop112/hdp/HDP-UTILS/centos7/1.1.0.22/

gpgcheck=0

enabled=1

priority=1

2.2.6.4 修改hdp.gpl.repo，配置为本地源

vim /var/www/html/hdp/HDP-GPL/centos7/3.1.0.0-78/hdp.gpl.repo



#VERSION_NUMBER=3.1.0.0-78

[HDP-GPL-3.1.0.0]

name=HDP-GPL Version - HDP-GPL-3.1.0.0

baseurl=http://hadoop112/hdp/HDP-GPL/centos7/3.1.0.0-78/

gpgcheck=0

gpgkey=http://hadoop112/hdp/HDP-GPL/centos7/3.1.0.0-78/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

enabled=1

priority=1

2.3安装MySQL

前提：先卸载自带数据库

yum remove mariadb*

2.3.1安装MySQL服务端

rpm -ivh MySQL-server-5.6.24-1.el6.x86_64.rpm

查看产生的随机密码

cat /root/.mysql_secret

XrBwJEENp2yhVe4O

启动MySQL

service mysql start

2.3.2安装MySQL客户端

安装MySQL客户端

rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm

2.3.3进入客户端修改配置

mysql -uroot -pXrBwJEENp2yhVe4O

修改密码

mysql>SET PASSWORD=PASSWORD('000000');

mysql>use mysql;



查询user表

mysql>select User, Host, Password from user;



修改user表，把Host表内容修改为%

mysql>update user set host='%' where host='localhost';



删除root用户的其他host

mysql>

delete from user where Host='hadoop112';

delete from user where Host='127.0.0.1';

delete from user where Host='::1';



刷新

mysql>flush privileges;



退出

mysql>quit;

2.4安装Ambari

2.4.1安装ambari-server

yum install ambari-server

2.4.2 拷贝mysql驱动

mkdir /usr/share/java

cp /opt/software/mysql-libs/mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar /usr/share/java/mysql-connector-java.jar

2.4.3 修改ambari.properties文件

vim  /etc/ambari-server/conf/ambari.properties

添加

server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

2.4.4 在MySQL中创建数据库

mysql -u root -p000000



创建ambari库

mysql >create database ambari;



使用Ambari自带脚本创建表

mysql > use ambari;

mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql



赋予用户root权限：

mysql> grant all privileges on *.* to 'root'@'%' identified by '000000';



刷新

mysql> flush privileges;

2.4.5 配置Ambari

 ambari-server setup

Using python /usr/bin/python

Setup ambari-server

Checking SELinux...

SELinux status is 'disabled'

2.5.1提示是否自定义设置

Customize user account for ambari-server daemon [y/n] (n)? y

2.5.2 ambari-server 账号(默认选择root用户)

Enter user account for ambari-server daemon (root):

Adjusting ambari-server permissions and ownership...

Checking firewall status...

Checking JDK...

[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8

[2] Custom JDK

==============================================================================

2.5.4自定义JDK,则需要设置JAVA_HOME

Path to JAVA_HOME: /opt/module/jdk1.8.0_144/

Validating JDK on Ambari Server...done.

Check JDK version for Ambari Server...

JDK version found: 8

Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.

Checking GPL software agreement...

GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html

2.5.3设置JDK来源为自定义

Enter choice (1): 2

WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.

WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.

2.5.5允许ambari服务器下载并安装gpl许可的lzo包

Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y

Completing setup...

Configuring database...

2.5.6是否数据库配置

Enter advanced database configuration [y/n] (n)? y

Configuring database...

==============================================================================

Choose one of the following options:

[1] - PostgreSQL (Embedded)

[2] - Oracle

[3] - MySQL / MariaDB

[4] - PostgreSQL

[5] - Microsoft SQL Server (Tech Preview)

[6] - SQL Anywhere

[7] - BDB

==============================================================================

Enter choice (1): 3

2.5.7配置数据库地址、端口、数据库名、用户名、密码

Hostname (localhost): hadoop112

Port (3306):

Database name (ambari):

Username (ambari): root

Enter Database Password (bigdata):

Re-enter password:

Configuring ambari database...

Configuring remote database connection properties...

WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql