ambari 2.7.4部署集群步骤

最新推荐文章于 2022-03-19 13:19:08 发布

咖啡F

最新推荐文章于 2022-03-19 13:19:08 发布

阅读量1.6k

点赞数 1

分类专栏： hadoop 文章标签： hdfs hadoop big data ambari 大数据

本文链接：https://blog.csdn.net/weixin_43697701/article/details/121643649

版权

hadoop 专栏收录该内容

37 篇文章 2 订阅

订阅专栏

1、关闭防火墙

需要安装包私信

2、关闭selinux

3、安装jdk

mkdir /usr/local/java
tar -zxvf jdk-8u212-linux-x64.tar.gz  -C /usr/local/java/
vim /etc/profile
	export JAVA_HOME=/usr/local/java/jdk1.8.0_212
	export JRE_HOME=/usr/local/java/jdk1.8.0_212/jre
	export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
	export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
 source /etc/profile
 java -version

在这里插入图片描述

4、配置节点间免密（主节点和其他节点就可以了，和自己也要做）

	ssh-keygen -t rsa
ssh-copy-id 192.168.85.101
ssh-copy-id 192.168.85.102
ssh-copy-id 192.168.85.103

5、配置/etc/hosts文件

6、配置yum源，可以配置本地源或者阿里的

7、安装ntp服务

8、关闭THP服务–会影响性能

vim /etc/rc.d/rc.local
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
    echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
    echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

在这里插入图片描述

chmod +x /etc/rc.d/rc.local

9、配置打开文件最大限制

vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

##########前面的步骤集群所有机器都需要操作

10、主节点开启httpd服务

11、配置本地源

yum install yum-utils createrep -y
rpm -ivh yum-plugin-priorities-1.1.31-54.el7_8.noarch.rpm	这个包默认是没有需要下载

vim /etc/yum/pluginconf.d/priorities.conf 
gpgcheck=0

解压
mkdir /var/www/html/ambari
tar -zxvf ambari-2.7.4.0-centos7.tar.gz -C /var/www/html/ambari
tar -zxvf HDP-3.1.4.0-centos7-rpm.tar.gz -C /var/www/html/ambari
tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/ambari

在这里插入图片描述

12、配置ambari源为本地

/var/www/html/ambari
vim ambari/centos7/2.7.4.0-118/ambari.repo

在这里插入图片描述

cp ambari/centos7/2.7.4.0-118/ambari.repo /etc/yum.repos.d/
 vim HDP/centos7/3.1.4.0-315/hdp.repo

在这里插入图片描述

cp HDP/centos7/3.1.4.0-315/hdp.repo /etc/yum.repos.d/

此处可以先访问yum源，如果每层都能访问那就可以跳过下面删除步骤，如果下图错误

Error:[object Object]
在这里插入图片描述

则需要/var/www/html/ambari/ambari/centos7/2.7.4.0-118目录下
rm -fr ambari.repo
rm -fr index.html 
在 HDP/centos7/3.1.4.0-315/hdp.repo 目录下
也删除对应的两个文件

13、制作本地源

cd /var/www/html/ambari
createrepo ./

 yum clean all
 yum repolist
 yum makecache

14、yum配置每台主机都发送

scp -rp /etc/yum.repos.d/ambari.repo node102:/etc/yum.repos.d/
scp -rp /etc/yum.repos.d/ambari.repo node103:/etc/yum.repos.d/
scp -rp /etc/yum.repos.d/hdp.repo node102:/etc/yum.repos.d/
scp -rp /etc/yum.repos.d/hdp.repo node103:/etc/yum.repos.d/

15、默认使用postgresql，换成mysql

1）卸载默认mysql

在这里插入图片描述

2）卸载mariadb
[root@node101 mysql-libs]# rpm -qa | grep mariadb
mariadb-libs-5.5.68-1.el7.x86_64
[root@node101 mysql-libs]# rpm -e --nodeps  mariadb-libs-5.5.68-1.el7.x86_64

3）安装mysql---此处安装包需要下载
rpm -ivh  MySQL-server-5.6.24-1.el6.x86_64.rpm 
[root@node101 mysql-libs]# cat /root/.mysql_secret ##	查看这个默认密码 后面会用到
# The random password set for the root user at Sun Nov 28 13:02:02 2021 (local time): qsey7J411sjgbgck
4）修改MySQL密码，授权
/sbin/chkconfig mysql on
service mysql start
rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm
mysql -uroot -pqsey7J411sjgbgck		##进库用的密码就是前面默认密码
SET PASSWORD=PASSWORD('Thinker@123');	##改密码

use mysql;
update user set host='%' where host='localhost';
grant all privileges on *.* to 'root'@'%' identified by 'Thinker@123';
flush privileges;

16、安装ambari

1）基本安装
yum install ambari-server -y
tar -zxvf mysql-connector-java-5.1.27.tar.gz 这个包在解压的mysq-lib包中
cp -rp mysql-connector-java-5.1.27-bin.jar /usr/share/java
cd /usr/share/java
mv mysql-connector-java-5.1.27-bin.jar  mysql-connector-java.jar
/usr/ambari/mysql-libs/mysql-connector-java-5.1.27
 cp -rp mysql-connector-java-5.1.27-bin.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar

vim  /etc/ambari-server/conf/ambari.properties 
最后行加
server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar


2）创建数据库
create database ambari;
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;	##建表
grant all privileges on *.* to 'root'@'%' identified by 'Thinker@123';
flush privileges;

3）配置初始化
[root@node101 java]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root): ambari
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/local/java/jdk1.8.0_212		##这里可以 echo $JAVA_HOME
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? n
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y 
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (bigdata): 	##此处密码我是ambari
Re-enter password: 
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.4.0.118.jar
....
Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it...
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

17、启动ambari

ambari-server start
启动时报错

在这里插入图片描述

解决： 
mysql -uroot -pThinker@123
use mysql;

在这里插入图片描述

 flush privileges;
use ambari;					##如果安装hive或者ozzoe需要mysql也是同样语句
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';  
CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambariambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';  
CREATE USER 'ambari'@'master' IDENTIFIED BY 'ambariambari';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'master';  
FLUSH PRIVILEGES;  
重置 ambari-server setup，再次启动，就欧克

18、页面登陆，账号密码都是admin

在这里插入图片描述

19、部署集群

在这里插入图片描述

这里的id_rsa上传主管理节点的就可以

在这里插入图片描述

选服务

在这里插入图片描述

选择部署节点

在这里插入图片描述

后续就是集群部署了


如果要装hive等，就需要
show tables;  
use mysql;  
select Host User Password from user where user='ambari';  
CREATE DATABASE hive;  
use hive;  
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';  
CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost';  
CREATE USER 'hive'@'master' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'master';  
FLUSH PRIVILEGES;  


CREATE DATABASE oozie;  
use oozie;  
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';  
CREATE USER 'oozie'@'localhost' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost';  
CREATE USER 'oozie'@'master' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'master';  
FLUSH PRIVILEGES;

问题1：到这里时候一直报错
这里一直报错，后来加了个包
在这里插入图片描述

装了一个包
 rpm -ivh libtirpc-devel-0.2.4-0.16.el7.x86_64.rpm 	查看最后发现少这个包 ，或者直接下次部署提前安装就好了
yum install hadoop_3_1_4_0_315-yarn-timelineserver.x86_64


部署好后就如下图

在这里插入图片描述