Hadoop 学习第一篇之Ambari部署。

最新推荐文章于 2023-12-29 00:00:00 发布

Chang Tienhao

最新推荐文章于 2023-12-29 00:00:00 发布

阅读量226

点赞数

本文链接：https://blog.csdn.net/qq_37348649/article/details/103583939

版权

Hadoop 同时被 3 个专栏收录

5 篇文章 0 订阅

订阅专栏

ambari

2 篇文章 0 订阅

订阅专栏

mysql

1 篇文章 0 订阅

订阅专栏

一）install JDK

必须下载jdk1.8，这才能稳定版本服务，

安装步骤

#解压下载好的安装包
tar -zxvf jdk1_8xxxxxxx


#配置环境变量
vim /etc/profile

export JAVA_HOME=/usr/local/java/jdk1.8.0_171
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

#让配置文件生效
source /etc/profile
#查看版本
java -version

注意：所涉及到的集群服务器都要部署jdk。

二）配置本机名

注意：根据自己的需求自行定义

三）修改文件打开限制

[root@master ~]# vi /etc/security/limits.conf
# End of file
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

四）设置时间同步

#安装ntp服务：如果是企业服务器的话不需要管理
yum -y install ntp

五）SSH免密

#每台服务器都要进行操作
ssh-keygen -t rsa
详情:https://blog.csdn.net/qq_36523839/article/details/78885216。

六）安装Httpd服务

[root@master ~]# yum -y install httpd
[root@master ~]# service httpd restart
Redirecting to /bin/systemctl restart httpd.service
[root@master ~]# chkconfig httpd on

注意：1、端口号默认是80，系要去/etc/httpd/httpd.conf进行修改，切勿冲突

2、通过Httpd 去创建数据源路径，常理是放在/var/www/html下面，但是内存不足的话建议使用软连接

#软连接的格式
# ln -s /你想要查找的路径   httpd的URl后端拼接的名称
ln  -s  /onstardata/hadoop/  ambari

七）安装本地数据源制作相关工具，只在主服务器上即可

[root@master ambari]#  yum install yum-utils createrepo yum-plugin-priorities -y
[root@master ambari]#  createrepo  ./

八）修改文件里面的源地址（主服务器）

[root@master ambari]# vi ambari/centos7/2.6.0.0-267/ambari.repo
#VERSION_NUMBER=2.6.0.0-267
[ambari-2.6.0.0]
name=ambari Version - ambari-2.6.0.0
baseurl=http://ip+port/ambari/ambari/centos7/2.6.0.0-267
gpgcheck=1
gpgkey=http://ip+port/ambari/ambari/centos7/2.6.0.0-267/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@master ambari]# cp ambari/centos7/2.6.0.0-267/ambari.repo /etc/yum.repos.d/
[root@master ambari]# vi HDP/centos7/2.6.3.0-235/hdp.repo
#VERSION_NUMBER=2.6.3.0-235
[HDP-2.6.3.0]
name=HDP Version - HDP-2.6.3.0
baseurl=http://ip+port/ambari/HDP/centos7/2.6.3.0-235
gpgcheck=1
gpgkey=http://ip+port/ambari/HDP/centos7/2.6.3.0-235/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1


[HDP-UTILS-1.1.0.21]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.21
baseurl=http://ip+port/ambari/HDP-UTILS
gpgcheck=1
gpgkey=http://ip+port/ambari/HDP-UTILS/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
[root@master ambari]# cp HDP/centos7/2.6.3.0-235/hdp.repo /etc/yum.repos.d/

#清理yum缓存
[root@master ambari]# yum clean all
[root@master ambari]# yum makecache
[root@master ambari]# yum repolist

注意：把这些数据源cp到子服务器上

#方法一： 
scp HDP-2.6.3.0-centos7-rpm.tar.gz hadoop@10.218.103.24:/onstardata
#方法二：
# 各个服务器上执行命令，利用 rz 和 sz 进行导入导出
 1、yum -y install lrzsz

九）安装数据库只需要在主服务器上操作

#1、下载mysql rpm源
wget http://repo.mysql.com/mysql57-community-release-el7-8.noarch.rpm
#如果提示-bash: wget: command not found
yum install wget

#2、安装rpm源
rpm -ivh mysql57-community-release-el7-8.noarch.rpm

#3、安装mysql-server
yum install mysql-server

#4、连接服务器 
mysql -uroot -p root
#错误如下：ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
#错误分析以及解决 /var/lib/mysql权限问题，修改MySQL权限为当前用户
sudo chown -R xxx:xxx /var/lib/mysql
#如上xxx:xxx当前的用户名以及所属组 我这里是root:root
#重启MySQL服务

#5、重新登陆
mysql -u root -p root
#出现如下问题 
#ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
#解决方案(获取初始密码)
cat /var/log/mysqld.log  | grep password

#如上图所示为初始密码_获得后继续登录
# Mysql登录成功之后
set global validate_password_policy=0;
set global validate_password_length=4;
#设置密码
alter user 'root'@'localhost' identified by 'root';

#部署数据库执行
yum –y install ambari
CREATE DATABASE ambari;  
use ambari;  
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambarizk123';  
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
FLUSH PRIVILEGES;  
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql  
show tables;
use mysql;  
select Host User Password from user where user='ambari';  

CREATE DATABASE hive;  
use hive;  
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';  
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
FLUSH PRIVILEGES;  

CREATE DATABASE oozie;  
use oozie;  
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';  
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';
FLUSH PRIVILEGES;

十）建立Mysql和Ambari-Server建立连接

# 自行下载 mysql-connector-java.jar 5.7版本的 
mkdir /usr/share/java
cp /root/mysql-connector-java-5.1.40.jar /usr/share/java/mysql-connector-java.jar
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/mysql-jdbc-driver.jar
vi /etc/ambari-server/conf/ambari.properties
#添加server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

十一）初始化ambari-server并启动

[root@master ~]# ambari-server setup
下面是配置执行流程，按照提示操作
（1） 提示是否自定义设置。输入：y
Customize user account for ambari-server daemon [y/n] (n)? y
（2）ambari-server 账号。
Enter user account for ambari-server daemon (root):
如果直接回车就是默认选择root用户
如果输入已经创建的用户就会显示：
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
（3）检查防火墙是否关闭
Adjusting ambari-server permissions and ownership...
Checking firewall...
WARNING: iptables is running. Confirm the necessary Ambari ports are accessible. Refer to the Ambari documentation for more details on ports.
OK to continue [y/n] (y)?
直接回车
（4）设置JDK。输入：3
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[3] Custom JDK
============================================================================
Enter choice (1): 3
如果上面选择3自定义JDK,则需要设置JAVA_HOME。输入：/usr/java/jdk1.8.0_161
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /usr/java/jdk1.8.0_131
Validating JDK on Ambari Server...done.
Completing setup...
（5）数据库配置。选择：y
Configuring database...
Enter advanced database configuration [y/n] (n)? y
（6）选择数据库类型。输入：3
Configuring database...
============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
============================================================================
Enter choice (3): 3
（7）设置数据库的具体配置信息，根据实际情况输入，如果和括号内相同，则可以直接回车。如果想重命名，就输入。
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):ambarizk123
Re-Enter password: ambarizk123
（8）将Ambari数据库脚本导入到数据库
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql 
Proceed with configuring remote database connection properties [y/n] (y)?  
[root@master ~]# ambari-server start

这里出问题一定看日志，因为问题各有个的不同。/var/log/ambari-server/ambari-server.log