centos系统下ambari离线部署hadoop

最新推荐文章于 2024-04-26 08:50:32 发布

coder_who

最新推荐文章于 2024-04-26 08:50:32 发布

阅读量742

点赞数

文章标签： hadoop ambari 大数据

本文链接：https://blog.csdn.net/QQ83512272/article/details/126548508

版权

一、准备工作

1.准备环境

ambari-2.7.3.0

HDP-3.1.0

HDP-UTILS-1.1.0.22

JDK：1.8

MySQL：5.7

主机若干台：

主机名称	IP地址	用途	备注
node1.domain	172.27.10.171	Ambari-server、namendoe	yum源主机
node2.domain	172.27.10.172	Datanode、NodeManager
node3.domain	172.27.10.173	Datanode、NodeManager
node4.domain	172.27.10.174	Datanode、NodeManager
node5.domain	172.27.10.175	Datanode、NodeManager

2.关闭防火墙(所有节点)

# service iptables stop

# chkconfig iptables off

关闭SELINUX

# vi /etc/selinux/config

修改SELINUX=enforcing为 SELINUX=disabled

3.配置打开文件数(所有节点)

# vi /etc/security/limits.conf
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

4.修改主机名host

#vi /etc/hosts
172.27.10.171 node1.domain node1
172.27.10.172 node2.domain node2
172.27.10.173 node3.domain node3
172.27.10.174 node4.domain node4
172.27.10.175 node5.domain node5

修改完成之后重启服务器生效

reboot

5.修改语言为英文

# vi /etc/locale.conf
LANG="en_US.UTF-8"

6.配置免密登录

配置master节点无密码登录到其他节点，在master节点上操作
生成密钥对（所有机器）

# ssh-keygen -t rsa   ## 一路回车即可(每个机器执行此操作)

在主节点上将公钥拷到一个特定文件authorized_keys中。

[root@node1 ~]# cd .ssh
[root@node1 .ssh]# ls
id_rsa  id_rsa.pub
[root@node1 .ssh]# cp id_rsa.pub authorized_keys
[root@node1 .ssh]# ls
authorized_keys  id_rsa  id_rsa.pub

修改文件权限

[root@node1 ~]# chmod 700 ~/.ssh
[root@node1 ~]# chmod 600 ~/.ssh/authorized_keys

将authorized_keys文件拷到下一个节点，并将该节点的ssh密钥id_rsa.pub加入该文件中在(每一个机器重复该步骤)

#在vm1上使用scp命令实现远程文件拷贝
[root@node1 .ssh]# scp authorized_keys  root@node2:/root/.ssh/

The authenticity of host 'node2(192.168.44.11)' can't be established.
ECDSA key fingerprint is SHA256:MyB1zs0E3J/fm8pC0AN8ycsgEIBNHtUqd9xS0WAyv3s.
ECDSA key fingerprint is MD5:88:48:3a:ba:3e:14:a7:d7:86:f6:51:74:00:10:f9:00.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop02,192.168.44.11' (ECDSA) to the list of known hosts.
root@node1's password: 
authorized_keys                                                                100%  395   306.2KB/s   00:00    

#登录node2主机
[root@node2~]# cd .ssh/
[root@node2.ssh]# ls
authorized_keys id_rsa id_rsa.pub
[root@node2.ssh]# cat id_rsa.pub >> authorized_keys #使用cat追加方式

重复上一步的操作，依次将node3、node4、node5节点的ssh密钥加入到authorized_keys文件中，并将node5节点生成的authorized_keys文件拷贝到其他四个节点（node1、node2、node3、node4）即可

[root@node5 .ssh]# scp authorized_keys  root@node5:/root/.ssh/
[root@node5 .ssh]# scp authorized_keys  root@node5:/root/.ssh/
[root@node5 .ssh]# scp authorized_keys  root@node5:/root/.ssh/
[root@node5 .ssh]# scp authorized_keys  root@node5:/root/.ssh/

验证免密登录
使用ssh 用户名@节点名或ssh ip地址命令验证免密码登录。

[root@node5 .ssh]# ssh node1

将创建的秘钥拷贝出来
因为后面ambari安装的时候需要上传这个秘钥。创建秘钥是在隐藏文件夹/root/.ssh/下面的，所以需要先把秘钥拷贝到可见区域，我们这边把它拷贝到/home/tools下，也拷贝到本地桌面一份

[root@node1~]# mkdir /home/tools
[root@node1~]# cp /root/.ssh/id_rsa /home/tools/
[root@node1~]# ls /home/tools
id_rsa
注：是id_rsa，而不是id_rsa.pub

7.服务器时间同步

安装ntp服务（所有机器）

###进入tools文件夹 创建ntp文件夹 （所有机器一起安装）
[root@node1 ~]# cd /home/tools
[root@node1 ~]# mkdir ntp
###上传三个rpm包到ntp文件夹
###   ntpdate-4.2.6p5-29.el7.centos.2.x86_64.rpm
###   autogen-libopts-5.18-5.el7.x86_64.rpm
###   ntp-4.2.6p5-29.el7.centos.2.x86_64.rpm
### 依次安装
[root@node1 ~]# rpm -ivh ntpdate-4.2.6p5-29.el7.centos.2.x86_64.rpm
[root@node1 ~]# rpm -ivh autogen-libopts-5.18-5.el7.x86_64.rpm
[root@node1 ~]# rpm -ivh ntp-4.2.6p5-29.el7.centos.2.x86_64.rpm

设置node1为主服务器，开启nptd服务(主服务器)

# vi /etc/ntp.conf

需要修改的部份：
restrict 192.168.0.0 mask 255.255.255.0
 
#注释掉其他上游时间服务器
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
 
新增：
server 127.127.1.0
fudge 127.127.1.0 stratum 10

启动服务

# systemctl start ntpd.service

开机自启

# systemctl enable ntpd.service

子节点设置同步（子节点）

主服务器开启ntp服务器以后，子节点就不需要开启了，因为当server与client之间的时间误差过大时（可能是1000秒），处于对修改时间可能对系统和应用带来不可预知的问题，NTP将停止时间同步！所以如果发现NTP启动之后时间并不进行同步时，应该考虑到可能是时间差过大引起的，此时需要先手动进行时间同步！所以直接使用定时手动同步的方式就可以了。

添加任务计划实时同步master服务器时间

[root@node2~]# crontab -e
0-59/10 * * * * /usr/sbin/ntpdate vm1
[root@node2~]# crontab -l
0-59/10 * * * * /usr/sbin/ntpdate vm1

8.禁用SELinux和PackageKit将检查umask值

禁用sellinux(所有机器)

[root@node1 ~]# vi /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

SELINUX改为disabled

umask值(所有机器)

[root@node1 ~]# sudo sh -c "echo umask 0022 >> /etc/profile"
[root@node1 ~]#  source /etc/profile

9.服务器的文件句柄数设置(所有机器)

[root@node1 ~]# vi /etc/security/limits.conf
# End of file
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

10.jdk安装(所有机器)

上传jdk到/home/tools/目录下 

[root@node1~]# cd /home        ###所有机器
[root@node1 tools]# mkdir tools      ###所有机器创建tools文件夹
[root@node1~]# cd /home/tools
#把JDK复制到其他四个机器然后解压
[root@node1 tools]# scp -r /home/tools/ root@vm2:/home/tools
[root@node1 tools]# tar -zxvf jdk-8u271-linux-x64.tar.gz
[root@node1 tools]# rm -rf jdk-8u271-linux-x64.tar.gz

进入配置JAVA环境（所有机器） 添加环境配置
[root@node1 tools]# vi /etc/profile
###添加环境变量配置
export JAVA_HOME=/home/tools/jdk1.8.0_271
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
 
###使环境变量生效
[root@node1 tools]# source /etc/profile
###查看jdk安装版本
[root@node1 tools]# java -version

11.mysql安装

本地安装或docker拉取官方镜像mysql:5.7（建议）

mysql安装完成之后：

启动mysql：

systemctl start mysqld

开机启动：

systemctl enable mysqld

获取并修改初始密码：

修改初始密码
# cat /var/log/mysqld.log |grep password
# mysql -uroot -p'初始密码'
> uninstall plugin validate_password;
> set password=password('新密码');

ambari库和hive元数据库创建

> create database ambari;

> grant all privileges on ambari.* to ambari@'node1.domain' identified by 'bigdata';

> grant all privileges on ambari.* to ambari@'192.168.100.99' identified by 'bigdata';

> grant all privileges on ambari.* to ambari@'192.168.100.%' identified by 'bigdata';

> flush privileges;

> create database metastore default character set latin1;
         //字符必须为拉丁(latin1),如果uft8可能会报错Specified key was too long; max key length is 767 bytes

> create database metastore default character set latin1;

> grant all privileges on hive.* to hive@'192.168.100.%' identified by 'hive';

> grant all privileges on hive.* to hive@'node1.domain' identified by 'hive';
         > flush privileges;

12.JDBC驱动安装

下载并上传mysql-connector-java

[root@localhost java]# mkdir -p /usr/share/java && cd /usr/share/java
[root@localhost java]# ll
total 5564
lrwxrwxrwx. 1 root root      23 Jul  1 06:22 icedtea-web.jar -> ../icedtea-web/netx.jar
lrwxrwxrwx. 1 root root      25 Jul  1 06:22 icedtea-web-plugin.jar -> ../icedtea-web/plugin.jar
-rw-r--r--. 1 root root   62891 Jun 10  2014 jline.jar
-rw-r--r--. 1 root root 1079759 Aug  1  2017 js.jar
-rw-r--r--  1 root root 4434926 Aug 26 08:17 mysql-connector-java-5.1.46.tar.gz
-rw-r--r--. 1 root root   18387 Aug  1  2017 rhino-examples.jar
lrwxrwxrwx. 1 root root       6 Jul  1 06:21 rhino.jar -> js.jar
-rw-r--r--. 1 root root   92284 Mar  6  2015 tagsoup.jar
[root@localhost java]# tar zxf mysql-connector-java-5.1.46.tar.gz
[root@localhost java]# cp mysql-connector-java-5.1.46/mysql-connector-java-5.1.46-bin.jar .
[root@localhost java]# mv mysql-connector-java-5.1.46-bin.jar mysql-connector-java.jar
[root@localhost java]# chmod +x mysql-connector-java.jar
[root@localhost java]# ll
total 6548
lrwxrwxrwx. 1 root root      23 Jul  1 06:22 icedtea-web.jar -> ../icedtea-web/netx.jar
lrwxrwxrwx. 1 root root      25 Jul  1 06:22 icedtea-web-plugin.jar -> ../icedtea-web/plugin.jar
-rw-r--r--. 1 root root   62891 Jun 10  2014 jline.jar
-rw-r--r--. 1 root root 1079759 Aug  1  2017 js.jar
drwxr-xr-x  3 root root     178 Feb 26  2018 mysql-connector-java-5.1.46
-rw-r--r--  1 root root 4434926 Aug 26 08:17 mysql-connector-java-5.1.46.tar.gz
-rwxr-xr-x  1 root root 1004840 Aug 26 08:17 mysql-connector-java.jar
-rw-r--r--. 1 root root   18387 Aug  1  2017 rhino-examples.jar
lrwxrwxrwx. 1 root root       6 Jul  1 06:21 rhino.jar -> js.jar
-rw-r--r--. 1 root root   92284 Mar  6  2015 tagsoup.jar

13.配置本地yum源

参考Linux（麒麟V10）搭建本地yum源_Truman UP的博客-CSDN博客

用下载好的centos镜像代替麒麟镜像即可

centos镜像下载地址：http://mirrors.aliyun.com/centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso?spm=a2c6h.25603864.0.0.60196aeachezLa

14.httpd安装

# yum install httpd createrepo -y

15.配置ambari源和hdp源

上传ambari和hdp包到服务器并解压到制定路径下：

[root@localhost home]# cd hdp/
[root@localhost hdp]# ll
total 10845148
-rw-r--r-- 1 root root 1947685893 Aug 26 07:32 ambari-2.7.3.0-centos7.tar.gz
-rw-r--r-- 1 root root 9066967592 Aug 26 08:01 HDP-3.1.0.0-centos7-rpm.tar.gz
-rw-r--r-- 1 root root     162100 Aug 26 07:25 HDP-GPL-3.1.0.0-centos7-gpl.tar.gz
-rw-r--r-- 1 root root   90606616 Aug 26 07:25 HDP-UTILS-1.1.0.22-centos7.tar.gz
[root@localhost hdp]# tar -zxf ambari-2.7.3.0-centos7.tar.gz -C /var/www/html/
[root@localhost hdp]# tar -zxf HDP-3.1.0.0-centos7-rpm.tar.gz -C /var/www/html/
[root@localhost hdp]# tar -zxf HDP-GPL-3.1.0.0-centos7-gpl.tar.gz -C /var/www/html/
[root@localhost hdp]# tar -zxf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/

创建repo：

[root@localhost hdp]# createrepo /var/www/html/HDP/centos7/3.1.0.0-78
Spawning worker 0 with 51 pkgs
Spawning worker 1 with 50 pkgs
Spawning worker 2 with 50 pkgs
Spawning worker 3 with 50 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@localhost hdp]# createrepo /var/www/html/ambari/centos7/2.7.3.0-139
Spawning worker 0 with 4 pkgs
Spawning worker 1 with 3 pkgs
Spawning worker 2 with 3 pkgs
Spawning worker 3 with 3 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@localhost hdp]# createrepo /var/www/html/HDP-GPL/centos7/3.1.0.0-78
Spawning worker 0 with 1 pkgs
Spawning worker 1 with 1 pkgs
Spawning worker 2 with 1 pkgs
Spawning worker 3 with 1 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@localhost hdp]# createrepo /var/www/html/HDP-UTILS/centos7/1.1.0.22
Spawning worker 0 with 4 pkgs
Spawning worker 1 with 4 pkgs
Spawning worker 2 with 4 pkgs
Spawning worker 3 with 4 pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete

配置repo：

# cd /etc/yum.repos.d/
# vim ambari.repo 
#VERSION_NUMBER=2.7.3.0-139
[ambari-2.7.3.0]
name=ambari Version - ambari-2.7.3.0
baseurl=http://192.168.100.99/ambari/centos7/2.7.3.0-139/
enabled=1
gpgcheck=0
priority=1
# vim hdp.gpl.repo
#VERSION_NUMBER=3.1.0.0-78
[HDP-GPL-3.1.0.0]
name=HDP-GPL Version - HDP-GPL-3.1.0.0
baseurl=http://192.168.100.99/HDP-GPL/centos7/3.1.0.0-78/
enabled=1
gpgcheck=0
priority=1
# vim hdp.repo
[HDP-3.1.0.0]
name=HDP Version - HDP-3.1.0.0
baseurl=http://192.168.100.99/HDP/centos7/3.1.0.0-78/
enabled=1
gpgcheck=0
priority=1
# vim hdp-utils.repo
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://192.168.100.99/HDP-UTILS/centos7/1.1.0.22/
enabled=1
gpgcheck=0
priority=1

启动httpd：

systemctl start httpd

验证：

浏览器访问http://10.142.33.227/HDP-UTILS/centos7/1.1.0.22/

二、安装ambari-server及ambari-agent

安装

yum install ambari-server ambari-agent -y

初始化ambari数据库连接配置

ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

配置ambari-server

[root@localhost java]# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)?
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /home/jdk/jdk1.8.0_333
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)?
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Configuring ambari database...
Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)?
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)?
Extracting system views...
ambari-admin-2.7.3.0.139.jar
....
Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

初始化ambari数据库

mysql -uroot -p -Dambari

>source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;

启动ambari-server

[root@localhost java]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start.......................................ERROR: Exiting with exit code -1.
REASON: Ambari Server java process has stopped. Please check the logs for more information.

启动报错，查看log：

The last packet successfully received from the server was 1 milliseconds ago.  The last packet sent successfully to the server was 1 milliseconds ago.
        at sun.reflect.GeneratedConstructorAccessor55.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
        at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:201)
        at com.mysql.jdbc.MysqlIO.negotiateSSLConnection(MysqlIO.java:4912)
        at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1663)
        at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1224)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2190)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2221)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2016)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:776)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.GeneratedConstructorAccessor48.newInstance(Unknown Source)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
        at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
        at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
        at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
        at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
        at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
        at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
        at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
        at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
        at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
        at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:106)
        at sun.security.ssl.TransportContext.kickstart(TransportContext.java:238)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:405)

找到关键报错信息：

Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)

解决办法：

修改JDK 下的为 java.security文件：

打开文件所在目录 /home/jdk/jdk1.8.0_333/jre/lib/security/
	修改文件：java.security
	搜索以下内容：
	jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, \
	    DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL, \
	    include jdk.disabled.namedCurves
	去除SSLv3, TLSv1, TLSv1.1这三个选项

修改之后：

# Example:
#   jdk.tls.disabledAlgorithms=MD5, SSLv3, DSA, RSA keySize < 2048, \
#       rsa_pkcs1_sha1
#jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, \
jdk.tls.disabledAlgorithms=RC4, DES, MD5withRSA, \
    DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL, \
    include jdk.disabled.namedCurves

再次启动ambari-server:

[root@localhost security]# ambari-server start
Using python  /usr/bin/python
Starting ambari-server
Ambari Server running with administrator privileges.
Organizing resource files at /var/lib/ambari-server/resources...
Ambari database consistency check started...
Server PID at: /var/run/ambari-server/ambari-server.pid
Server out at: /var/log/ambari-server/ambari-server.out
Server log at: /var/log/ambari-server/ambari-server.log
Waiting for server start..................
Server started listening on 8080

DB configs consistency check: no errors and warnings were found.
Ambari Server 'start' completed successfully.

ambari-server启动成功！

启动ambari-agent

[root@localhost ~]# ambari-agent start
Verifying Python version compatibility...
Using python  /usr/bin/python
Checking for previously running Ambari Agent...
Checking ambari-common dir...
Starting ambari-agent
Verifying ambari-agent process status...
Ambari Agent successfully started
Agent PID at: /run/ambari-agent/ambari-agent.pid
Agent out at: /var/log/ambari-agent/ambari-agent.out
Agent log at: /var/log/ambari-agent/ambari-agent.log

验证：

浏览器打开http://服务器IP:8080/

出现如下登录页面，则ambari安装成功

三、安装Hadoop

1. 浏览器打开http://服务器IP:8080/

登陆账号:admin

登陆密码:admin

2.登录之后点击“Launch Install Wizard”按钮

3.输入集群名字，下一步

4.配置HDP安装Repositories:

选择Use Local Repository，填入前面创建好的hdp源地址

5.配置集群节点和密钥

注册主机

选择要安装的服务：

确认主机信息资源节点分配

角色分配

设置密码

注：hive的密码需要跟前面MySQL中授权的用户密码一致，否则无法连接数据库

配置hive数据库

目录配置，直接下一步

账户配置，直接下一步

预览

点击DEPLOY开始部署

如果报错：Cannot find a valid baseurl for repo: HDP-3.1-repo-1

解决办法：

cd /etc/yum.repos.d/
rm -rf ambari-hdp-1.repo
ambari-server stop
cd /usr/lib/ambari-server/web/javascipts
vi app.js

定位到39892行，修改如下代码：
  /**
   * Use Local Repo if some network issues exist
   */
  onNetworkIssuesExist: function () {
    if (this.get('networkIssuesExist')) {
      this.get('content.stacks').forEach(function (stack) {
          stack.setProperties({
            usePublicRepo: false,
            useLocalRepo: true
          });
          stack.cleanReposBaseUrls();
      });
    }
  }.observes('networkIssuesExist'),
修改后：
  /**
   * Use Local Repo if some network issues exist
   */
  onNetworkIssuesExist: function () {
    if (this.get('networkIssuesExist')) {
      this.get('content.stacks').forEach(function (stack) {
        if(stack.get('useLocalRepo') != true){
          stack.setProperties({
            usePublicRepo: false,
            useLocalRepo: true
          });
          stack.cleanReposBaseUrls();
        } 
      });
    }

连接ambari数据库，修改repo_definition表，添加base_url信息：

重新启动ambari-server

ambari-server start

安装过程如果报错：Package: hadoop_3_1_0_0_78-hdfs-2.7.3.2.6.0.3-8.ppc64le (HDP-3.1) Requires: libtirpc-devel

解决办法：

下载依赖libtirpc-devel，下载地址：http://mirror.centos.org/centos/7/os/x86_64/Packages/libtirpc-devel-0.2.4-0.16.el7.x86_64.rpm

安装：

rpm -ivh libtirpc-devel-0.2.4-0.16.el7.x86_64.rpm

等待所有组件安装完成

查看监控页面：

coder_who

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
1
评论
centos系统下ambari离线部署hadoop

centos系统下ambari离线部署hadoop
复制链接

扫一扫

centos系统下ambari离线部署hadoop

一、准备工作

1.准备环境

2.关闭防火墙(所有节点)

3.配置打开文件数(所有节点)

4.修改主机名host

5.修改语言为英文

6.配置免密登录

7.服务器时间同步

安装ntp服务（所有机器）

设置node1为主服务器，开启nptd服务(主服务器)

启动服务

开机自启

子节点设置同步（子节点）

8.禁用SELinux和PackageKit将检查umask值

禁用sellinux(所有机器)

umask值(所有机器)

9.服务器的文件句柄数设置(所有机器)

10.jdk安装(所有机器)

11.mysql安装

12.JDBC驱动安装

13.配置本地yum源

14.httpd安装

15.配置ambari源和hdp源

二、安装ambari-server及ambari-agent

三、安装Hadoop

“相关推荐”对你有帮助么？