1.8 Ambari+HDP搭建hadoop(一)

总目录：https://blog.csdn.net/qq_41106844/article/details/105553392

Hadoop - 子目录：https://blog.csdn.net/qq_41106844/article/details/105553369

 

Ambari+HDP简介

虚拟机准备

主机名	IP	角色	内存	核数	磁盘
ambari.lh.com	192.168.10.190	Ambari+HDP	2G	1	20G
master.lh.com	192.168.10.191	Master	2G	1	20G
slave1.lh.com	192.168.10.192	Slave1	2G	1	20G
slave2.lh.com	192.168.10.193	Slave2	2G	1	20G

环境配置

网络配置

ifcfg-ens33

 

 

 

其他几个也是这样配置。

 

hostname

[root@ambari ~] cat /etc/hostname
ambari.lh.com

修改主机名。

network

[root@ambari ~] cat /etc/sysconfig/network
# Created by anaconda
NETWORKING=yes
HOSTNAME=ambari.lh.com

配置主机名与网络状态的关系。

hosts

[root@ambari ~] cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.10.190 ambari.lh.com
192.168.10.191 master.lh.com
192.168.10.192 slave1.lh.com
192.168.10.193 slave2.lh.com

ip与主机名映射表。

关闭防火墙

systemctl stop firewalld.service
systemctl disable firewalld.service

服务配置

安装NTP时间同步服务

yum install ntp ntpdate -y

安装httpd服务

yum install httpd -y
systemctl start httpd.service
systemctl enable httpd.service

 

检查http

关闭SELinux

vim /etc/selinux/config
修改SELINUX=disabled

ssh免密登录

生成密钥

ssh-keygen -t rsa -f ~/.ssh/id_rsa

传输密钥

ssh-copy-id ambari.lh.com
ssh-copy-id master.lh.com
ssh-copy-id slave1.lh.com
ssh-copy-id slave2.lh.com

四台互传

[root@ambari .ssh] ssh-copy-id ambari.lh.com
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'ambari.lh.com (192.168.10.190)' can't be established.
ECDSA key fingerprint is SHA256:WHSTGFtt/3LnDgU60HxnG20y4sDogJb3ECIfA/2L3Mk.
ECDSA key fingerprint is MD5:53:ae:4a:0d:8e:55:3c:d7:17:44:c9:5b:00:67:a7:cb.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any th
at are already installed/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it 
is to install the new keysroot@ambari.lh.com's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'ambari.lh.com'"
and check to make sure that only the key(s) you wanted were added.

[root@ambari .ssh] ssh-copy-id master.lh.com
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'master.lh.com (192.168.10.191)' can't be established.
ECDSA key fingerprint is SHA256:WHSTGFtt/3LnDgU60HxnG20y4sDogJb3ECIfA/2L3Mk.
ECDSA key fingerprint is MD5:53:ae:4a:0d:8e:55:3c:d7:17:44:c9:5b:00:67:a7:cb.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any th
at are already installed/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it 
is to install the new keysroot@master.lh.com's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'master.lh.com'"
and check to make sure that only the key(s) you wanted were added.

[root@ambari .ssh] ssh-copy-id slave1.lh.com
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'slave1.lh.com (192.168.10.192)' can't be established.
ECDSA key fingerprint is SHA256:WHSTGFtt/3LnDgU60HxnG20y4sDogJb3ECIfA/2L3Mk.
ECDSA key fingerprint is MD5:53:ae:4a:0d:8e:55:3c:d7:17:44:c9:5b:00:67:a7:cb.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any th
at are already installed/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it 
is to install the new keysroot@slave1.lh.com's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'slave1.lh.com'"
and check to make sure that only the key(s) you wanted were added.

[root@ambari .ssh] ssh-copy-id slave2.lh.com
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'slave2.lh.com (192.168.10.193)' can't be established.
ECDSA key fingerprint is SHA256:WHSTGFtt/3LnDgU60HxnG20y4sDogJb3ECIfA/2L3Mk.
ECDSA key fingerprint is MD5:53:ae:4a:0d:8e:55:3c:d7:17:44:c9:5b:00:67:a7:cb.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any th
at are already installed/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it 
is to install the new keysroot@slave2.lh.com's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'slave2.lh.com'"
and check to make sure that only the key(s) you wanted were added.

jdk环境

首先创建放软件的文件夹。

cd /usr
mkdir hdk

然后上传安装包，解压

[root@ambari hdk] tar -zxvf jdk-1.8.tar.gz 
[root@ambari hdk] ll
总用量 155296
drwxr-xr-x. 8   10  143       255 6月  17 2014 jdk1.8.0_11
-rw-r--r--. 1 root root 159019376 4月   3 15:16 jdk-1.8.tar.gz
[root@ambari hdk] rm jdk-1.8.tar.gz 
rm：是否删除普通文件 "jdk-1.8.tar.gz"？y
[root@ambari hdk] mv jdk1.8.0_11/ jdk8
[root@ambari hdk] ll
总用量 0
drwxr-xr-x. 8 10 143 255 6月  17 2014 jdk8

配置环境变量

vi /etc/profile

# jdk8
export JAVA_HOME=/usr/hdk/jdk8
PATH=$JAVA_HOME/bin:$PATH

export PATH

刷新环境变量

[root@ambari hdk] source /etc/profile
[root@ambari hdk] java -version
java version "1.8.0_11"
Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)

把环境变量文件和jdk8目录传到其他节点上。

scp /etc/profile root@master.lh.com:/etc
scp /etc/profile root@slave1.lh.com:/etc
scp /etc/profile root@slave2.lh.com:/etc

scp -r jdk8/ root@master.lh.com:/usr/hdk
scp -r jdk8/ root@slave1.lh.com:/usr/hdk
scp -r jdk8/ root@slave2.lh.com:/usr/hdk

然后各自刷新环境变量。
这样四个节点的java环境就ok了。

mysql5.6

先检查是否已经安装了MySQL。

rpm -qa | grep mysql

返回空就是没有安装。

然后卸载mariadb数据库，因为在centos7中默认集成mariadb数据库。

rpm -qa|grep mariadb|xargs rpm -e --nodeps

检查是否卸掉

rpm -qa|grep -i mariadb

接着正式安装mysql5.6

下载

wget http://repo.mysql.com/mysql-community-release-el7-5.noarch.rpm

安装

rpm -ivh mysql-community-release-el7-5.noarch.rpm

yum install mysql-server

设置服务

systemctl start mysqld.service #启动 mysql
systemctl restart mysqld.service #重启 mysql
systemctl stop mysqld.service #停止 mysql
systemctl enable mysqld.service #设置 mysql 开机启动

设置密码

mysql -u root

mysql> use mysql;
mysql> update user set password=PASSWORD("123456") where User='root';
mysql> flush privileges; 

设置远程访问

mysql> use mysql
mysql> select User, Host, Password from user;

mysql> update user set host='%' where host='localhost';
mysql> delete from user where Host='ambari.gey.com';
mysql> delete from user where Host='127.0.0.1';
mysql> delete from user where Host='::1';

mysql> flush privileges;