文章连接
1.spark学习-hadoop安装与启动
2.spark学习-spark安装和启动
安装前准备
1.首先准备三台服务器.一台master,两台slave.
172.18.101.157 spark-master
172.18.101.162 spark-slave1
172.18.132.162 spark-slave2
2.设置免密登录
1. 生成私钥和公钥
[root@spark-master data]# ssh-keygen -t rsa
一直敲回车,最后生产密钥
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:xmlOlzYrf0obAn5T5rN+nI39yf+lXTibaETnu72yEN8 root@spark-master
The key's randomart image is:
+---[RSA 2048]----+
| |
| |
| |
| . . .. . |
| . S Bo o |
| . * * o+ o. |
| . * *+ *oEo|
| . * =Oo=B+|
| .*=..B*X|
+----[SHA256]-----+
[root@spark-master data]#
2. 将公钥复制到slave机器上
[root@spark-master data]# scp /root/.ssh/id_rsa.pub root@172.18.101.162:/data/
[root@spark-master data]# scp /root/.ssh/id_rsa.pub root@172.18.132.162:/data/
进入slave机器,如公钥导入授权文件中
[root@spark-slave1 ~]# cd /data/
[root@spark-slave1 data]# cat id_rsa.pub >> /root/.ssh/authorized_keys
#返回master机器,测试是否配置成功
#测试slave1
[root@spark-master ~]# ssh spark-slave1
Last login: Fri Jun 22 20:07:35 2018 from 172.18.101.157
Welcome to Alibaba Cloud Elastic Compute Service !
#测试slave2
[root@spark-master ~]# ssh spark-slave2
Last login: Fri Jun 22 20:07:31 2018 from 172.18.101.157
Welcome to Alibaba Cloud Elastic Compute Service !
3.安装jdk
vim /etc/profile
配置环境变量
JAVA_HOME=/opt/jdk8
CLASSPATH=.:$JAVA_HOME/lib.tools.jar
PAT