（二）Spark-Linux环境准备-Java&Python版Spark

Spark-Linux环境准备

视频教程：

1、优酷

2、YouTube

 

硬软件环境

1、虚拟机：VMware Workstation 12

2、虚拟机操作系统：RedHat5u4，单核，1G内存，2两台

3、虚拟机运行环境：

java version "1.7.0_79" 64位

Scala version 2.10.5

hadoop-2.6.0

spark-2.0.0-bin-hadoop2.6

Python 2.7.12

 

配置IP     

192.168.3.100 master

192.168.3.101 slave

 

配置主机名

[root@localhost ~]# vim /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME=master

[root@localhost ~]# hostname master

三台主机都配置主机名

[root@localhost ~]# vim /etc/sysconfig/network

NETWORKING=yes

NETWORKING_IPV6=no

HOSTNAME= slave

[root@localhost ~]# hostname slave

重启生效

 

关闭防火墙和selinux

[root@master ~]# vim /etc/sysconfig/selinux

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

#       enforcing - SELinux security policy is enforced.

#       permissive - SELinux prints warnings instead of enforcing.

#       disabled - SELinux is fully disabled.

SELINUX=disabled

# SELINUXTYPE= type of policy in use. Possible values are:

#       targeted - Only targeted network daemons are protected.

#       strict - Full SELinux protection.

SELINUXTYPE=targeted

 

[root@master ~]# service iptables status

表格：filter

Chain INPUT (policy ACCEPT)

num  target     prot opt source               destination         

 

Chain FORWARD (policy ACCEPT)

num  target     prot opt source               destination         

 

Chain OUTPUT (policy ACCEPT)

num  target     prot opt source               destination    

 

域名解析

使得master和slave互相连接对方

[root@master ~]# vim /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1               localhost.localdomain localhost

::1             localhost6.localdomain6 localhost6

 

192.168.3.100 master

192.168.3.101 slave

 

[root@slave ~]# vim /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1               localhost.localdomain localhost

::1             localhost6.localdomain6 localhost6

 

192.168.3.100 master

192.168.3.101 slave

 

转载于:https://www.cnblogs.com/LgyBean/p/6251211.html