CentOS 6.5离线安装cloudera manager hadoop spark集群

1、主机规划

主机名 chd1 10.10.5.201 8G 2CPU
主机名 chd2 10.10.5.202 8G 2CPU
主机名 chd3 10.10.5.203 8G 2CPU
主机名 chd4 10.10.5.204 8G 2CPU

2、防火墙关闭

 # service iptables stop
 # chkconfig iptables off restart

3、yum源更新

更新国内的yum源，速度更快。

# cd /etc
# cp -r yum.repos.d/ yum.repos.d.bak
# cd yum.repos.d
# rm -fr *
# vi CentOS6-Base-163.repo 

以下为CentOS6-Base-163.repo的内容。

# CentOS-Base.repo
#
# The mirror system uses the connecting IP address of the client and the
# update status of each mirror to pick mirrors that are updated to and
# geographically close to the client.  You should use this for CentOS updates
# unless you are manually picking other mirrors.
#
# If the mirrorlist= does not work for you, as a fall back you can try the 
# remarked out baseurl= line instead.
#
#

[base]
name=CentOS-$releasever - Base - 163.com
baseurl=http://mirrors.163.com/centos/$releasever/os/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-6

#released updates 
[updates]
name=CentOS-$releasever - Updates - 163.com
baseurl=http://mirrors.163.com/centos/$releasever/updates/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-6

#additional packages that may be useful
[extras]
name=CentOS-$releasever - Extras - 163.com
baseurl=http://mirrors.163.com/centos/$releasever/extras/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras
gpgcheck=1
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-6

#additional packages that extend functionality of existing packages
[centosplus]
name=CentOS-$releasever - Plus - 163.com
baseurl=http://mirrors.163.com/centos/$releasever/centosplus/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus
gpgcheck=1
enabled=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-6

#contrib - packages by Centos Users
[contrib]
name=CentOS-$releasever - Contrib - 163.com
baseurl=http://mirrors.163.com/centos/$releasever/contrib/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contrib
gpgcheck=1
enabled=0
gpgkey=http://mirror.centos.org/centos/RPM-GPG-KEY-CentOS-6

更新玩yum源后，执行下边命令更新yum配置，使操作立即生效

# yum makecache

4、关闭selinux

# cd /etc/selinux/
# vim config

将SELINUX 修改为disabled

 # This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

5、安装openssh等

# yum -y install openssh-server openssh-clients wget vim
# yum install mlocate -y
updatedb

6、免密码登录

# vi /etc/hosts

编辑hosts表

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.10.5.201 cdh1
10.10.5.202 cdh2
10.10.5.203 cdh3
10.10.5.204 cdh4

每台机器上执行：

# ssh-keygen -t rsa

一路回车

# cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
# scp cdh2:~/.ssh/id_rsa.pub id_rsa.pub.cdh2
# scp cdh3:~/.ssh/id_rsa.pub id_rsa.pub.cdh3
# scp cdh4:~/.ssh/id_rsa.pub id_rsa.pub.cdh4
# cat id_rsa.pub.cdh2 >> authorized_keys
# cat id_rsa.pub.cdh3 >> authorized_keys
# cat id_rsa.pub.cdh4 >> authorized_keys
# chmod 600 authorized_keys
# scp authorized_keys cdh2:~/.ssh/authorized_keys
# scp authorized_keys cdh3:~/.ssh/authorized_keys
# scp authorized_keys cdh4:~/.ssh/authorized_keys

7、oracle官网下载 jdk8， 并安装

jdk-8u60-linux-x64.rpm

# rpm -qa | grep java

如果有以下则删除。

java-1.7.0-openjdk-1.7.0.45-2.4.3.3.el6.x86_64
java-1.6.0-openjdk-1.6.0.0-1.66.1.13.0.el6.x86_64
tzdata-java-2013g-1.el6.noarch

如果有则删除

# rpm -e --nodeps java-1.7.0-openjdk-1.7.0.45-2.4.3.3.el6.x86_64
# rpm -e --nodeps java-1.6.0-openjdk-1.6.0.0-1.66.1.13.0.el6.x86_64

# rpm -ivh jdk-8u60-linux-x64.rpm
# java -version

java version "1.8.0_60"
Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)

8、环境变量

编辑 /etc/profile文件

#vim /etc/profile

export JAVA_HOME=/usr/java/jdk1.8.0_60
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

# source /etc/profile

9、配置ntp服务. ntp服务主节点为cdh4

每个节点

# yum install ntp -y

# service ntpd start
# chkconfig ntpd on

ntp服务主节点，此处为cdh4

# ntpstat
synchronised to NTP server (202.112.29.82)