Hadoop的伪分布式的部署

部署环境 :VMware-workstation-full-12 部署CentOS-6.5-x86_64-bin.iso 虚拟机

所需软件:hadoop-2.5.2.tar.gz,jdk-7u79-linux-x64.tar.gz

注意：本人的主机名的IP是192,168.100.15 主机名是node5

第一步：主机名映射和配置yum源和关闭防火墙

主机名映射

vi /etc/hosts

127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4

::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.100.15 node5
  

配置yum源           

mount /dev/sr0 /opt/centos/

cd /etc/yum.repos.d/

rm -rf *

vi local.repo

[centos]
name=centos
baseurl=file:///opt/centos
gpgcheck=0
enabled=1

yum clean all

yum list

关闭防火墙

iptables -F
iptables -X
iptables -Z
service iptables save
vi /etc/selinux/config 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

setenforce 0

[root@node5 hadoop]# getenforce 
Permissive

第二步：安装openssh-clients和配置SSH无密码之前的登录

yum install client-openssh -y

ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa

cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys

ssh localhost

第三步：安装JDK

mkdir /usr/java

tar -zxvf jdk-7u79-linux-x64.tar.gz -C /usr/java/.

vi /etc/profile

export JAVA_HOME=/usr/java/jdk1.7.0_79
export PATH=$PATH:$JAVA_HOME/bin

source /etc/profile

[root@node5 ~]# java -version
java version "1.7.0_79"
Java(TM) SE Runtime Environment (build 1.7.0_79-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.79-b02, mixed mode)

第四步：安装和配置Hadoop

tar -zxvf hadoop-2.5.2.tar.gz -C /home/.

vi /etc/profile
export HADOOP_HOME=/home/hadoop-2.5.2
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
source /etc/profile
[root@node5 ~]# hdfs
hdfs             hdfs.cmd         hdfs-config.cmd  hdfs-config.sh

第五步：配置Hadoop文件
[root@node5 ~]# cd /home/hadoop-2.5.2/etc/hadoop/
vi hadoop-env.sh 添加
export JAVA_HOME=/usr/java/jdk1.7.0_79

vi yarn-env.sh  添加
export JAVA_HOME=/usr/java/jdk1.7.0_79
vi core-site.xml
   <configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://node5:9000</value>
    </property>
</configuration>

vi hdfs-site.xml
   <configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
 </configuration>

vi mapred-site.xml
 <configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

vi yarn-site.xml
<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

第六步：格式化namenode
   hdfs namenode -format


第七步：启动集群
   start-all.sh 


第八步：检查集群运行的状态
[root@node5 ~]# jps
4720 DataNode
4866 SecondaryNameNode
5035 ResourceManager
5448 Jps
4637 NameNode
5122 NodeManager

在浏览器输入
http://192.168.100.15:50070

http://192.168.100.15:8088