一、hadoop核心组件:
HDFS(分布式文件系统)
YARN(运算资源调度系统)
MAPREDUCE(分布式运算编程框架)
二、下载解压
下载地址:http://mirrors.shu.edu.cn/apache/hadoop/common/
解压
[hadoop@vm2 ~]$ tar -zxvf hadoop-2.6.4.tar.gz
三、环境变量
安装jdk并配置环境变量
四、修改配置文件
修改hadoop-env.sh
[hadoop@vm2 hadoop]$ vi /home/hadoop/hadoop-2.6.4/etc/hadoop/hadoop-env.sh
修改hadoop-env.sh的JAVA_HOME变量
# The java implementation to use.
export JAVA_HOME=/root/apps/jdk1.7.0_45
修改core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://vm1:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/hadoop/tmp</value>
</property>
</configuration>
修改hdfs-site.xml
<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>/home/hadoop/hadoop/data/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/home/hadoop/hadoop/data/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>dfs.secondary.http.address</name>
<value>vm1:50090</value>
</property>
</configuration>
修改mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
修改yarn-site.xml
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.hostname</name>
<value>vm1</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
修改salves
vm1
vm2
vm3
五、分发
[hadoop@vm2 ~]$ scp -r hadoop/ hadoop@vm3:hadoop/
[hadoop@vm2 ~]$ scp -r hadoop/ hadoop@vm3:hadoop/
六、启动
初始化hdfs
[hadoop@vm2 hadoop]$ bin/hadoop namenode -format
启动hdfs
[hadoop@vm2 hadoop]$ sbin/start-dfs.sh
启动yarn
[hadoop@vm2 hadoop]$ sbin/start-yarn.sh
使用jps检查是否启动成功,如下则成功
[hadoop@vm2 hadoop]$ jps
1866 SecondaryNameNode
1608 NameNode
2016 ResourceManager
2301 NodeManager
2331 Jps
1725 DataNode