软件环境 | |
---|---|
操作系统 | CentOS 6.4 64bit (Basic Server + 桌面环境) |
虚拟机 | VMware Workstation 12.0 |
JDK | 1.7 |
1 安装Hadoop-2.6.0
1.1 下载
wget http://mirrors.hust.edu.cn/apache/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz
1.2 解压并安装
mkdir local
mkdir local/opt
tar -zxf hadoop-2.6.0.tar.gz -C ~/local/opt
1.3 配置环境变量
vim ~/.bashrc
末尾追加:
export HADOOP_PREFIX=$HOME/local/opt/hadoop-2.6.0
export HADOOP_COMMON_HOME=$HADOOP_PREFIX
export HADOOP_HDFS_HOME=$HADOOP_PREFIX
export HADOOP_MAPRED_HOME=$HADOOP_PREFIX
export HADOOP_YARN_HOME=$HADOOP_PREFIX
export HADOOP_CONF_DIR=$HADOOP_PREFIX/etc/hadoop
export PATH=$PATH:$HADOOP_PREFIX/bin:$HADOOP_PREFIX/sbin
2 文件配置(共6个)
2.1 hadoop-env.sh
export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.101.x86_64
2.2 core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/local/var/hadoop/tmp/hadoop-${user.name}</value>
</property>
</configuration>
2.3 hdfs-site.xml
<configuration>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///home/hadoop/local/var/hadoop/hdfs/datanode</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///home/hadoop/local/var/hadoop/hdfs/namenode</value>
</property>
<property>
<name>dfs.namenode.checkpoint.dir</name>
<value>file:///home/hadoop/local/var/hadoop/hdfs/namesecondry</value>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
</property>
</configuration>
2.4 yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce.shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>master</value>
</property>
</configuration>
2.5 mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.jobtracker.staging.root.dir</name>
<value>/user</value>
</property>
</configuration>
2.6 slaves
master
slave1
slave2
3 64位Native Library替换
3.1 下载64位Native Library
wget http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.6.0.tar
3.2 解压Native Library,并覆盖hadoop-2.6.0/lib/native中文件
4 启动Hadoop
4.1 格式化HDFS
hdfs namenode -format
4.2 启动Hadoop集群
拷贝local文件到slave1和slave2并各自进行.bashrc配置
start-dfs.sh
start-yarn.sh
4.3 验证
jps
如果一切顺利,将可以看到:
master:
xxxx NameNode
xxxx DataNode
xxxx SecondaryNameNode
xxxx Jps
slave1:
xxxx DataNode
xxxx Jps
slave2:
xxxx DataNode
xxxx Jps