Hadoop源码学习

Hadoop下载地址：
https://github.com/wangqiaowqo/hadoop-common

其中hadoop-core对应的工程为：
hadoop-common-trunk.zip\hadoop-common-trunk\hadoop-mapreduce-project\hadoop-mapreduce-client\hadoop-mapreduce-client-core\src\main\java\org\apache\hadoop\mapred

hadoop deg 设置：
vi hadoop-env.sh
#export HADOOP_NAMENODE_OPTS="-Dcom.sun.management.jmxremote $HADOOP_NAMENODE_OPTS"
export HADOOP_NAMENODE_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8788,server=y,suspend=y"

#HADOOP_SECONDARYNAMENODE_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8789,server=y,suspend=y"

#HADOOP_DATANODE_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8790,server=y,suspend=y"

#HADOOP_BALANCER_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8791,server=y,suspend=y"

#HADOOP_JOBTRACKER_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8792,server=y,suspend=y"

#HADOOP_TASKTRACKER_OPTS="-Xdebug -Xrunjdwp:transport=dt_socket,address=8793,server=y,suspend=y"

vi mapred-site.xml
<property>
<name>mapred.child.java.opts</name>
<value>-agentlib:jdwp=transport=dt_socket,address=8883,server=y,suspend=y</value>
</property>

/opt/hadoop/bin/stop-all.sh

/opt/hadoop/bin/start-all.sh
starting namenode, logging to /opt/hadoop/libexec/../logs/hadoop-hadoop-namenode-Master.Hadoop.out
Listening for transport dt_socket at address: 8788

参考：http://blog.csdn.net/huyu_hy/article/details/7857578

下载 hadoop-eclipse-plugin-1.1.2.jar插件 放入D:\eclipse\plugins 中
重启eclipse

修改 /etc/hosts
hadoop下的配置信息中localhost更改为10.1.77.23

#127.0.0.1 Master.Hadoop
10.1.77.213 Master.Hadoop

在Map/Reduce 中新增 MapReduce locations
如下：
Map/Reduce Master
host:10.1.77.213
port:54311

DFS Master
host:10.1.77.213
port:54310

在左侧 Project Explorer中会看的 DFS Locations