一、Ubuntu14.04下安装Hadoop2.4.0(单机模式)
http://www.cnblogs.com/kinglau/p/3794433.html
二、Ubuntu14.04下安装Hadoop2.4.0(伪分布模式)
http://www.cnblogs.com/kinglau/p/3796164.html
http://www.cnblogs.com/kinglau/p/3802705.html
四、在Ubuntu中使用eclipse连接HDFS配置开发环境及出现问题解决方案
http://www.aboutyun.com/thread-8029-1-1.html
五、Hadoop源码编译,hadoop-2.4.1+Lubuntu(Ubuntu)
http://blog.csdn.net/xinsuixiaofeiyu/article/details/38439195
http://www.cnblogs.com/wrencai/p/3897438.html
bash 腳本(該腳本來源於網絡):
#移除所有失敗的下載
find ~/.m2 -name"*.lastUpdated" -exec grep -q "Could not transfer" {} \;-print -exec rm {} \;
以上在Ubuntu 12.04 下測試通過。
Maven的HTTP代理设置(如果用代理上网)
http://blog.sina.com.cn/s/blog_4f925fc30102ed3y.html
六、生成工程文件导入Eclipse
$ cd ${HADOOP_HOME} $ mvn eclipse:eclipse -DskipTests 最后在Eclipse中按照以下流程导入源代码:“File” → “Import”→ “Existing Projects into Workspace”
Error#1:hadoop-streaming里面的build path有问题,显示/root/workspace/hadoop-2.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf(missing)
解决办法,remove掉引用就好。
Error#2:Eclipse里的project报Unboundclasspath variable: 'M2_REPO/**/***/***.jar'
但是实际上这个jar是存在于我本地机器上MAVEN的目录里:
\home\hadoop\.m2\repository\**\***\***.jar
说明M2_REPO这个参数Eclipse不认得,需要给它添上:
Eclipse->Windows->Preferences->java->BuildPath->Classpath Variables
new一个,名字是M2_REPO,属性选folder,指向\home\hadoop\.m2\repository
Error#3: /hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/XmlEditsVisitor.java里面显示
importcom.sun.org.apache.xml.internal.serialize.OutputFormat;
import com.sun.org.apache.xml.internal.serialize.XMLSerializer;
失败,这是由于Eclipse的强检查原则,打开Java-> Compiler -> Errors/Warnings and under "Deprecated and restrictedAPI" change the setting of "Forbidden reference (access rules)" 将error级别调整到warning级别就好。
Error#4: /hadoop-common/src/test/java/org/apache/hadoop/io/serializer/avro/TestAvroSerialization.java显示没有AvroRecord类,在网上搜索到AvroRecord类放入到同级包中就行了。 http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-common/2.2.0/org/apache/hadoop/io/serializer/avro/AvroRecord.java#AvroRecord
Error#5: org.apache.hadoop.ipc.protobuf包是空的,需要在/hadoop-common/target/generated-sources/java中找到profobuf拷贝到/hadoop-common/src/test/java中就好了.同时包里面还缺少了以下三个引用,在GrepCode上找一下,把hadoop-common2.x的相应文件下下来导入。
org.apache.hadoop.ipc.protobuf.TestProtos.EchoRequestProto;
org.apache.hadoop.ipc.protobuf.TestProtos.EchoResponseProto;
org.apache.hadoop.ipc.protobuf.TestRpcServiceProtos.TestProtobufRpcProto;
实际上只需下载TestProtos.java和TestRpcServiceProtos.java两个文件放到profobuf下。
参照: