Hadoop
lming_08
这个作者很懒,什么都没留下…
展开
-
用eclipse编写MapReduce程序的基本要点
1.要想在eclipse上编写MapReduce,那么就需要在eclipse上安装hadoop插件,具体操作是将hadoop安装目录下的contrib/eclipse-plugin/hadoop-0.20.2-eclipse-plugin.jar复制到eclipse安装目录里的plugins目录中。2.插件安装完成之后,就可以新建Map/Reduce Project了,在java文件中需要导入原创 2013-12-28 16:12:25 · 3361 阅读 · 0 评论 -
安装与配置Hadoop出现的常见问题
1.当格式化HDFS后,执行bin/hadoop fs -ls /,出现$ bin/hadoop fs -ls /13/12/28 11:38:22 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:8888. Already tried 0 time(s).13/12/28 11:38:24 INFO i原创 2013-12-28 11:55:35 · 861 阅读 · 0 评论 -
关于Hadoop-Streaming学习中碰到的问题
Hadoop在分布式计算方面很强大,而Python在文本处理也是相当方便,那么有这两者的结合吗?有,答案就是Hadoop-Streaming。Hadoop-Streaming可以将Hadoop与主流语言结合起来,使用方便,效果很好。个人觉得Pig在处理数据集时很不方便,特别是在计算百分比等运算时,而Hadoop-Streaming是可以替代Pig的。1.Streaming固定的代码,该代码可以...原创 2014-07-28 20:02:59 · 10883 阅读 · 0 评论 -
Diagnostics: Exception from container-launch
16/05/09 11:02:14 INFO mapreduce.Job: Job job_1459494184870_43303 failed with state FAILED due to: Application application_1459494184870_43303 failed 1 times due to AM Container for appattempt_14594941原创 2016-05-09 11:32:06 · 11367 阅读 · 3 评论 -
running beyond physical memory limits
6/07/19 14:10:30 INFO mapreduce.Job: Task Id : attempt_1459504722049_479821_m_000041_2, Status : FAILED Container [pid=167394,containerID=container_1459504722049_479821_01_000180] is running beyond ph原创 2016-07-19 16:04:33 · 4802 阅读 · 0 评论