java大数据demo_MapReduce Demo详解

17/11/08 11:34:55 INFO mapreduce.Job: Task Id : attempt_1509957441313_0002_m_000000_0, Status : FAILED Error: java.io.IOException: Initialization of all the collectors failed. Error in last collect or was :class com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$Text at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:415) at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81) at org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:698) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746

以上错误可看出hadoop引用的Text包出错，需要将DataCount类中Text的包引用改为 import org.apache.hadoop.io.Text;

再次运行：

[[email protected] bin]# hadoop jar /home/hadoop/mpCount.jar cn.terry.mr.DataCount /data3.txt /MROut4

17/11/08 16:23:45 INFO client.RMProxy: Connecting to ResourceManager at master/x.x.x.x:8032

17/11/08 16:23:46 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.

17/11/08 16:23:47 INFO input.FileInputFormat: Total input paths to process : 1

17/11/08 16:23:47 INFO mapreduce.JobSubmitter: number of splits:1

17/11/08 16:23:47 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1509957441313_0008

17/11/08 16:23:48 INFO impl.YarnClientImpl: Submitted application application_1509957441313_0008

17/11/08 16:23:48 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1509957441313_0008/

17/11/08 16:23:48 INFO mapreduce.Job: Running job: job_1509957441313_0008

17/11/08 16:24:02 INFO mapreduce.Job: Job job_1509957441313_0008 running in uber mode : false

17/11/08 16:24:02 INFO mapreduce.Job:  map 0% reduce 0%

17/11/08 16:24:14 INFO mapreduce.Job:  map 100% reduce 0%

17/11/08 16:24:25 INFO mapreduce.Job:  map 100% reduce 100%

17/11/08 16:24:26 INFO mapreduce.Job: Job job_1509957441313_0008 completed successfully

查看结果：

[[email protected] bin]# hdfs dfs -ls /MROut4

Found 2 items

-rw-r--r--   2 root supergroup          0 2017-11-08 16:24 /MROut4/_SUCCESS

-rw-r--r--   2 root supergroup        106 2017-11-08 16:24 /MROut4/part-r-00000

[[email protected] bin]# hdfs dfs -cat /MROut4/part-r-00000

13112345678     1800    400     2200

13512345678     9500    400     9900

13612345678     8000    4000    12000

13812345678     3500    400     3900

由于我的chrome和IE版本无法兼容cnblogs的插入code和picture功能，抱歉没能将代码及结果以友好的方式呈现。