环境: win10 x64、centos7.2、hadoop2.4.1、jdk1.8
报错:执行 hadoop jar wc.jar WCRunner
[yx@bigdata00 ~]$ hadoop jar wc.jar WCRunner
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/yx/app/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
19/03/04 18:47:50 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/03/04 18:47:51 INFO client.RMProxy: Connecting to ResourceManager at bigdata00/192.168.2.100:8032
19/03/04 18:47:51 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
19/03/04 18:47:51 WARN mapreduce.JobSubmitter: No job jar file set. User classes may not be found. See Job or Job#setJar(String).
19/03/04 18:47:51 INFO input.FileInputFormat: Total input paths to process : 1
19/03/04 18:47:51 INFO mapreduce.JobSubmitter: number of splits:1
19/03/04 18:47:52 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1551674462324_0009
19/03/04 18:47:52 INFO mapred.YARNRunner: Job jar is not present. Not adding any jar to the list of resources.
19/03/04 18:47:52 INFO impl.YarnClientImpl: Submitted application application_1551674462324_0009
19/03/04 18:47:52 INFO mapreduce.Job: The url to track the job: http://bigdata00:8088/proxy/application_1551674462324_0009/
19/03/04 18:47:52 INFO mapreduce.Job: Running job: job_1551674462324_0009
19/03/04 18:47:58 INFO mapreduce.Job: Job job_1551674462324_0009 running in uber mode : false
19/03/04 18:47:58 INFO mapreduce.Job: map 0% reduce 0%
19/03/04 18:48:01 INFO mapreduce.Job: Task Id : attempt_1551674462324_0009_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WCMapper not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class WCMapper not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
... 8 more
19/03/04 18:48:06 INFO mapreduce.Job: Task Id : attempt_1551674462324_0009_m_000000_1, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WCMapper not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class WCMapper not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
... 8 more
19/03/04 18:48:10 INFO mapreduce.Job: Task Id : attempt_1551674462324_0009_m_000000_2, Status : FAILED
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WCMapper not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1895)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:186)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:722)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1556)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.lang.ClassNotFoundException: Class WCMapper not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1801)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1893)
... 8 more
19/03/04 18:48:16 INFO mapreduce.Job: map 100% reduce 100%
19/03/04 18:48:16 INFO mapreduce.Job: Job job_1551674462324_0009 failed with state FAILED due to: Task failed task_1551674462324_0009_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0
19/03/04 18:48:16 INFO mapreduce.Job: Counters: 9
Job Counters
Failed map tasks=4
Launched map tasks=4
Other local map tasks=3
Rack-local map tasks=1
Total time spent by all maps in occupied slots (ms)=11765
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=11765
Total vcore-seconds taken by all map tasks=11765
Total megabyte-seconds taken by all map tasks=12047360
[yx@bigdata00 ~]$ hadoop fs -rmr /wc/output
rmr: DEPRECATED: Please use 'rm -r' instead.
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/yx/app/hadoop-2.4.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
19/03/04 18:53:54 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
19/03/04 18:53:55 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval = 0 minutes, Emptier interval = 0 minutes.
Deleted /wc/output