hadoop jar singlnode下运行的命令行输出


[hadoop@aaa ~]$ hadoop jar /opt/workspace/PGMdriver.jar GibbsSamplingDriver /output
14/05/22 20:39:13 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/05/22 20:39:13 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
14/05/22 20:39:14 INFO mapreduce.JobSubmitter: number of splits:2
14/05/22 20:39:14 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
14/05/22 20:39:14 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
14/05/22 20:39:14 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
14/05/22 20:39:14 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
14/05/22 20:39:14 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1400777095877_0003
14/05/22 20:39:15 INFO impl.YarnClientImpl: Submitted application application_1400777095877_0003 to ResourceManager at /0.0.0.0:8032
14/05/22 20:39:15 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1400777095877_0003/
14/05/22 20:39:15 INFO mapreduce.Job: Running job: job_1400777095877_0003
14/05/22 20:39:22 INFO mapreduce.Job: Job job_1400777095877_0003 running in uber mode : false
14/05/22 20:39:22 INFO mapreduce.Job:  map 0% reduce 0%
14/05/22 20:39:30 INFO mapreduce.Job:  map 100% reduce 0%
14/05/22 20:39:37 INFO mapreduce.Job:  map 100% reduce 100%
14/05/22 20:39:38 INFO mapreduce.Job: Job job_1400777095877_0003 completed successfully
14/05/22 20:39:38 INFO mapreduce.Job: Counters: 43
	File System Counters
		FILE: Number of bytes read=34651
		FILE: Number of bytes written=308564
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=40
		HDFS: Number of bytes written=33845
		HDFS: Number of read operations=7
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters 
		Launched map tasks=2
		Launched reduce tasks=1
		Other local map tasks=2
		Total time spent by all maps in occupied slots (ms)=10975
		Total time spent by all reduces in occupied slots (ms)=4275
	Map-Reduce Framework
		Map input records=0
		Map output records=200
		Map output bytes=34045
		Map output materialized bytes=34657
		Input split bytes=40
		Combine input records=0
		Combine output records=0
		Reduce input groups=176
		Reduce shuffle bytes=34657
		Reduce input records=200
		Reduce output records=200
		Spilled Records=400
		Shuffled Maps =2
		Failed Shuffles=0
		Merged Map outputs=2
		GC time elapsed (ms)=201
		CPU time spent (ms)=2410
		Physical memory (bytes) snapshot=662515712
		Virtual memory (bytes) snapshot=2658840576
		Total committed heap usage (bytes)=512229376
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=33845

以上位运行成功

下面换了个jar,运行没错,但是结果不对,貌似没读到hdfs里的文件,命令行显示结果如下:

[hadoop@aaa~]$ hadoop jar /opt/workspace/PGM2205.jar /output
14/05/22 21:23:10 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/05/22 21:23:11 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
14/05/22 21:23:11 INFO mapreduce.JobSubmitter: number of splits:3
14/05/22 21:23:11 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files.filesizes is deprecated. Instead, use mapreduce.job.cache.files.filesizes
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files is deprecated. Instead, use mapreduce.job.cache.files
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.mapoutput.value.class is deprecated. Instead, use mapreduce.map.output.value.class
14/05/22 21:23:11 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
14/05/22 21:23:11 INFO Configuration.deprecation: mapreduce.inputformat.class is deprecated. Instead, use mapreduce.job.inputformat.class
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.cache.files.timestamps is deprecated. Instead, use mapreduce.job.cache.files.timestamps
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.mapoutput.key.class is deprecated. Instead, use mapreduce.map.output.key.class
14/05/22 21:23:11 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
14/05/22 21:23:12 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1400777095877_0009
14/05/22 21:23:12 INFO impl.YarnClientImpl: Submitted application application_1400777095877_0009 to ResourceManager at /0.0.0.0:8032
14/05/22 21:23:12 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1400777095877_0009/
14/05/22 21:23:12 INFO mapreduce.Job: Running job: job_1400777095877_0009
14/05/22 21:23:18 INFO mapreduce.Job: Job job_1400777095877_0009 running in uber mode : false
14/05/22 21:23:18 INFO mapreduce.Job:  map 0% reduce 0%
14/05/22 21:23:27 INFO mapreduce.Job:  map 33% reduce 0%
14/05/22 21:23:29 INFO mapreduce.Job:  map 100% reduce 0%
14/05/22 21:23:34 INFO mapreduce.Job:  map 100% reduce 100%
14/05/22 21:23:34 INFO mapreduce.Job: Job job_1400777095877_0009 completed successfully
14/05/22 21:23:34 INFO mapreduce.Job: Counters: 43
	File System Counters
		FILE: Number of bytes read=6
		FILE: Number of bytes written=318973
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=60
		HDFS: Number of bytes written=0
		HDFS: Number of read operations=9
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters 
		Launched map tasks=3
		Launched reduce tasks=1
		Other local map tasks=3
		Total time spent by all maps in occupied slots (ms)=21498
		Total time spent by all reduces in occupied slots (ms)=3932
	Map-Reduce Framework
		Map input records=0
		Map output records=0
		Map output bytes=0
		Map output materialized bytes=18
		Input split bytes=60
		Combine input records=0
		Combine output records=0
		Reduce input groups=0
		Reduce shuffle bytes=18
		Reduce input records=0
		Reduce output records=0
		Spilled Records=0
		Shuffled Maps =3
		Failed Shuffles=0
		Merged Map outputs=3
		GC time elapsed (ms)=143
		CPU time spent (ms)=2560
		Physical memory (bytes) snapshot=946880512
		Virtual memory (bytes) snapshot=3543068672
		Total committed heap usage (bytes)=717225984
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=0
为什么呢?

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值