【Hadoop十五】Hadoop Counter

最新推荐文章于 2024-07-09 11:29:48 发布

axxbc123

最新推荐文章于 2024-07-09 11:29:48 发布

阅读量278

点赞数

分类专栏： Hadoop 文章标签：大数据

本文链接：https://blog.csdn.net/axxbc123/article/details/84721526

版权

本文详细探讨了Hadoop Counters在MapReduce作业中的应用，包括只包含Map任务的作业和同时包含Map与Reduce任务的作业。通过实例解析Counter如何在不同阶段跟踪和统计关键指标。

摘要由CSDN通过智能技术生成

1. 只有Map任务的Map Reduce Job

	File System Counters
		FILE: Number of bytes read=3629530
		FILE: Number of bytes written=98312
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=8570654
		HDFS: Number of bytes written=1404469
		HDFS: Number of read operations=6
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters 
		Launched map tasks=1
		Data-local map tasks=1
		Total time spent by all maps in occupied slots (ms)=14522
		Total time spent by all reduces in occupied slots (ms)=0
		Total time spent by all map tasks (ms)=14522
		Total vcore-seconds taken by all map tasks=14522
		Total megabyte-seconds taken by all map tasks=14870528
	Map-Reduce Framework
		Map input records&#