![](https://img-blog.csdnimg.cn/20201014180756780.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
Hadoop源码分析
文章平均质量分 62
maixia24
大数据、机器学习、区块链
展开
-
Hadoop源码分析DataNode类
DataNode类源码分析DataNode 类存储,DFS部署所需要的blocks。一个部署需要一个或多个DataNodes。每一个DataNode和一个单独的NameNode定期的交流。DataNode同样也不时地,和客户端代码和其它的DataNodes交流。DataNodes存储一些命名的blocks。DataNode允许客户端代码来读这些块,或者写进新的block原创 2013-07-27 00:27:17 · 1216 阅读 · 0 评论 -
inputSplit.java分析
/** * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding原创 2013-12-03 15:52:45 · 1192 阅读 · 0 评论 -
hadoop源码分析 jobsplit
ClusterMetrics.java类: Status information on the current state of the Map-Reduce cluster. * Mapreduce集群的当前状态 * 提供给client如下信息: * 集群大小 * 黑名单和过期的tracker * 集群的容量 * 当前Map reduce占用 * 当前运行的map原创 2013-12-03 15:51:43 · 791 阅读 · 0 评论 -
jobcontrol类
* 这个类包括一系列Mapreduce jobs和它的依赖 * 它通过根据状态 把job放到不同的表里面,跟踪jobs的状态, * 当一个job被添加进来 一个组的唯一标示ID 被分配给这个job。 * * 这个类有一个线程,当job准备好的时候,提交job,监控运行的job的状态,并且依据他们的依赖job状态,更新job的状态。 * 这个类提供挂起 恢复 停止线原创 2013-12-03 13:03:10 · 1085 阅读 · 0 评论 -
Hadoop源码分析之Id.java
ID.java 接口, implements WritableComparable 是 jobId taskId TaskAttemptID的基类JobId.java * jobid是job的唯一不变的代表。jobid包括两部分,第一部分代表jobtracker的唯一,对于集群安装来说,这个字符串是 * jobtracker的开始时间 对于本地设置来说,他就是本地原创 2013-12-03 15:46:10 · 599 阅读 · 0 评论 -
JobContext源码分析
/** * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding原创 2013-12-01 16:30:44 · 2642 阅读 · 0 评论 -
Job源码分析
/** * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding原创 2013-12-01 17:12:18 · 2476 阅读 · 0 评论 -
hadoop job.class 源码分析
waitForCompletion()方法 里面,提交作业给集群,并且等待作业完成 /** * Submit the job to the cluster and wait for it to finish. * @param verbose print the progress to the user * @return true if the job succeeded原创 2013-11-28 23:03:44 · 833 阅读 · 0 评论 -
DistributedCache源码分析
/** * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding原创 2013-11-14 15:06:44 · 1879 阅读 · 0 评论 -
Mapper reducer 的生命周期
/** * Called once at the start of the task.只在任务开始的时候 运行一次 */ protected void setup(Context context ) throws IOException, InterruptedException { // NOTHING } /*原创 2013-11-26 16:03:09 · 872 阅读 · 0 评论 -
MultipleInputs源码分析
/** * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed with this work for additional information * regarding原创 2013-11-25 14:22:40 · 1332 阅读 · 0 评论 -
DBInputFormat的key value 格式
DBInputFormat emits LongWritables containing the record number as key and DBWritables as value. DBInputFormat key 类型 LongWritables record numbervalue类型:DBWritables原创 2013-11-25 13:56:38 · 804 阅读 · 0 评论 -
Hbase源码分析 RowCounter
/** * Copyright 2008 The Apache Software Foundation * * Licensed to the Apache Software Foundation (ASF) under one * or more contributor license agreements. See the NOTICE file * distributed wit原创 2013-11-04 18:18:53 · 3055 阅读 · 0 评论 -
Hadoop源码分析
http://blog.csdn.net/shirdrn/article/details/4569702 地址原创 2014-01-26 14:09:37 · 777 阅读 · 1 评论