MapReduce
关注
分享
扫一扫
coder.yang
这个作者很懒，什么都没留下…
展开
-
MapReduce Demo
MainClasspackage com.bjsxt.mr.wordcount;import java.io.IOException;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.Path;import org.apache.hadoop.io.LongWritable;import ...原创 2019-03-25 11:09:03 · 232 阅读 · 0 评论 -
Mapper源码
public class MyMapper2 extends Mapper<LongWritable, Text,Text,LongWritable> { @Override protected void map(LongWritable key, Text value, Context context) throws IOException, Interrupted...原创 2019-03-25 11:43:38 · 361 阅读 · 0 评论 -
Reducer源码学习
import org.apache.hadoop.io.LongWritable;import org.apache.hadoop.io.Text;import org.apache.hadoop.mapreduce.Reducer;import java.io.IOException;public class MyReducer2 extends Reducer<Text, L...原创 2019-03-25 13:14:05 · 227 阅读 · 0 评论 -
Job Class
Set the Jar by finding where a given class came from.@param cls the example class public void setJarByClass(Class<?> cls) { ensureState(JobState.DEFINE); conf.setJarByClass(cls); ...原创 2019-03-25 13:53:08 · 408 阅读 · 0 评论 -
Partitioner 源码学习
Partitions the key space.Partitioner controls the partitioning of the keys of the intermediate map-outputs. The key (or a subset of the key) is used to derivethe partition, typically by a hash func...原创 2019-03-25 15:43:05 · 171 阅读 · 0 评论 -
hadoop 任务提交源码分析
原创 2019-04-06 17:08:12 · 186 阅读 · 0 评论 -
MapTask.java
@SuppressWarnings("unchecked") private <INKEY,INVALUE,OUTKEY,OUTVALUE> void runNewMapper(final JobConf job, final TaskSplitIndex splitIndex, final T...原创 2019-03-28 16:22:42 · 187 阅读 · 0 评论 -
Mr源码分析
Job.class@InterfaceAudience.Public@InterfaceStability.Evolvingpublic class Job extends JobContextImpl implements JobContext { 省略1000多行代码。。。 public boolean waitForCompletion(boolean verbose ) ...原创 2019-03-28 23:04:04 · 329 阅读 · 0 评论