[Hadoop]MapReduce中的Partitioner与Combiner

Partitioners负责划分Maper输出的中间键值对的key，分配中间键值对到不同的Reducer。Maper输出的中间结果交给指定的Partitioner，确保中间结果分发到指定的Reduce任务。在每个Reducer中，键按排序顺序处理（Within each reducer, keys are processed in sorted order）。Combiners是MapReduce中的一个优化，允许在shuffle和排序阶段之前在本地进行聚合。Combiners的首要目标是通过最小化键值对的数量来节省尽可能多的带宽，键值对将通过网络在mappers和reducers之间进行shuffle操作（The primary goal of combiners is to save as much bandwidth as possible by minimizing the number of key/value pairs that will be shuffled across the network between mappers and reducers）。我们可以把Combiners理解为发生在shuffle和sort阶段之前，对Mapper输出进行操作的"mini-reducers"。每个Combiner单独操作于一个Mapper，因此不能访