MapReduce的学习体会

建立map task reduce task 持久到file system

一个(master)JobTracker 以及每个执行节点的TaskTracker
The master is responsible for scheduling the jobs' component tasks on the slaves, monitoring them and re-executing the failed tasks. The slaves execute the tasks as directed by the master.


可配置(configuration) 可写(Writable) 可写且可比较(WritableComparable)

(input) <k1, v1> -> map -> <k2, v2> -> combine* -> <k2, v2> -> reduce -> <k3, v3> (output)


如何加载数据到应用上下文中
应用，允许加载jar到classpath
Applications can specify a comma-separated list of paths which would be present in the current working directory of the task using the option -files. The -libjars option allows applications to add jars to the classpaths of the maps and reduces. The option -archives allows them to pass comma separated list of archives as arguments.