Python 进行 hadoop MapReduce操作

一般可以在linux下可以进行测试操作

cat inputfile | python mapper.py | sort | python reducer.py > outputfile 

单词统计 WordCount

制作一个mapReduce操作需要一个map.py 和 reduce.py

map.py

import sys

for line in sys.stdin:
	line_words = line.strip().split(' ')
	for word in line_words:
		print '%s\t1'%(word.strip())

reduce.py

import sys

cur_word = None
sum = 0

for line in sys.stdin:
	word,count = line.strip().split('\t')
	try:
		count = int(count