本文基于实验室已经搭建好的Hadoop平台。
参考http://www.michael-noll.com/tutorials/writing-an-hadoop-mapreduce-program-in-python/
1.编写mapper.py
#!/usr/bin/python2.6 import sys for line in sys.stdin: line = line.strip() words = line.split() for word in words: print '%s %s' % (word, 1)
2.编写reducer.py
#!/usr/bin/python import sys from operator import itemgetter current_word = None current_count = 0 word = None for line in sys.stdin: line = line.strip() word, count = line.split(' ', 1) try: count = int(count) except ValueError: continue if current_word == word: current_count += count else: if current_word: print '%s %s' % (current_word, c