python的mapreduce_如何在python中实现mapreduce对模式

我尝试在python中尝试mapreduce pairs模式。需要检查一个单词是否在文本文件中,然后找到它旁边的单词并生成两个单词对。继续遇到:neighbors = words[words.index(w) + 1]

ValueError: substring not found

或者

^{pr2}$

文件工作_试用版.py在from mrjob.job import MRJob

class MRCountest(MRJob):

# Word count

def mapper(self, _, document):

# Assume document is a list of words.

#words = []

words = document.strip()

w = "the"

neighbors = words.index(w)

for word in words:

#searchword = "the"

#wor.append(str(word))

#neighbors = words[words.index(w) + 1]

yield(w,1)

def reducer(self, w, values):

yield(w,sum(values))

if __name__ == '__main__':

MRCountest.run()

编辑:

尝试使用pairs模式在文档中搜索特定单词的每个实例,然后每次都找到它旁边的单词。然后为每个实例生成一对结果,即查找“the”的实例及其旁边的单词,即[the]、[book]、[the]、[cat]等from mrjob.job import MRJob

class MRCountest(MRJob):

# Word count

def mapper(self, _, document):

# Assume document is a list of words.

#words = []

words = document.split(" ")

want = "the"

for w, want in enumerate(words, 1):

if (w+1) < len(words):

neighbors = words[w + 1]

pair = (want, neighbors)

for u in neighbors:

if want is "the":

#pair = (want, neighbors)

yield(pair),1

#neighbors = words.index(w)

#for word in words:

#searchword = "the"

#wor.append(str(word))

#neighbors = words[words.index(w) + 1]

#yield(w,1)

#def reducer(self, w, values):

#yield(w,sum(values))

if __name__ == '__main__':

MRCountest.run()

就目前的情况来看,我得到了每一个词对的产出率是同一对的倍数。在

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值