python处理文本文件 提取英文单词看成一个整体_如何在python中将文本文件拆分为单词？...

没有人建议发电机,我很惊讶.这是我将如何做到这一点：

def words(stringIterable):

#upcast the argument to an iterator, if it's an iterator already, it stays the same

lineStream = iter(stringIterable)

for line in lineStream: #enumerate the lines

for word in line.split(): #further break them down

yield word

现在,这可以用在你可能已经在内存中的简单句子列表中：

listOfLines = ['hi there', 'how are you']

for word in words(listOfLines):

print(word)

但它在文件上也可以正常工作,而无需在内存中读取整个文件：

with open('words.py', 'r') as myself:

for word in words(myself):

print(word)