scrapy获取html标签文本,如何使用Scrapy从网站上获取所有纯文本？

小编典典

最简单的选择是to 并且找到所有内容：extract //body//text()join

''.join(sel.select("//body//text()").extract()).strip()

这里sel是一个Selector实例。

另一种选择是使用nltk的clean_html()：

>>> import nltk

>>> html = """

...

...

...

I would like to have all the text visible from a website, after the HTML is rendered. I'm working in Python with Scrapy framework.

... With xpath('//body//text()') I'm able to get it, but with the HTML tags, and I only want the text. Any solution for this? Thanks !

...

...

"""

>>> nltk.clean_html(html)

"I would like to have all the text visible from a website, after the HTML is rendered. I'm working in Python with Scrapy framework.\nWith xpath('//body//text()&