python爬取指定多个网页数据_Python爬取网站，前几个有数据，之后返回None？

想获取廖雪峰python教程网站的内容练练手，发现有的章节能返回数据，但到Python基础这一章开始返回的都是None，没明白问题出在哪，求教

错误如下：

Traceback (most recent call last):

File "scraping_the_tutorial.py", line 36, in

get_urls()

File "scraping_the_tutorial.py", line 34, in get_urls

parse_url_to_html(a.attrs["href"],a.get_text())

File "scraping_the_tutorial.py", line 23, in parse_url_to_html

content = content.get_text()

AttributeError: 'NoneType' object has no attribute 'get_text'

代码如下：

import requests

from bs4 import BeautifulSoup

# session对象

session = requests.Session()

# 用于表明自己是何浏览器以及指定返回数据格式

headers = {"User-Agent":"Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 \

(KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36",

"Accept":"text/html,application/xhtml+xml,application/xml;\

q=0.9,image/webp,*/*;q=0.8"}

# 要获取数据的网站

def parse_url_to_html(url,file_name):

# 通过get(url,headers)函数来接受返回的数据

req = session.get("https://www.liaoxuefeng.com"+url, headers=headers)

# 用bs解析便于处理数据,第二个参数是何种解析方法

bsObj = BeautifulSoup(req.text,"html.parser")

# 获取教程内容，在class为x-wiki-content x-main-content里

content = bsObj.find("div",{"class":"x-content"})

# 用str不用getText是因为有时候有None值

# 上面错了，是因为服务器反爬机制返回HTTP error503

print(content)

content = content.get_text()

# 存到txt文件夹中

with open("txt/%s.txt" % file_name,'w') as f:

f.write(content)

def get_urls():

html = session.get("https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000", \

headers=headers)

bsObj = BeautifulSoup(html.text,"html.parser")

directory = bsObj.find("ul",{"id":"x-wiki-index"})

for a in directory.findAll("a",{"class":"x-wiki-index-item"}):

parse_url_to_html(a.attrs["href"],a.get_text())

if __name__ == "__main__":

get_urls()