在古诗文网下载论语全文
url
https://so.gushiwen.cn/guwen/book_46653FD803893E4F75696240258265D2.aspx
请求
headers = { "User-Agent": "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Mobile Safari/537.36 Edg/96.0.1054.29"} url = 'https://so.gushiwen.cn/guwen/book_46653FD803893E4F75696240258265D2.aspx' page_text= requests.get(url=url,headers=headers).text
实例化BeautifulSoup对象
soup = BeautifulSoup(page_text,'lxml') span_list = soup.select('.bookcont > ul > span')
创造论语txt文件
fp=open('论语.txt','w',encoding= 'utf-8')
对应文章详情页的url
for span in span_list: title = span.a.string zj_url = span.a['href']
对详情页发起请求,对章节内容进行解析
zj_url_text = requests.get(url=zj_url,headers=headers).tex