Python爬虫——简单爬取豆瓣图书top250

最新推荐文章于 2024-06-18 11:18:45 发布

XIANGYUEDAYUCHEN

最新推荐文章于 2024-06-18 11:18:45 发布

阅读量319

点赞数

文章标签： Python 爬虫

本文链接：https://blog.csdn.net/weixin_45891711/article/details/103470615

版权

import requests
from bs4 import BeautifulSoup
import lxml
import re
headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36'}
url='https://book.douban.com/top250?start=0'
for i in range(0,225,25):
    url='https://book.douban.com/top250?start='+str(i)
    res=requests.get(url=url,headers=headers)
    res.encoding='utf-8'
    soup=BeautifulSoup(res.text,'lxml')      #创建一个beautifulSoup对
    for i in range(2,52,2):
        book=soup.select('div > table:nth-child('+str(i)+')')
        for i in book:
            rex =re.findall('<a.*?title="(.*?)"',str(i))
            print((rex))
            bb=i.find('span')
            print(bb.string,end=' ')
            bb1=i.find('p')
            print(bb1.string)
            with open('d:/2019/py-pro/book1.txt','a',encoding='utf-8') as f:
                f.write(rex[0]+'  ')
                f.write(str(bb.string)+'  ')
                f.write(str(bb1.string)+'\n')