python爬虫爬取百度文档

1.获取文档内容，百度文库的文档大多是图片形式

2.导入数据模块请求数据

浏览器中搜索找到图片的来源

这里面就有所需的所有图片内容

3.将图片文件爬取下来

import requests

url = 'https://wenku.baidu.com/gsearch/rec/pcviewdocrec2023?sessionId=0313150659-0324210646--&docId=9f874fd5b3717fd5360cba1aa8114431b90d8ebf&query=%E5%9F%BA%E4%BA%8E%E8%AF%AD%E4%B9%89%E5%88%86%E6%9E%90%E6%8A%80%E6%9C%AF%E7%9A%84%E6%83%85%E6%84%9F%E5%88%86%E6%9E%90%E7%AE%97%E6%B3%95%E7%A0%94%E7%A9%B6&recPositions=catalog%2Ctoplist'
headers={
        'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0'
    }
data = {
    'sessionId': '0313150659-0324210646--',
    'docId': '9f874fd5b3717fd5360cba1aa8114431b90d8ebf',
    'query': '基于语义分析技术的情感分析算法研究',
    'recPositions': 'catalog,toplist'
}
response = requests.get(url=url,params=data,headers=headers)
num = 1
for index in response.json()['data']['catalogDoc']:
    pic = index['pic']
    img_content = requests.get(url=pic,headers=headers).content
    with open('img22\\' + str(num) + '.jpg',mode='wb') as f:
        f.write(img_content)
    num += 1
    print(pic)

 

打开爬取到的图片内容为想要的文档，后续可通过图片转文字获取详情文档