Python爬虫学习5：使用cookie访问网页（以豆瓣为例）

最新推荐文章于 2024-06-27 10:43:48 发布

zhuzuwei

最新推荐文章于 2024-06-27 10:43:48 发布

阅读量1.1w

点赞数 5

分类专栏：爬虫文章标签：爬虫 cookie requests

本文链接：https://blog.csdn.net/zhuzuwei/article/details/80875703

版权

爬虫专栏收录该内容

7 篇文章 0 订阅

订阅专栏

1. 先在浏览器上登录豆瓣，登录成功后打开开发者工具，可以查看到Cookie.

第一次登录后，短时间内再次打开此页面时，会发现系统已经保存了cookie, 不用再重新登录。时间长了cookie会失效，需重新登录.

2. 实现代码

import requests

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 \
           (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36'}
cookies = {'cookie': 'bid=iDf0tyAI54I; ps=y; ll="118183"; __utmc=30149280; _ga=\
           GA1.2.1325106029.1530404146; _gid=GA1.2.1270378106.1530405800; ue="965454764@qq.com";\
           dbcl2="180531938:E/xiLFShgbg"; ck=UDKl; _pk_ref.100001.8cb4=%5B%22%22%2C%22%22%2C153\
           0409505%2C%22https%3A%2F%2Faccounts.douban.com%2Flogin%3Falias%3D965454764%2540qq.com\
           %26redir%3Dhttps%253A%252F%252Fwww.douban.com%26source%3DNone%26error%3D1011%22%5D;\
           _pk_id.100001.8cb4=cdbf383efde098e6.1530404145.2.1530409505.1530405796.; _pk_ses.100001.\
           8cb4=*; push_noty_num=0; push_doumail_num=0; __utma=30149280.1325106029.1530404146.153040\
           4146.1530409505.2; __utmz=30149280.1530409505.2.2.utmcsr=accounts.douban.com|utmccn=(referral)\
           |utmcmd=referral|utmcct=/login; __utmt=1; __utmv=30149280.18053; __utmb=30149280.2.10.1530409505;\
           __yadk_uid=5eZwp3s8j7joGLL911UWJkWQpVQg6IX4'}

url = 'http://www.douban.com'
r = requests.get(url, cookies = cookies, headers = headers)
with open('douban_2.txt', 'wb+') as f:
    f.write(r.content)