今天在豆瓣爬电影数据时,由于过多访问导致ip受限,必须只允许登录状态才能访问网站,携带cookie的方式就可以让程序模拟登录状态
步骤
手动在网站上登录账号,再来到豆瓣主页https://www.douban.com/, f12
复制Cookie
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36'}
cookies = {'cookie': 'bid=W_1rgry0JIM; __yadk_uid=cIPFroYZzwZnNXLDmW7o3puF5Z8jUE21; gr_user_id=eda685e8-e667-4a7b-9a36-8ff7bcaa9be6; _vwo_uuid_v2=A48A2D05D645647F12E2A930D7CBB7B0|39f173efdb5b7935b3bb59119f34942d; ll="118185"; ap=1; viewed="24264880_4086032_26574906_25779298"; __utmc=30149280; ct=y; ps=y; push_noty_num=0; push_doumail_num=0; __utmv=30149280.17242; __utma=30149280.1025821821.1510046429.1515669972.1515673758.18; __utmz=30149280.1515673758.18.14.utmcsr=mail.qq.com|utmccn=(referral)|utmcmd=referral|utmcct=/; _pk_ref.100001.8cb4=%5B%22%22%2C%22%22%2C1515673822%2C%22https%3A%2F%2Fwww.baidu.com%2Flink%3Furl%3D6rw9spgRraUd3heo0P1DWdmTY3ZMOtWqYuxfoDjJPOC%26wd%3D%26eqid%3Dea9162980000b12b000000045a5749ca%22%5D; _pk_ses.100001.8cb4=*; _ga=GA1.2.1025821821.1510046429; _gid=GA1.2.898985495.1515675052; as="https://www.douban.com/"; __utmt=1; dbcl2="172422435:JCdy613zdqg"; ck=qKpS; _pk_id.100001.8cb4=c254c8a29bbdae47.1509538624.12.1515675181.1515669979.; __utmb=30149280.6.10.1515673758'}
url = 'http://www.douban.com'
r = requests.get(url, cookies = cookies, headers = headers)
print r.content