关于请求头的反爬

最新推荐文章于 2024-04-29 15:12:27 发布

xiaoShengSanYue

最新推荐文章于 2024-04-29 15:12:27 发布

阅读量706

点赞数

分类专栏： Python 文章标签： Python 反爬

本文链接：https://blog.csdn.net/yehuaner33/article/details/80766567

版权

Python 专栏收录该内容

38 篇文章 0 订阅

订阅专栏

from urllib import request
import random

base_url = 'http://www.xicidaili.com'
#构建请求头

UA = [
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.79 Safari/537.36',
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv2.0.1) Gecko/20100101 Firefox/4.0.1',
    'Mozilla/5.0 (Windows NT 6.1; rv2.0.1) Gecko/20100101 Firefox/4.0.1',
    'Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; en) Presto/2.8.131 Version/11.11',
    'Opera/9.80 (Windows NT 6.1; U; en) Presto/2.8.131 Version/11.11',
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_0) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11'
]

headers = {
    'User-Agent':random.choice(UA)
}

req = request.Request(base_url,headers = headers)

response = request.urlopen(request)

html = response.read().decode('utf-8')

print(html)