爬虫伪装随机浏览器

最新推荐文章于 2021-12-14 00:27:50 发布

MachinePanda

最新推荐文章于 2021-12-14 00:27:50 发布

阅读量425

点赞数

分类专栏： python 文章标签：爬虫

本文链接：https://blog.csdn.net/Oh_science/article/details/111849700

版权

本文介绍了如何在爬虫中利用fake-useragent模块，为请求添加随机的USER-AGENT，以避免被目标网站识别为爬虫。

摘要由CSDN通过智能技术生成

My_USER_AGENT = ['Mozilla/5.0 (Windows NT 6.2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1464.0 Safari/537.36',
                 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.16 Safari/537.36',
                 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.3319.102 Safari/537.36',
                 'Mozilla/5.0 (X11; CrOS i686 3912.101.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36',
                 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36',
                 'Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1667.0 Safari/537.36',
                 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:17.0) Gecko/20100101 Firefox/17.0.6',
                 'Mozilla/5.0 (Window