爬虫伪装用户之User-Agent，绕过反爬虫机制

编码高手，可自行绕过...

你是否一直在考虑，User-Agent+动态IP就可以轻松绕过各大平台反爬虫机制，模仿真实用户做请求动作，万无一失。

「User-Agent+IP」==“我就是一个真实用户好吧！”

动态IP有成熟的代理商，调用个接口或者购买搭建一个代理IP池子即可解决，那User-Agent呢？

Python中给我们提供了这个三方库 fake_useragent，用之不尽的User-Agent

如果你没有安装 ，直接在cmd下，pip install fake_useragent

像这样：

C:\Windows\System32>pip install fake_useragent
Collecting fake_useragent
  Downloading fake_useragent-1.5.1-py3-none-any.whl.metadata (15 kB)
Collecting importlib-resources>=5.0 (from fake_useragent)
  Downloading importlib_resources-6.4.0-py3-none-any.whl.metadata (3.9 kB)
Collecting zipp>=3.1.0 (from importlib-resources>=5.0->fake_useragent)
  Downloading zipp-3.19.2-py3-none-any.whl.metadata (3.6 kB)
Downloading fake_useragent-1.5.1-py3-none-any.whl (17 kB)
Downloading importlib_resources-6.4.0-py3-none-any.whl (38 kB)
Downloading zipp-3.19.2-py3-none-any.whl (9.0 kB)
Installing collected packages: zipp, importlib-resources, fake_useragent
Successfully installed fake_useragent-1.5.1 importlib-resources-6.4.0 zipp-3.19.2

[notice] A new release of pip is available: 24.0 -> 24.1
[notice] To update, run: python.exe -m pip install --upgrade pip

C:\Windows\System32>

from fake_useragent import UserAgent
# Create your tests here.

for i in range(1,6):
    headers = {'UserAgent': UserAgent().random}
    print(headers)

输出结果，如下：「随机取了5条User-Agent」 

{'UserAgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0'}
{'UserAgent': 'Mozilla/5.0 (Linux; Android 8.1.0; C5 2019 Build/OPM2.171019.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.6261.64 Mobile Safari/537.36'}
{'UserAgent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 17_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) EdgiOS/120.0.2210.86 Version/17.0 Mobile/15E148 Safari/604.1'}
{'UserAgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Config/92.2.2788.20'}
{'UserAgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36'}

是不是User-Agent轻松就搞定了，就这么简单，当然我们的User-Agent，每次调用一条就可以了，就像代码中的headers变量，放在参数中即可