Python爬虫实战：使用代理（requests模块）

对于requests来说，代理设置加简单，我们只需要传入proxies参数即可

import requests

url = 'http://ip.tool.chinaz.com/'

headers = {
    'Host': "ip.tool.chinaz.com",
    'User-Agent': "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"
}
proxy = '58.240.53.196:8080'

proxy_support = {
    'http':'http://'+proxy,
    'https':'https://'+proxy,
}

response = requests.get(url,proxies=proxy_support)
print(response.text)

如果代理需要认证，同样在代理的前面加上用户名密码即可

proxies='username:password@127.0.0.1:9743'