reqeusts库,在使用ip代理时,单ip代理和多ip代理的写法不同
(目前测试通过,如有错误,请评论指正)
单ip代理模式
省去headers等import requests proxy = { 'HTTPS': '162.105.30.101:8080' } url = '爬取链接地址' response = requests.get(url,proxies=proxy)
多ip代理模式
import requests
#导入random,对ip池随机筛选
import random
proxy = [
{
'http': 'http://61.135.217.7:80',
'https': 'http://61.135.217.7:80',
},
{
'http': 'http://118.114.77.47:8080',
'https': 'http://118.114.77.47:8080',
},
{
'http': 'http://112.114.31.177:808',
'https': 'http://112.114.31.177:808',
},
{
'http': 'http://183.159.92.117:18118',
'https': 'http://183.159.92.117:18118',
},
{
'http': 'http://110.73.10.186:8123',
'https':