requests
冥想10分钟大师
持续的坚持学习才是真
展开
-
Python爬虫request爬取妹子图片、保存
工具Pycharm、Chrome、Xpath插件思路——总结页面规律、标题查找、图片链接查找、文件保存"""url = http://www.mmxyz.net/rosi-2221/"""#导入模块import requests,time,osfrom lxml import etree#爬取页面def get_html(url): res = requests.get(...原创 2019-03-24 15:35:53 · 1972 阅读 · 0 评论 -
Python爬虫request批量代理设置
大量爬取为了必备封IP,需要大量代理IP,代码如下from urllib import errorimport random,requests# 代理IPproxies_list = [ {'http':'39.137.77.66:8080'}, { 'http':'124.42.7.103:80'}]url = 'http://www.baidu.com'# 免费...原创 2019-03-26 07:52:58 · 3557 阅读 · 1 评论 -
Python爬虫之selenium百度搜索
通过find_element_by_id查找搜索输入框,百度一下按钮,输入、剪切、清空搜索内容,保存屏幕截图from selenium import webdriverfrom selenium.webdriver.common.keys import Keysimport timedriver = webdriver.Chrome()url = "http://www.baidu.c...原创 2019-03-26 08:13:44 · 946 阅读 · 0 评论 -
Python爬虫requests之扇贝单词
代码,使用xpath筛选import requestsfrom lxml import etree#词汇表words = []def shanbei(page): url ='https://www.shanbay.com/wordlist/104899/202159/?page=%s'%page print(url) rsp = requests.g...转载 2019-03-23 23:43:20 · 769 阅读 · 0 评论 -
Python爬虫requests基础
数据、headers、代理怎么用上import requestsurl='http://www.renren.com/PLogin.do'data = {}headers = {}proxies = {}res = requests.post(url,data=data, headers=headers, proxies=proxies)二进制网页res.content经过默...原创 2019-04-04 18:35:55 · 214 阅读 · 0 评论