![](https://img-blog.csdnimg.cn/20201014180756754.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
爬虫
邓噔噔!
这个作者很懒,什么都没留下…
展开
-
BeautifulSoup/Scrapy/Selenium爬虫框架的不同和使用方法
记录一下几个爬虫框架的区别和使用方法:Request & BeautifulSoup最简单基础使用方法import requestsfrom bs4 import BeautifulSoupfor page in range(20): url = 'https://www.网址.com/'.format(page) req = requests.get(url) html = req.text soup = BeautifulSoup(html, 'lx原创 2020-05-13 17:03:27 · 1088 阅读 · 0 评论 -
用BeautifulSoup爬取指定类div标签下的网址href
html界面如下首先导入requests和BeautifulSoup模块import requestsfrom bs4 import BeautifulSoupheader = {'user-agent': 'Mozilla/5.0'} #模拟浏览器,防止被禁req = requests.get(url, headers = header) html = req.text soup = BeautifulSoup(html, 'lxml')之前一直分不清.select()原创 2020-05-12 10:04:37 · 3271 阅读 · 3 评论