爬虫（一）——用Requests模块获取网页信息

最新推荐文章于 2023-06-26 15:55:25 发布

ZT_0910

最新推荐文章于 2023-06-26 15:55:25 发布

阅读量1.6k

点赞数

分类专栏： Python 文章标签：爬虫

本文链接：https://blog.csdn.net/ZT_0910/article/details/80240358

版权

Python 专栏收录该内容

6 篇文章 0 订阅

订阅专栏

调用requests库里面的get方法，获取网页的信息，调用page.text获取网页源码，然后通过print打印出网页源码

import requests
page = requests.get('https://blog.csdn.net/zt_0910/article/details/80075742')
text = page.text
print(text.encode("utf-8"))

import requests
import re

head = {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'}
TimeOut = 30


def requestpageText(url):
    try:
        Page = requests.session().get(url, headers=head, timeout=TimeOut)
        Page.encoding = "gb2312"
        return Page.text
    except BaseException as e:
        print("联网失败了...", e)


site = "http://www.meizitu.com/a/qingchun_3_1.html"
text = requestpageText(site)  # 抓取网页源码
patterns = re.compile(r'http:.*?/\d*?.html')  # 匹配需要的数据
istp = re.findall(patterns, text)
for photo in istp:
    print(photo)