python 不同库解析request 的 response

最新推荐文章于 2024-02-21 16:31:23 发布

Damonhaus

最新推荐文章于 2024-02-21 16:31:23 发布

阅读量545

点赞数

分类专栏： python 文章标签： python 开发语言

本文链接：https://blog.csdn.net/Damonhaus/article/details/132602100

版权

python 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

1、 lxml

pip install lxml

import requests
from lxml import etree

# 发送HTTP请求获取文本内容
url = 'https://example.com'
response = requests.get(url)
html_content = response.text

# 将文本内容解析为Element对象
tree = etree.HTML(html_content)

# 使用XPath选择元素
selected_elements = tree.xpath('//h1')  # 使用实际的XPath表达式

# 打印提取到的元素文本
for element in selected_elements:
    print(element.text)

2、bs4

pip install beautifulsoup4

from bs4 import BeautifulSoup
import requests

url = 'https://example.com'
response = requests.get(url)
html_content = response.text

soup = BeautifulSoup(html_content, 'html.parser')
selected_elements = soup.select('h1')

for element in selected_elements:
    print(element.text)

3、xml.etree.ElementTree

内置库

import xml.etree.ElementTree as ET
import requests

url = 'https://example.com'
response = requests.get(url)
xml_content = response.text

root = ET.fromstring(xml_content)
selected_elements = root.findall('.//element_name')

for element in selected_elements:
    print(element.text)

4、scrapy

pip install scrapy

import scrapy
from scrapy.http import HtmlResponse

# 要转换的字符串内容
html_content = "<html><body><h1>Hello, World!</h1></body></html>"

# 创建一个 HtmlResponse 对象
response = HtmlResponse(url='http://example.com', body=html_content, encoding='utf-8')

# 使用 XPath 选择元素
selected_elements = response.xpath('//h1')

for element in selected_elements:
    print(element.xpath('string()').get())

四种方法得到的对象都可以说使用.xpath 去匹配对应的html元素。