python爬虫之PyQuery的基本使用

最新推荐文章于 2024-05-11 22:30:00 发布

Time_xiaoxia

最新推荐文章于 2024-05-11 22:30:00 发布

阅读量229

点赞数

文章标签： python html

本文链接：https://blog.csdn.net/Time_xiaoxia/article/details/108760701

版权

本文介绍了PyQuery库在Python爬虫中的基本使用，重点在于如何利用CSS选择器进行元素查找。通过示例展示了如何查找子标签和父标签，同时也提到了菜鸟教程作为参考资料。

摘要由CSDN通过智能技术生成

from pyquery import PyQuery as pq
 
html = '''<div>
    <ul>
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul></div>'''
 
doc = pq(html)
print(doc)
print(type(doc))
print(doc('li'))

运行结果

<div>
    <ul>
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul></div>
<class 'pyquery.pyquery.PyQuery'>
<li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>

基于CSS选择器查找

from pyquery import PyQuery as pq
 
html = '''<div>
    <ul id = 'haha'>
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul></div>'''
 
doc = pq(html)
print(doc)
#id等于haha下面的class等于item-0下的a标签下的span标签（注意层级关系以空格隔开）
print(doc('#haha .item-0 a span'))

运行结果

<div>
    <ul id="haha">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul></div>
<span class="bold">third item</span>

在这里插入图片描述
菜鸟教程里有完整的说明：https://www.runoob.com/cssref/css-selectors.html

可以通过已经查找的标签，查找这个标签下的子标签或者父标签，而不用从头开始查找。

from pyquery import PyQuery as pq
 
html = '''<div class=‘content’>
    <ul id = 'haha'>
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul></div>'''
 
doc = pq(html)
item = doc('div ul')
print(item)
#我们可以通过已经查找到的标签，再此查找这个标签下面的标签
print(item.parent()) # 父级标签 通俗来讲就是包含它的上一个标签
print(item.children())	# 子级标签	通俗来讲就是它包含的下一个标签

运行结果

<ul id="haha">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
<div class="&#x2018;content&#x2019;">
    <ul id="haha">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul></div>
<li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>