Scrapy框架之selecter.xpath()函数的语法总结

最新推荐文章于 2022-11-15 16:02:19 发布

weixin_43343144

最新推荐文章于 2022-11-15 16:02:19 发布

阅读量785

点赞数

分类专栏： Scrapy python语言

本文链接：https://blog.csdn.net/weixin_43343144/article/details/87892865

版权

python语言同时被 2 个专栏收录

81 篇文章 1 订阅

订阅专栏

Scrapy

15 篇文章 0 订阅

订阅专栏

Xpath（XML的语法规则）语法规则官方手册：http://www.w3school.com.cn/xpath/xpath_syntax.asp

具体用法总结：（函数部分参考xml手册）

 html = '''
        <div>
            <ul>
            <li class="toctree-l1"><a class="reference internal" href="intro/overview.html">Scrapy at a glance</a></li>
            <li class="toctree-l1"><a class="reference internal" href="intro/install.html">Installation guide</a></li>
            <li class="toctree-l1"><a class="reference internal" href="intro/tutorial.html">Scrapy Tutorial</a></li>
            <li class="toctree-l1"><a class="reference internal" href="intro/examples.html">Examples</a></li>
            </ul>
            <p class="caption"><span class="caption-text">Basic concepts</span></p>
            <ul>
            <li class="toctree-l1"><a class="reference internal" href="topics/commands.html">Command line tool</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/spiders.html">Spiders</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/selectors.html">Selectors</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/items.html">Items</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/loaders.html">Item Loaders</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/shell.html">Scrapy shell</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/item-pipeline.html">Item Pipeline</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/feed-exports.html">Feed exports</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/request-response.html">Requests and Responses</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/link-extractors.html">Link Extractors</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/settings.html">Settings</a></li>
            <li class="toctree-l1"><a class="reference internal" href="topics/exceptions.html">Exceptions</a></li>
            </ul>
        </div>

        '''

        sel = Selector(text=html)

        xpath是基于XML的路径寻找，"/"代表html根路径
        path001 = sel.xpath("/")

        xpath是基于XML的路径寻找，"//"代表html根后代的子元素(html后代所以ul元素)
        path002 = sel.xpath("//ul")

        xpath是基于XML的路径寻找，".//"代表当前目录下所以后代的子元素(当前div目录后代所以ul元素)
        path003 = sel.xpath(".//ul")

        当前节点元素(div)
        path004 = sel.xpath(".")

        ul的父亲：div
        path005 = sel.xpath(".//ul/..")

        xpath获取文本值text()函数
        path005 = sel.xpath(".//li/a/text()")

        xpath获取属性值函数@属性名
        path005 = sel.xpath(".//li/a/@href")


        最后:[谓语]的用法,[1]、[last()]、()[1]、((1)[])

        xpath[谓语]的用法之一: 不带()的用法
        path = sel.xpath("//ul[2]/li[3]/a/text()")

        xpath[谓语]的用法之一: 带()的用法
        特别注意：(//li)[2]如何后面还要查询子元素,必须加()
        node = sel.xpath("( (//li)[2] )/a/text()")

        [谓语]运法之last()函数变量运算
        path = sel.xpath("(//li)[last()-2]/a/text()")

        [谓语]运法之position()函数变量运算
        path = sel.xpath("(//li)[position() <= 3]/a/text()")