自动化工具DrissionPage的使用(一)

dh_浩开玩笑

已于 2024-04-15 17:25:02 修改

阅读量1.3k

点赞数 9

分类专栏：自动化工具文章标签： python javascript selenium 网络爬虫

于 2024-04-02 09:25:16 首次发布

本文链接：https://blog.csdn.net/Not__Cry/article/details/137256304

版权

自动化工具专栏收录该内容

2 篇文章 0 订阅

订阅专栏

概要

- 讲解自动化工具 DrissionPage 的使用
- 用到的工具 Pycharm Chrome浏览器
- 官方网站: http://drissionpage.cn/

工具的优势

无 webdriver 特征
无需为不同版本的浏览器下载不同的驱动
…具体看官网,仅这两个条件就已经勾起了我的兴趣

开始整活

安装及导入

pip install DrissionPage -i https://pypi.org/simple

from DrissionPage import ChromiumPage
from DrissionPage import SessionPage as Session
from DrissionPage import WebPage
driver = ChromiumPage()

工具讲解

- ChromiumPage 
	- 类似 selenium的 driver.Chrome(...) 会打开一个浏览器窗口
    - 只能进行GET请求
- SessionPage 
	- 使用起来类似 requests模块的 session,但比session更强大
    - 为了方便我直接取了别名
    - GET/POST 都可以满足以及代理
- WebPage
	- 集成了 ChromiumPage 和 SessionPage,还可以在两者之间自由切换,非常强大

工具介绍说明

由于内容实在较多,所以我不会全部讲解, 剩下的那些感兴趣的小伙伴可以自行研究哈

ChromiumPage

网站以百度一下为案例(https://www.baidu.com/)

请求网址拿到响应数据

driver.get('https://www.baidu.com/')
print(driver.html) # 返回的数据

搜索框定位并输入搜索数据后点击搜索

driver.get('https://www.baidu.com/')
driver.ele("#kw").input("IP地址")  # 定位到搜索框并填充数据
driver.wait.load_start()  # 等待页面加载完成
driver.ele("#su").click()  # 找到搜索按钮并点击搜索
print(driver.html)

数据包监听(很强)

driver.get('https://www.baidu.com/')
driver.ele("#kw").input("IP地址")  # 定位到搜索框并填充数据
driver.wait.load_start()  # 等待页面加载完成
driver.ele("#su").click()  # 找到搜索按钮并点击搜索

while 1:
    # 浏览器打开F12随便找一个数据包进行监听,我筛选的数据包接口是 https://qifu-api.baidubce.com/ip/local/geo/v1/district?
    driver.listen.start('qifu-api.baidubce.com/ip/local/geo/v1/district')
    res = driver.listen.wait()  # 等待并获取一个数据包
    print(res.url)  # 输出数据包url
    print(res.response.headers)  # 输出响应头
    print(res.response.statusText)  # 输出响应状态码
    print(res.response.body)  # 输出响应内容

如果不加 while循环则只会抓取一次, 加上循环以后,在浏览器中操作,只要有包含该关键url的数据包都会经过这里

在这里插入图片描述

SessionPage

无需控制浏览器

GET请求

from DrissionPage import SessionPage as Session
session = Session()

session.get('https://www.baidu.com/')
print(session.response.headers)
print(session.response.cookies)
print(session.html)

POST请求

# 百度登录接口

from DrissionPage import SessionPage as Session
session = Session()


headers = {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    "Accept-Language": "zh-CN,zh;q=0.9",
    "Cache-Control": "no-cache",
    "Connection": "keep-alive",
    "Content-Type": "application/x-www-form-urlencoded",
    "Origin": "https://www.baidu.com",
    "Pragma": "no-cache",
    "Referer": "https://www.baidu.com/",
    "Sec-Fetch-Dest": "iframe",
    "Sec-Fetch-Mode": "navigate",
    "Sec-Fetch-Site": "same-site",
    "Sec-Fetch-User": "?1",
    "Upgrade-Insecure-Requests": "1",
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.6045.160 Safari/537.36",
    "sec-ch-ua": "\"Chromium\";v=\"119\", \"Not?A_Brand\";v=\"24\"",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": "\"Windows\""
}
cookies = {
    "BAIDUID_BFESS": "E9CEB2E0983A2AB141D3211CA12899E6:FG=1",
    "ab_sr": "1.0.1_MWE4N2JkZGFjNTk5ODRmYzYwNjVkMmI5YzNkNzM0MTk0NjUxMGE2ZTQ3ZjNiOGU3YzdlMzU4ZjNiM2MzZmQwNjk5ZTFmYmViNWFmZThlMjJlMzE4ZTRjOWFjMzUzMTgxNjg3MWFiYzViMmY2MDkyYmJkYzU0OGNiZTQ5YWY0Yjc3YjQ0MzE2MWMyZjJiMjU4OTFlZDNhZDAwMzJlZDliZg==",
    "RT": "\"z=1&dm=baidu.com&si=udc7z7rrpr&ss=luh0vrcr&sl=2&tt=4ka&bcn=https%3A%2F%2Ffclog.baidu.com%2Flog%2Fweirwood%3Ftype%3Dperf&ld=drg&ul=l3b6&hd=l3io\"",
    "BIDUPSID": "E9CEB2E0983A2AB141D3211CA12899E6",
    "PSTM": "1711981308",
    "H_PS_PSSID": "40080_40373_40415_40304_40467_40457_39662_40506_40446_60043_60027_60030_60046_40510",
    "BA_HECTOR": "85050g8k80050000al0ha4a4fp4j4p1j0lgnu1s",
    "ZFY": "B1poPwprCaKme4YuP7GXTLVDAC:AsMU2Whb8R948kDXA:C",
    "BDORZ": "B490B5EBF6F3CD402E515D22BCDA1598",
    "delPer": "0",
    "PSINO": "3",
    "HOSUPPORT": "1",
    "HOSUPPORT_BFESS": "1",
    "pplogid": "9717PFujKqreRhK0Sf1UDuYoieiE6LAEB9mQBb%2FbLe7K%2BWewdqLYOvXPFFkN0UXhP7M0LEDJ5x8ciWALy2DCVdNabGCEp5MCCWzQk3LEeo2R82s7Ho%2BepjD%2Fh%2BGzUB17cSrO",
    "pplogid_BFESS": "9717PFujKqreRhK0Sf1UDuYoieiE6LAEB9mQBb%2FbLe7K%2BWewdqLYOvXPFFkN0UXhP7M0LEDJ5x8ciWALy2DCVdNabGCEp5MCCWzQk3LEeo2R82s7Ho%2BepjD%2Fh%2BGzUB17cSrO",
    "UBI": "fi_PncwhpxZ%7ETaJcyy29ZDt5D3Z7CU16TJGiAOJ0O1FXtgvbWCC27b0ljhod-zRyJkycnaAfhmAd%7Eh2mLtF",
    "UBI_BFESS": "fi_PncwhpxZ%7ETaJcyy29ZDt5D3Z7CU16TJGiAOJ0O1FXtgvbWCC27b0ljhod-zRyJkycnaAfhmAd%7Eh2mLtF",
    "logTraceID": "c66c90544e2d50c8c62df72576703a62a071995531d7a64564"
}
url = "https://passport.baidu.com/v2/api/?login"
data = {
    "staticpage": "https://www.baidu.com/cache/user/html/v3Jump.html",
    "charset": "UTF-8",
    "token": "991c7792bfee9906fe2ba468a3e0b5f9",
    "tpl": "mn",
    "subpro": "",
    "apiver": "v3",
    "tt": "1711983720050",
    "codestring": "",
    "safeflg": "0",
    "u": "https://www.baidu.com/",
    "isPhone": "",
    "detect": "1",
    "gid": "1E0EF04-C04E-4140-AD02-CD91B109E0C1",
    "quick_user": "0",
    "logintype": "dialogLogin",
    "logLoginType": "pc_loginDialog",
    "idc": "",
    "loginmerge": "true",
    "mkey": "",
    "splogin": "rate",
    "username": "188888888888",
    "password": "jtMa/F0Xc0St7r2ElsZ5Hio/vBPLPmsLt2acPP3KnSej0RL5loRjjV4TvVU6XLUoQslkvdbte7Xl6eRUqewG232tRiQczkbR2wKvebM3BQ6iSeFcGcBIZFJLSf2Rczyg0jCX23ZJjiFbz626KqJp/gfw2FCOAV9tDjNl/nShHyM=",
    "mem_pass": "on",
    "isagree": "on",
    "rsakey": "Sroqd9745nnbn7xNKsgphHMFE5bVujvQ",
    "crypttype": "12",
    "ppui_logintime": "29529",
    "countrycode": "",
    "fp_uid": "",
    "fp_info": "",
    "loginversion": "v4",
    "supportdv": "1",
    "bdint_sync_cookie": "",
    "ds": "UhoXZLMh4lSAasL1X8wWKbzUBBeDkmVPGD7bfSTp4lnok0eCqYlIdgjLsZbaGavXKOU9Begg+sZWXgXP3XAfvQfRDA6CXnBzqhFtzJM6YaZYM33tPc11Ja+r0AzWboioUPzk59jdNX4NrN/uOyTBWEj9X9JUAzqu895bSmEi2u3N6CLGLKl6dggeQVj2wla3OTdVDOb0N6t08Xz5I74sreKuyTX+S5jod1S72S/iZLijpVpJhaUaD+GhgQaST6smIsXjkNFBwxkHktphpP6d+/5uOrafp3veBZhGUIZX2RMvi8E9oTdKZ/V4rKRZWajxpmdFEjnYvdgNPcowzOj/SbkGDMpuBu9Fwh7weeWUHIaXqtfYfuz+G4tg//YOse+ak6qSznPzd02VeeWH3AvjLXQHNeu2EW9aJl4L8Gl6vWjF7yMQfP1gln6RQuAWMX6X72xntzJKtM2Kgfvwxu3E0+EFGs1JFtPNQMmltioQHcN4tgdsGRtAlSd42EIxTp/pSdnGGvWzFgtTf2oAMjjvChc4IbERZIsXKVDyGMDVrW6mg1B3Z/Ld3QFo6HmjHEVZObVNTZQ3TtW1iJ1a4OJzUk8dxJ8vq0VOPSkAXf7LicbMuBw8Ab8gdxIxf56JSIqC7Kb1Cbev74xG01XwIC7liKuLf05CTG81yj0guDOqx9JTx7/cIkhZYW7hZ/PHOHyBFyYfOVubeO4GxB1ybNDIp9ldwNGf64LSTo094IIoPSgcUvVaWqFvhtAN8kx2TxpKJp8Avwi0uJLtxvBuYSYv7ACiORDmKupeFFZqyOEOFTuvHBhxX6JItRtPCZFeP4eKJrYi1BKVfCLHF9pRTzDtX6cXqgmrNj/giEqcHH8J89UvlC6x4mnBppcqaZ4nmYdkURzXsH3pjzj+dmFN3+/IM2Pqy+7L7wDRRYwLDUTMd4hJXEJ39W5m8PlNbteIyLXdzFZk+9K+MpqIX1vnlWkV+EOpPcCwgM/p0Q+l4bmLaJKQbx7ntSVAOFPBw6UHs5z8bVFz1tb7H0KOkThwO/jK32SGnCYxN4a97VDkwyw1SWcf3+HhmujfbLQJ8grozAUcGXBkREn1UEGfHdvDGU7OSwNWRtmSqAR86aENQSi59rS4D2etPLeunbTD1ruuscH2/bO4h5R1VzSyqlhTXiyFA1jMr7Ac31FSEYeUbfeHBSmRDyzvrHnMsFEPhwbid3yUcPtP2BHOglxB437whad9Ov4rzIq/pSwaYQsjyXTBPRhOOVQwcv5zfQI2h4LNr3pMQD22DOPNfkRMpO1PWyX8tQPwz/yLIsBs7YRopnLUhgZOdSYQDc2IXGx0xc/WVASVBZeKc/sn98A8pHAff13y/F+Y9KOW8MM8PU2+F92ueRM0/GfWJZR3EFbMgL6vs1Ef2XjvfqmJhjtAc+/Ew6VqAWB1cpq9qEgJgCv84T0T52g=",
    "tk": "9717PFujKqreRhK0Sf1UDuYoieiE6LAEB9mQBb/bLe7K+WewdqLYOvXPFFkN0UXhP7M0LEDJ5x8ciWALy2DCVdNabGCEp5MCCWzQk3LEeo2R82s7Ho+epjD/h+GzUB17cSrO",
    "dv": "tk0.270197286062948241711983691046@bbv0UyCFEcsmwH8H~nHVt6LgY8HghDsFtDLR7zGL5CN9g3uk8RCk7isFqlukqc7otC0g5hF7YDHt68LzowLgYgE-7rFJtQGBblCFEbDkob6mbbut0hFVPB8HgDLghFHtiws7YDPL63EV~Sp9Hc6Fob6Lbwsk8csmwH8H~nHVt6LgY8HghDsFtDLR7zGL5CN9g3ukHgCk3isFqlukqc7otC0g5hF7YDHt68LzowLgYgE-7rFJtQGBbg6zsgDq__pv0M~sFoz6BbzsFHc6zE~ut0hFVPB8HgDLghFHtiws7YDN-SSpJP3H-gz8-YVGH3lG9gisFqy6BbzCFscsFEysmwH8H~nHVt6LgY8HghDsFtDLR6QEghap-~3DkobCkEcszVrukvzsFNc7otC0g5hF7YDHt68LzowLgYgE-7rFJtQGLbwskswukHR6Bbr6zKyut0hFVPB8HgDLghFHtiws7YDER7Wp93l7R5SEnh3EMbwskKruks-Cmbz61sgut0hFVPB8HgDLghFHtiws7YDPL63EV~Sp97iyrrI~rCuwcmJhCvIsBbrukowC8__xvlP2cbu1vRsko~6zvy61q-s1VlCkvlsFEwsFVyszN~sFql6K__ivmIn0lEnsfurYRPREXNJtUGnHXN-YQurwgpJ03GJ3XG98_Fvtsmbbuko~s1qcCFsyuko~s1qcsFqysmbwCFvbuko~s1qcsFqlsq__",
    "fuid": "FOCoIC3q5fKa8fgJnwzbE67EJ49BGJeplOzf+4l4EOvDuu2RXBRv6R3A1AZMa49I27C0gDDLrJyxcIIeAeEhD8JYsoLTpBiaCXhLqvzbzmvy3SeAW17tKgNq/Xx+RgOdb8TWCFe62MVrDTY6lMf2GrfqL8c87KLF2qFER3obJGkmn0oq5xTWD43OudD/pGogpk16NONhKjHXnomVUMd6suti94/aebzvoussIeRnUjurLwKpIToVhpDcCJmzk5uUnA5KwEIhsekptPbUQc4p207/65aYBEEf7/i0oFaJ37zSD3rH8JOudPxJORNS7SCt6rEeCerZiOC2HS/q3kWYbpA5RdGpipmODqHTwPvOqFLOQHvyhUQrrHDS6OPGdMUX7jyZCUzvCVemR8SJFAwk1UHv1DxZiH330Apc6oKkZo0Y6IJXad8xN1gQMZ1tOmckSecr9yhMSRLVoFktEC1isIoEUuh2L0ZKEXbxZOgmKXcFMb7+JFWxNGoA0JNiv6hCb0gkXpkEpISi6tVHh+hsQifjACGGz0MbLI9AAutvQNmLovQE8DrrUkOPSWZkiBwIUvxonSGS2lgiNZBxgK/Nad6P3sfvyvYhyXNwxm6SzH+Oja1l6cy9uoP7y446ILa1CLEOaV1jDkGoksNhRtn7B1VPovN1TRU04qLrmECuDGMBVR4vlhy8DqZQ1/LUEQ9mrM1XTShMu8Y6z7mcjIEx0SRhpMWhMo8MNW10I79rYiEZqj4cFtwDdJ/UZaa6iAMtId63mg/OycCYejlS4t8D766rmDHhDudxgjeO0sD8tNRkUnExmTzgWh7Lwq6FXx6a9+JIpQ6nVWxL4vL34i6mfzL4hLXcGAwm/blGCaj2qqlWEsBt+M7lZPMgMjubSlY5i2BFDFEI/FRG/MmQuDtpb0mbiXgFv9JPE1K/bdG7qMeU4yMp72LTzOOrJ6QBGMchBmwlpfnaMegzMZbIZt1BAjzkovG4kdAIyuxdvhnpBOcuRCO3PwUSWeipr32kg7rh+26VQGmUjUFB9qdWM7cM+xhEMNzi6OXbElHkA3erw54tXrxfzO8r5i/hVAMD5lmRW7zauvTts98AVvqIMSVA8vSA4Tm4Z6OA+zAWRuAfqTZYRlp9I4pEOc85gAEpZ4sd1dl7ZsDQM77UWVHypv4+aIVxlWi6jkMK+VCGNGG+ki2NSHb/I1/IeOdtnUD+FFEjsEVUO27UponOPHnWdM3X7JqhncwlFWp7ecHEL1Omsg98jm/klJ3aRE1kTuzptCsDnIwjfVJ9QODEOux3qU+1RQ==",
    "s": "je5vXz0aN3Z3TMAyAg/0AeLlggFPJ5H3fi+YhuoX4iGrRCRn56zWCRVqMJQVuiiI7rTNWkRFF4BrhrfKn0CPvH7MYaTyA4hGZ9tnbj1edPSVScjIEu3KViHm6qRWvtHKkI9gtaOw08jdZJYDTsFOrDmFD6FqGqbz1nbfGw9Ny5LiiFGCDIRgR8t6R3Xii7RV2ggQGNPe9Yct4AA0uC1gj1g2UKV91s5btJ8lYEqJVLZQZLlXW/Vukat/ADTGl1oHqJsmXG9BUjguLIT7PucqgFnpxttpE2RYgShaPZ3xn8CFjMIHFwzrc3j8CSeEHTiKG9rC7HwftVpdRhVnf8vzNk0I8jrRDkSUEV1xzTzkHyCTJPO6votUyUw5+VB2IIpMks6p2tsr1OkuWrTtQPCbk2rEe6RV4WbxkVh3e8jygMA=",
    "k": "91342SAlGaAC8/vdQGkTgrr7W2qgfE5XmRM7k1tmwhbSXjYMW8z17JHd/8LMn93nFNhYBa+RzZta+YCEYN8eDFlGXizWzI/Te1+RNI+6zCsEgycBroM7LkvluS/D3Ne0XZViZxUiO2pwoz3PtlJs5f1ocYh4ADkbAPC9PPJtFvzbqLI=",
    "traceid": "26449A01",
    "callback": "parent.bd__pcbs__he30mq",
    "time": "1711983720",
    "alg": "v3",
    "sig": "dnpPNkVZRTZaSExMQlVock9nazJ6VE9JSjYwdThTZDBTVE9BS28rVWJYcnRiK2g1RWt4cnZVT1lvcndSVGlWMg==",
    "elapsed": "2",
    "shaOne": "007923016aec0fea625c90adbba1c428e8e3b405",
    "rinfo": "{\"fuid\":\"205a59d18029be956d5580d67214be3c\"}"
}


session.post(url, headers=headers, data=data, cookies=cookies)
print(session.response.headers)
print(session.response.cookies)
print(session.html)

cookie设置

# 这些设置是全局参数，设置后每次请求都会使用它们
session.set.cookies([{'name': 'a', 'value': '1'}, {'name': 'b', 'value': '2'}])

为什么不直接用requests模块?

首先ChromiumPage和SessionPage之间是可以互通的,包括cookie信息
SessionPage在请求数据之后可以直接使用内置方法解析数据,不用导其他库了
其他还有很多,我就不总结了

WebPage

整合了SessionPage和ChromiumPage，实现了两者之间的互通
它既可以操控浏览器，也可以收发数据包，并且会在两者之间同步登录信息
剩下的就不讲了,睡觉!!!

贴一下我的名片