python爬取动态网页selenium

安装selenium

安装浏览器驱动

https://www.cnblogs.com/wenchaoz/p/7875365.html

代码

比如爬取PTA网页题目

• 写上自己浏览器驱动的位置
• 找到的是WebElement对象，并不是html

import time
from selenium import webdriver

url = "https://pintia.cn/problem-sets/994805260223102976/problems/type/7"

# init browser
driver = webdriver.Edge(r'E:\VirtualDesktop\code\pyCode\msedgedriver.exe')
driver.get(url)
time.sleep(3)

# get data
html = driver.find_element_by_css_selector("div.DataTableContainer_3cQiI > table > tbody > tr:nth-child(1) > td:nth-child(3)")
print(type(html)) 
print(html.text)