selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
问题原因就是找不到这个element
在xpath输入正确的前提下,使用time.sleep(2)可以有效的解决这个问题
以下为详细代码
import time
from selenium import webdriver
# options = webdriver.ChromeOptions()
# options.add_argument('--headless')
driver = webdriver.Chrome()
url = "https://www.douyu.com/directory/all"
driver.get(url)
while True:
time.sleep(3)
room_list = driver.find_elements_by_xpath('//*[@id="listAll"]/section[2]/div[2]/ul/li/div')
for room in room_list:
print(room)
rooms = {}
# 房间标题
rooms['title'] = room.find_element_by_xpath('./a[1]/div[2]/div[1]/h3').text
# 主播姓名
rooms['zhubo'] = room.find_element_by_xpath('./a[1]/div[2]/div[2]/h2').text
# 房间类型
rooms['kind'] = room.find_element_by_xpath('./a[1]/div[2]/div[1]/span').text
# 人气程度
rooms['hot'] = room.find_element_by_xpath('./a[1]/div[2]/div[2]/span').text
print(rooms)
# 一页遍历完之后,进行下一页的遍历
try:
next_botton = driver.find_element_by_xpath('//*[@id="listAll"]/section[2]/div[2]/div/ul/li[9]/span')
driver.execute_script('scrollTo(0,100000)')
next_botton.click()
# 如果找不到下一页,就结束
except Exception as e:
print(e)
break
代码存在每遍历一页要sleep的速度缺陷,以后希望通过学习更多加以修复