python模拟点击下一页_使用Python + Selenium在while循环中点击下一页

I'm trying to move through different pages on a website called iens. I'm using selenium + python to click on "volgende" (which means "next" in dutch), but I want my program to keep on clicking on next until there are no more pages left using a while loop. So in this case I want my program to end at page 23. Right now I'm able to get to page 2, by closing the cookie pop up message, waiting until it's closed en then clicking on the "Volgende" button.

My code looks like this:

from selenium.webdriver.support.ui import WebDriverWait

from selenium import webdriver

from selenium.webdriver.common.by import By

from selenium.webdriver.support import expected_conditions as EC

chrome_path = '/Users/user/Downloads/chromedriver'

driver = webdriver.Chrome(chrome_path)

driver.get('https://www.iens.nl/restaurant+utrecht')

#wait for cookie message

close_icon = WebDriverWait(driver, 5,

0.25).until(EC.visibility_of_element_located([By.CSS_SELECTOR,

'.cookiePolicy-close']))

close_icon.click()

#wait for cookie message to disappear

WebDriverWait(driver, 5,

0.25).until(EC.invisibility_of_element_located([By.CSS_SELECTOR,

'.cookiePolicy-close']))

click_icon = WebDriverWait(driver, 5,

0.25).until(EC.visibility_of_element_located([By.LINK_TEXT,

'Volgende']))

click_icon.click()

Thanks in advance!

解决方案

Because of automatic scrolling each time you switch to next page, webdriver tries to click Next button, but some other element receives click. You can use this solution:

while True:

try:

click_icon = WebDriverWait(driver, 5, 0.25).until(EC.visibility_of_element_located([By.LINK_TEXT, 'Volgende']))

click_icon.click()

WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'main:not([style*="margin-top"])')))

except:

break

Another simple solution (only if your goal is to reach last page) is to get last page without clicking Next all the time:

driver.find_elements_by_xpath('//div[@class="pagination"]/ul/li/a')[-2].click()