selenium用于驱动浏览器,爬虫中用于解决js渲染问题。
基本使用:
1.声明浏览器对象
from selenium import webdriver
browser = webdriver.Chrome()
2.得到页面
from selenium import webdriver
browser = webdriver.Chrome()
browser.get('http://www.taobao.com')
print(browser.page_source)
browser.close()
3.查找元素
a)查找单个元素:
#-*-coding:utf-8-*-
from selenium import webdriver
from selenium.webdriver.common.by import By
browser = webdriver.Chrome()
browser.get('http://www.taobao.com')
input_first = browser.find_element_by_id('q')
input_two = browser.find_element_by_css_selector('#q')
input_three = browser.find_element(By.ID,'q')
print(input_first)
print(input_two)
print(input_three)
browser.close()
可以通过find_element(By.xxx,'xxx')来通用地寻找元素。
#-*-coding:utf-8-*-
from selenium import webdriver
from selenium.webdriver.common.by import By
browser = webdriver.Chrome()
browser.get('http://www.taobao.com')
lis = browser.find_elements_by_css_selector('.service-bd li')
print(lis)
lis1 = browser.find_elements(By.CSS_SELECTOR,'.service-bd li')
print(lis1)
browser.close()
运行效果;
b)元素交互
对获取的元素调用交互方法
#-*-coding:utf-8-*-
from selenium import webdriver
import time
browser = webdriver.Chrome()
browser.get(url='http://www.taobao.com')
input1 = browser.find_element_by_id('q')
button = browser.find_element_by_class_name('btn-search')
input1.send_keys('ipad')
time.sleep(1)
input1.clear()
time.sleep(1)
input1.send_keys('iphoneX')
button.click()
time.sleep(1)
browser.close()
往文本框中输入文字需要调用send_keys()方法。
或者
input.send_keys(Keys.ENTER)
c)执行JavaScript
#-*-coding:utf-8-*-
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Chrome()
browser.get(url='https://www.zhihu.com/explore')
browser.execute_script('window.scrollTo(0,document.body.scrollHeight)')
browser.execute_script('alert("scroll to buttom")')
time.sleep(5)
browser.close()
执行JavaScript交互动作
d)获取属性
#-*-coding:utf-8-*-
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Chrome()
browser.get(url='https://www.zhihu.com/explore')
print(browser.current_url)
input = browser.find_element_by_class_name('zu-top-add-question')
print(input.text)
print(input.get_attribute('class'))
browser.close()
用input.get_attribute('class')可以得到属性值
用input.text 可以得到文本值。
4.异常:
from selenium.common.exceptions import TimeoutException,NoSuchElementException
browser = webdriver.Chrome()
browser.get(url='https://www.zhihu.com/explore')
try:
input = browser.find_element_by_id('p')
except NoSuchElementException:
print('nosuchelementexception')