1、phantomjs+selenium
示例代码
def phantomjs_url_test(url='http://gaia.imilive.cn/share.html?uid=0&videoid=116682377418697098&cc=TG45624'):
dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = (
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.139 Safari/537.36"
)
# dcap["phantomjs.page.settings.loadImages"] = False
driver = webdriver.PhantomJS(desired_capabilities=dcap, executable_path='/Users/tv365/phantomjs-2.1.1-macosx/bin/phantomjs')
driver.get(url)
video_url = driver.find_element_by_xpath("//video/@src")
driver.quit()
return video_url
mac版本phantomjs下载地址(linux服务器同样适用):
http://phantomjs.org/download.html
解压完成后,配置phantomjs的路径即可,示例:
2、google无头模式+selenium
google_driver下载(linux&mac)
http://chromedriver.storage.googleapis.com/index.html
浏览器版本:chrome 70.0.3538.77 驱动版本:linux243,mac243
服务器安装谷歌浏览器 服务器安装谷歌浏览器:
https://segmentfault.com/a/1190000007705458
代码示例:
def google_driver(url):
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-gpu')
client = webdriver.Chrome(chrome_options=chrome_options, executable_path='/soft/chromedriver')
# executable_path谷歌driver的路径
client.get(url)
content = client.page_source
print(content)
client.quit()
pass
google_driver('https://www.taobao.com/')
3、firefox无头模式+selenium
4.关于selenium的一些进度条滚动等操作(实质上是直接执行js)
https://blog.csdn.net/agent_x/article/details/78662860