selenium模拟浏览器操作时 element is not attached to the page document 等异常的处理

于 2019-08-22 16:01:38 发布
阅读量4.4k 收藏 1
点赞数
分类专栏： 问题解惑 文章标签： selenium
原文链接：https://www.deathearth.com/1073.html
版权

问题场景

利用selenium、chromedriver模拟浏览器操作网页。一套共280多个页面，循环获取页面元素内容，有输入、跳转等动作

问题现象
主要出现在以下这段代码处，打开页面后，获取页面上样式表为“.el-input__inner”的内容。

...
WebDriver w = new ChromeDriver()
w.get(baseUrl);
....
 //输入框、模拟输入搜索词
WebElement search = w.findElement(By.cssSelector(".el-input__inner"));
...

如果网络不正常、url访问很慢、页面访问排版错乱的情况发生就会出现以下错误

错误一

org.openqa.selenium.StaleElementReferenceException: stale element reference: element is not attached to the page document
  (Session info: chrome=67.0.3396.87)
  (Driver info: chromedriver=2.38.552518 (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS X 10.13.6 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html
Build info: version: '3.9.1', revision: '63f7b50', time: '2018-02-07T22:25:02.294Z'
System info: host: 'chenhailongdeMacBook-Pro.local', ip: 'fe80:0:0:0:c1f:5ed9:ab23:ed6b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.6', java.version: '1.8.0_171'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 2.38.552518 (183d19265345f5..., userDataDir: /var/folders/ll/dms152m534b...}, cssSelectorsEnabled: true, databaseEnabled: false, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, rotatable: false, setWindowRect: true, takesHeapSnapshot: true, takesScreenshot: true, unexpectedAlertBehaviour: , unhandledPromptBehavior: , version: 67.0.3396.87, webStorageEnabled: true}
Session ID: e9c1ee0bf9fc2354b7c45f2ae1dc0ceb
*** Element info: {Using=xpath, value=div}
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
	at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
	at org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:160)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:601)
	at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:279)
	at org.openqa.selenium.remote.RemoteWebElement.findElement(RemoteWebElement.java:179)
	at org.openqa.selenium.remote.RemoteWebElement.findElementByXPath(RemoteWebElement.java:255)
	at org.openqa.selenium.By$ByXPath.findElement(By.java:361)
	at org.openqa.selenium.remote.RemoteWebElement.findElement(RemoteWebElement.java:175)
	at com.chl.webmagic.processor.LawInnerProcessor.process(LawInnerProcessor.java:128)
	at us.codecraft.webmagic.Spider.processRequest(Spider.java:416)
	at us.codecraft.webmagic.Spider$1.run(Spider.java:321)
	at us.codecraft.webmagic.thread.CountableThreadPool$1.run(CountableThreadPool.java:74)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

错误二

org.openqa.selenium.WebDriverException: unknown error: Element <li class="number">...</li> is not clickable at point (365, 647). Other element would receive the click: <div class="el-loading-mask el-loading-fade-leave el-loading-fade-leave-active" style="">...</div>
  (Session info: chrome=67.0.3396.87)
  (Driver info: chromedriver=2.38.552518 (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS X 10.13.6 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: '3.9.1', revision: '63f7b50', time: '2018-02-07T22:25:02.294Z'
System info: host: 'chenhailongdeMacBook-Pro.local', ip: 'fe80:0:0:0:c1f:5ed9:ab23:ed6b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.6', java.version: '1.8.0_171'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 2.38.552518 (183d19265345f5..., userDataDir: /var/folders/ll/dms152m534b...}, cssSelectorsEnabled: true, databaseEnabled: false, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, rotatable: false, setWindowRect: true, takesHeapSnapshot: true, takesScreenshot: true, unexpectedAlertBehaviour: , unhandledPromptBehavior: , version: 67.0.3396.87, webStorageEnabled: true}
Session ID: effc4e41d823a1f2c57b007593554845
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
	at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
	at org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:160)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:601)
	at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:279)
	at org.openqa.selenium.remote.RemoteWebElement.click(RemoteWebElement.java:83)
	at com.chl.webmagic.processor.LawInnerProcessor.process(LawInnerProcessor.java:149)
	at us.codecraft.webmagic.Spider.processRequest(Spider.java:416)
	at us.codecraft.webmagic.Spider$1.run(Spider.java:321)
	at us.codecraft.webmagic.thread.CountableThreadPool$1.run(CountableThreadPool.java:74)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

问题三


org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":".el-input__inner"}
  (Session info: chrome=67.0.3396.87)
  (Driver info: chromedriver=2.38.552518 (183d19265345f54ce39cbb94cf81ba5f15905011),platform=Mac OS X 10.13.6 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Build info: version: '3.9.1', revision: '63f7b50', time: '2018-02-07T22:25:02.294Z'
System info: host: 'chenhailongdeMacBook-Pro.local', ip: 'fe80:0:0:0:c1f:5ed9:ab23:ed6b%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.13.6', java.version: '1.8.0_171'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, applicationCacheEnabled: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 2.38.552518 (183d19265345f5..., userDataDir: /var/folders/ll/dms152m534b...}, cssSelectorsEnabled: true, databaseEnabled: false, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, rotatable: false, setWindowRect: true, takesHeapSnapshot: true, takesScreenshot: true, unexpectedAlertBehaviour: , unhandledPromptBehavior: , version: 67.0.3396.87, webStorageEnabled: true}
Session ID: e7ad3e18335ec9aa43f55bfdc83bcadf
*** Element info: {Using=css selector, value=.el-input__inner}
	at sun.reflect.GeneratedConstructorAccessor68.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
	at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
	at org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:160)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:601)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:371)
	at org.openqa.selenium.remote.RemoteWebDriver.findElementByCssSelector(RemoteWebDriver.java:465)
	at org.openqa.selenium.By$ByCssSelector.findElement(By.java:430)
	at org.openqa.selenium.remote.RemoteWebDriver.findElement(RemoteWebDriver.java:363)
	at com.chl.webmagic.processor.LawInnerProcessor.process(LawInnerProcessor.java:73)
	at us.codecraft.webmagic.Spider.processRequest(Spider.java:416)
	at us.codecraft.webmagic.Spider$1.run(Spider.java:321)
	at us.codecraft.webmagic.thread.CountableThreadPool$1.run(CountableThreadPool.java:74)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

问题处理

我这里出现问题的主要原因有两个, 1.页面元素渲染慢，还没有正常加载完数据内容，就开始执行**w.findElement(By.cssSelector(".el-input__inner"));**逻辑。 2.页面样式错乱,加载错乱。如一些动态js绑定的操作是在页面加载后立马执行的，但是一些原因下没有生效。导致selenium无法执行操作，无法获取元素等。

处理方式，首先是增加线程等待，
Thread.sleep(8 * 1000);保证在获取元素时页面已经加载完毕，但是这种处理会让整个处理周期变长。另一个的办法，就是观察页面特征元素，确保一个元素是页面加载完必有的。通过判断是否存在来确定是否加载完成，如果没有完成执行刷新页面操作(针对加载不完全的处理)。

chenhailonghp
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值