Linux配置Selenium+Chrome+Java实现自动化测试
获取世界人口排名2021数据
文章目录
获取如下分页列表页面数据
获取世界人口排名2021数据
https://web.phb123.com/city/renkou/rk.html?ivk_sa=1024320u
编写程序以如下格式输出采集的数据。
世界排名 国家 人口数量 增长率 人口密度（公里²）
| | | | |
| | | | |
| | | | |
采用linux+java+selenium的方式
采用selenium自动化来对浏览器页面数据进行采集。
linux与java版本如下：
wangyetao@wangyetao-Lenovo-G510:~$ uname -a
Linux wangyetao-Lenovo-G510 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
wangyetao@wangyetao-Lenovo-G510:~$ java -version
java version "1.8.0_191"
Java(TM) SE Runtime Environment (build 1.8.0_191-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.191-b12, mixed mode)
wangyetao@wangyetao-Lenovo-G510:~$
下载并安装linux Chrome
查询出版本号为Google Chrome 96.0.4664.110
wangyetao@wangyetao-Lenovo-G510:~$ google-chrome --version
Google Chrome 96.0.4664.110
wangyetao@wangyetao-Lenovo-G510:~$
下载linux Chrome对应的驱动版本
官网下载地址：http://npm.taobao.org/mirrors/chromedriver/
下载对应版本：96.0.4664.18(3个版本均可)
chromedriver_linux64.zip
下载好后解压文件包到任意目录。
配置驱动程序
- 1.命令行手动启动
wangyetao@wangyetao-Lenovo-G510:~/Download/chromedriver_linux64$ chmod +x chromedriver
wangyetao@wangyetao-Lenovo-G510:~/Download/chromedriver_linux64$ l
chromedriver*
wangyetao@wangyetao-Lenovo-G510:~/Download/chromedriver_linux64$ ./chromedriver
Starting ChromeDriver 96.0.4664.18 (b8887b3d1742adb0873f871edc1d8d8c1d46bb96-refs/branch-heads/4664@{#236}) on port 9515
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
- 2.或将驱动程序放置在操作系统目录下 /usr/bin目录下。
/usr/bin/chromedriver
编码Java程序
单独下载java selenium依赖组合包(不推荐)
selenium官网：https://www.selenium.dev/
下载链接：https://www.selenium.dev/downloads/
对应API：https://www.selenium.dev/selenium/docs/api/java/overview-summary.html
下载：4.1.1 (December 17, 2021)
解压包selenium-java-4.1.0.zip，找到依赖包引入到java工程项目中：
selenium-api-4.1.0.jar
selenium-chrome-driver-4.1.0.jar
jar包组合依赖不对，测试运行老是提示类缺失/无法初始化，2021年 12月 26日 星期日 01:00:20 CST。
采用Maven引入所必须jar依赖包(推荐)
在maven工程项目pom.xml中< dependencies >< dependencies />内添加如下依赖项。
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>4.0.0</version>
</dependency>
</dependencies>
WebDriver的使用说明
官网文档链接：https://www.selenium.dev/documentation/webdriver/getting_started/install_drivers/
First Script：https://www.selenium.dev/documentation/webdriver/getting_started/first_script/
1.Start the session with a driver instance
WebDriver driver = new ChromeDriver();
2.Send command for browser to Navigate
driver.get("https://google.com");
3.Request information about the browser
driver.getTitle(); // => "Google"
4.Send command to find an element
WebElement searchBox = driver.findElement(By.name("q"));
WebElement searchButton = driver.findElement(By.name("btnK"));
5.Send command to take action on an element
searchBox.sendKeys("Selenium");
searchButton.click();
6.Request information about an element
driver.findElement(By.name("q")).getAttribute("value"); // => "Selenium"
7.End the session
driver.quit();
程序部分异常提示
1.java.lang.NoClassDefFoundError
Caused by: java.lang.NoClassDefFoundError: Could not initialize class io.netty.handler.codec.http.HttpMethod
log4j:WARN No appenders could be found for logger (io.netty.util.internal.logging.InternalLoggerFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Starting ChromeDriver 96.0.4664.18 (b8887b3d1742adb0873f871edc1d8d8c1d46bb96-refs/branch-heads/4664@{#236}) on port 52974
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
十二月 26, 2021 1:01:42 上午 org.openqa.selenium.remote.http.RetryRequest lambda$static$5
警告: Failure due to server error #1. Retrying.
十二月 26, 2021 1:01:44 上午 org.openqa.selenium.remote.http.RetryRequest lambda$static$5
警告: Failure due to server error #2. Retrying.
Exception in thread "main" org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '4.1.0', revision: '87802e897b'
System info: host: 'wangyetao-Lenovo-G510', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-91-generic', java.version: '1.8.0_191'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [], extensions: []}}}]
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:577)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:246)
at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:168)
at org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:108)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:104)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:91)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:46)
at simple.callback.seleniumtest.SeleniumtestDemo.main(SeleniumtestDemo.java:30)
Caused by: java.lang.NoClassDefFoundError: Could not initialize class io.netty.handler.codec.http.HttpMethod
at org.asynchttpclient.netty.request.NettyRequestFactory.newNettyRequest(NettyRequestFactory.java:114)
at org.asynchttpclient.netty.request.NettyRequestSender.newNettyRequestAndResponseFuture(NettyRequestSender.java:204)
at org.asynchttpclient.netty.request.NettyRequestSender.sendRequestWithCertainForceConnect(NettyRequestSender.java:136)
at org.asynchttpclient.netty.request.NettyRequestSender.sendRequest(NettyRequestSender.java:114)
at org.asynchttpclient.DefaultAsyncHttpClient.execute(DefaultAsyncHttpClient.java:259)
at org.asynchttpclient.DefaultAsyncHttpClient.executeRequest(DefaultAsyncHttpClient.java:228)
at org.asynchttpclient.DefaultAsyncHttpClient.executeRequest(DefaultAsyncHttpClient.java:249)
at org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:57)
at org.openqa.selenium.remote.http.RetryRequest.lambda$apply$6(RetryRequest.java:82)
at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48)
at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66)
at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66)
at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66)
at net.jodah.failsafe.Execution.executeSync(Execution.java:128)
at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:379)
at net.jodah.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:68)
at org.openqa.selenium.remote.http.RetryRequest.lambda$apply$7(RetryRequest.java:82)
at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
at org.openqa.selenium.remote.http.netty.NettyHttpHandler.execute(NettyHttpHandler.java:51)
at org.openqa.selenium.remote.http.RetryRequest.lambda$apply$6(RetryRequest.java:82)
at net.jodah.failsafe.Functions.lambda$get$0(Functions.java:48)
at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66)
at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66)
at net.jodah.failsafe.RetryPolicyExecutor.lambda$supply$0(RetryPolicyExecutor.java:66)
at net.jodah.failsafe.Execution.executeSync(Execution.java:128)
at net.jodah.failsafe.FailsafeExecutor.call(FailsafeExecutor.java:379)
at net.jodah.failsafe.FailsafeExecutor.get(FailsafeExecutor.java:68)
at org.openqa.selenium.remote.http.RetryRequest.lambda$apply$7(RetryRequest.java:82)
at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
at org.openqa.selenium.remote.http.netty.NettyClient.execute(NettyClient.java:108)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:102)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:84)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:62)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:156)
at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:164)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:139)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:559)
... 7 more
Process finished with exit code 1
2.java.lang.NoSuchMethodError
Caused by: java.lang.NoSuchMethodError: io.netty.util.internal.ObjectUtil.checkNonEmptyAfterTrim(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;
log4j:WARN No appenders could be found for logger (io.netty.util.internal.logging.InternalLoggerFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Starting ChromeDriver 96.0.4664.45 (76e4c1bb2ab4671b8beba3444e61c0f17584b2fc-refs/branch-heads/4664@{#947}) on port 55403
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
org.openqa.selenium.SessionNotCreatedException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '4.0.0', revision: '3a21814679'
System info: host: 'wangyetao-Lenovo-G510', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-91-generic', java.version: '1.8.0_191'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [null, newSession {capabilities=[Capabilities {browserName: chrome, goog:chromeOptions: {args: [--headless, --no-sandbox, --disable-dev-shm-usage, window-size=1920x3000, --disable-gpu, --hide-scrollbars, blink-settings=imagesEnable..., --headless], extensions: []}}], desiredCapabilities=Capabilities {browserName: chrome, goog:chromeOptions: {args: [--headless, --no-sandbox, --disable-dev-shm-usage, window-size=1920x3000, --disable-gpu, --hide-scrollbars, blink-settings=imagesEnable..., --headless], extensions: []}}}]
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:577)
at org.openqa.selenium.remote.RemoteWebDriver.startSession(RemoteWebDriver.java:246)
at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:168)
at org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:108)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:104)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:91)
at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:80)
at simple.callback.seleniumtest.SeleniumtestDemo.main(SeleniumtestDemo.java:39)
Caused by: java.lang.NoSuchMethodError: io.netty.util.internal.ObjectUtil.checkNonEmptyAfterTrim(Ljava/lang/String;Ljava/lang/String;)Ljava/lang/String;
at io.netty.handler.codec.http.HttpMethod.<init>(HttpMethod.java:123)
at io.netty.handler.codec.http.HttpMethod.<clinit>(HttpMethod.java:36)
at org.asynchttpclient.netty.request.NettyRequestFactory.newNettyRequest(NettyRequestFactory.java:114)
at org.asynchttpclient.netty.request.NettyRequestSender.newNettyRequestAndResponseFuture(NettyRequestSender.java:204)
at org.asynchttpclient.netty.request.NettyRequestSender.sendRequestWithCertainForceConnect(NettyRequestSender.java:136)
at org.asynchttpclient.netty.request.NettyRequestSender.sendRequest(NettyRequestSender.java:114)
at org.asynchttpclient.DefaultAsyncHttpClient.execute(DefaultAsyncHttpClient.java:259)
at org.asynchttpclient.DefaultAsyncHttpClient.executeRequest(DefaultAsyncHttpClient.java:228)
at org.asynchttpclient.DefaultAsyncHttpClient.executeRequest(DefaultAsyncHttpClient.java:249)
at org.openqa.selenium.remote.http.netty.NettyHttpHandler.makeCall(NettyHttpHandler.java:57)
at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
at org.openqa.selenium.remote.http.netty.NettyHttpHandler.execute(NettyHttpHandler.java:51)
at org.openqa.selenium.remote.http.AddSeleniumUserAgent.lambda$apply$0(AddSeleniumUserAgent.java:42)
at org.openqa.selenium.remote.http.Filter.lambda$andFinally$1(Filter.java:56)
at org.openqa.selenium.remote.http.netty.NettyClient.execute(NettyClient.java:119)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:102)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:84)
at org.openqa.selenium.remote.ProtocolHandshake.createSession(ProtocolHandshake.java:62)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:156)
at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:164)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:139)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:559)
... 7 more
Process finished with exit code 0
3.java.lang.IllegalStateException
Exception in thread "main" java.lang.IllegalStateException: The path to the driver executable The path to the driver executable must be set by the webdriver.gecko.driver system property; for more information, see https://github.com/mozilla/geckodriver. The latest version can be downloaded from https://github.com/mozilla/geckodriver/releases
at org.openqa.selenium.internal.Require$StateChecker.nonNull(Require.java:280)
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:132)
at org.openqa.selenium.firefox.GeckoDriverService.access$100(GeckoDriverService.java:44)
at org.openqa.selenium.firefox.GeckoDriverService$Builder.findDefaultExecutable(GeckoDriverService.java:185)
at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:434)
at org.openqa.selenium.firefox.FirefoxDriver.toExecutor(FirefoxDriver.java:232)
at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:186)
at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:164)
at simple.callback.seleniumtest.SeleniumtestDemo.main(SeleniumtestDemo.java:29)
Process finished with exit code 1
4.org.openqa.selenium.StaleElementReferenceException
Method threw ‘org.openqa.selenium.StaleElementReferenceException’ exception.
Exception in thread "main" org.openqa.selenium.StaleElementReferenceException: stale element reference: element is not attached to the page document
(Session info: chrome=96.0.4664.110)
For documentation on this error, please visit: https://selenium.dev/exceptions/#stale_element_reference
Build info: version: '4.0.0', revision: '3a21814679'
System info: host: 'wangyetao-Lenovo-G510', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-91-generic', java.version: '1.8.0_191'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Command: [56abd059e01fc24072f367c4c27e7601, getElementText {id=0fe90811-12ff-4cd9-855a-c821a14ed8f0}]
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 96.0.4664.110, chrome: {chromedriverVersion: 96.0.4664.45 (76e4c1bb2ab46..., userDataDir: /tmp/.com.google.Chrome.s3uLJs}, goog:chromeOptions: {debuggerAddress: localhost:38787}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: LINUX, platformName: LINUX, proxy: Proxy(), se:cdp: ws://localhost:38787/devtoo..., se:cdpVersion: 96.0.4664.110, setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Element: [[ChromeDriver: chrome on LINUX (56abd059e01fc24072f367c4c27e7601)] -> link text: 下一页]
Session ID: 56abd059e01fc24072f367c4c27e7601
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.createException(W3CHttpResponseCodec.java:200)
at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:133)
at org.openqa.selenium.remote.codec.w3c.W3CHttpResponseCodec.decode(W3CHttpResponseCodec.java:53)
at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:184)
at org.openqa.selenium.remote.service.DriverCommandExecutor.invokeExecute(DriverCommandExecutor.java:164)
at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:139)
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:559)
at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:251)
at org.openqa.selenium.remote.RemoteWebElement.getText(RemoteWebElement.java:201)
at simple.call.seleniumtest.SeleniumtestDemo.main(SeleniumtestDemo.java:69)
Process finished with exit code 1
原因是元素时效过时，解决方法(重新获取一遍就好)：https://www.selenium.dev/exceptions/#stale_element_reference
Selenium的八种元素定位方式
八种： id、linkText、partialLinkText、name、 tagName、xpath、className、cssSelector。
public static By id(String id) {
return new By.ById(id);
}
public static By linkText(String linkText) {
return new By.ByLinkText(linkText);
}
public static By partialLinkText(String partialLinkText) {
return new By.ByPartialLinkText(partialLinkText);
}
public static By name(String name) {
return new By.ByName(name);
}
public static By tagName(String tagName) {
return new By.ByTagName(tagName);
}
public static By xpath(String xpathExpression) {
return new By.ByXPath(xpathExpression);
}
public static By className(String className) {
return new By.ByClassName(className);
}
public static By cssSelector(String cssSelector) {
return new By.ByCssSelector(cssSelector);
}
WebElement.class
//
// Source code recreated from a .class file by IntelliJ IDEA
// (powered by Fernflower decompiler)
//
package org.openqa.selenium;
import java.util.List;
public interface WebElement extends SearchContext, TakesScreenshot {
void click();
void submit();
void sendKeys(CharSequence... keysToSend);
void clear();
String getTagName();
default String getDomProperty(String name) {
throw new UnsupportedOperationException("getDomProperty");
}
default String getDomAttribute(String name) {
throw new UnsupportedOperationException("getDomAttribute");
}
String getAttribute(String name);
default String getAriaRole() {
throw new UnsupportedOperationException("getAriaRole");
}
default String getAccessibleName() {
throw new UnsupportedOperationException("getAccessibleName");
}
boolean isSelected();
boolean isEnabled();
String getText();
List<WebElement> findElements(By by);
WebElement findElement(By by);
default SearchContext getShadowRoot() {
throw new UnsupportedOperationException("getShadowRoot");
}
boolean isDisplayed();
Point getLocation();
Dimension getSize();
Rectangle getRect();
String getCssValue(String propertyName);
}
Java DEMO完整样例程序
write this，2021年 12月 26日 星期日 01:13:06 CST。
/**
* Copyright (C), 2000-2021, XXX有限公司
* FileName: SeleniumtestDemo
* Author: wangyetao
* Date: 21-12-25 22:21:25
* Description: seleniumtest用例测试
* <p>
* History:
* <author> 作者姓名
* <time> 修改时间
* <version> 版本号
* <desc> 版本描述
*
* <author> wangyetao
* <time> 2021年 12月 26日 星期日 06:52:49 CST
* <version> 版本号
* <desc> 最近一次测试与修改
*
* <author> wangyetao
* <time> 2021年 12月 26日 星期日 07:24:27 CST
* <version> 版本号
* <desc> 最近一次修改，字符串输出对齐
*/
package simple.call.seleniumtest;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import java.util.List;
/**
* @ClassName: SeleniumtestDemo
* @Description: java类描述
* @Author: wangyetao
* @Date: 21-12-25 22:21:25
*/
class SeleniumtestDemo {
private static WebElement nextPageElement = null;
//测试用例
public static void main(String[] args) {
//世界人口排名2021 url
String url = "https://web.phb123.com/city/renkou/rk.html?ivk_sa=1024320u";
//预先设置驱动
System.setProperty("webdriver.chrome.driver", "/usr/bin/chromedriver");
// Initialize your driver as you normally would:
ChromeDriver driver = new ChromeDriver();
driver.get(url);
System.out.println(driver.getTitle());
//单行数据DOM结构
//<table class="rank-table" width="100%">
//<tbody>
//
//<tr>
// <td class="xh">1</td>
// <td>
// <a href="/city/renkou/country_1.html" title="中国" target="_blank" class="cty">
// <span class="fl"><img src="https://img.phb123.com/uploads/guoqi/CN.jpg" alt="中国" width="47" height="28"></span>
// <p>中国</p>
// </a>
// </td>
// <td>1,411,780,000</td>
// <td>0.39%</td>
// <td>144.30</td>
//</tr>
//
//<tbody/>
//<table/>
//下一页，页面元素
nextPageElement = driver.findElement(By.linkText("下一页"));
//System.out.println("世界排名\t\t\t\t国家\t\t\t\t人口数量\t\t\t\t增长率\t\t\t\t人口密度（公里²）\n");
System.out.format("%-20.20s %-20.20s %-20.20s %-20.20s %-20.20s", "世界排名", "国家", "人口数量", "增长率", "人口密度（公里²）");
System.out.println();
try {
while (nextPageElement != null) {
List<WebElement> search_results = driver.findElements(By.xpath("//table/tbody/tr"));
for (int i = 1; i < search_results.size(); i++) {
WebElement element = search_results.get(i);
//世界排名
String worldRanking = element.findElement(By.className("xh")).getText();
//国家
String country = element.findElement(By.tagName("p")).getText();
List<WebElement> tds = element.findElements(By.tagName("td"));
//人口数量
String populationSize = tds.get(2).getText();
//增长率
String growthRate = tds.get(3).getText();
//人口密度（公里²）
String populationDensity = tds.get(4).getText();
//System.out.println(worldRanking + "\t\t\t\t" + country + "\t\t\t\t\t\t" + populationSize + "\t\t\t\t\t\t" + growthRate + "\t\t\t\t" + populationDensity + "\n");
System.out.format("%-20.20s %-20.20s %-20.20s %-20.20s %-20.20s", worldRanking, country, populationSize, growthRate, populationDensity);
System.out.println();
}
//点击下一页
nextPageElement.click();
//设置两秒浏览器加载时间，等待浏览器渲染完毕
try {
Thread.sleep(2000);
} catch (InterruptedException e) {
e.printStackTrace();
}
//重新获取DOM，防止DOM失效而抛出NoSuchElementException
nextPageElement = driver.findElement(By.linkText("下一页"));
}
} catch (Exception e) {
}
driver.close();
}
}
运行程序
Running…
运行结果
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Starting ChromeDriver 96.0.4664.45 (76e4c1bb2ab4671b8beba3444e61c0f17584b2fc-refs/branch-heads/4664@{#947}) on port 57451
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
十二月 26, 2021 6:04:24 上午 org.openqa.selenium.remote.ProtocolHandshake createSession
信息: Detected dialect: W3C
十二月 26, 2021 6:04:24 上午 org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
警告: Unable to find an exact match for CDP version 96, so returning the closest version found: a no-op implementation
十二月 26, 2021 6:04:24 上午 org.openqa.selenium.devtools.CdpVersionFinder findNearestMatch
信息: Unable to find CDP implementation matching 96.
十二月 26, 2021 6:04:24 上午 org.openqa.selenium.chromium.ChromiumDriver lambda$new$3
警告: Unable to find version of CDP to use for . You may need to include a dependency on a specific version of the CDP using something similar to `org.seleniumhq.selenium:selenium-devtools-v86:4.0.0` where the version ("v86") matches the version of the chromium-based browser you're using and the version number of the artifact is the same as Selenium's.
世界人口排名 2021世界人口总数 全球各国人口排名2021-排行榜123网
世界排名 国家 人口数量 增长率 人口密度（公里²）
1 中国 1,411,780,000 0.39% 144.30
2 印度 1,354,051,854 1.11% 411.87
3 美国 326,766,748 0.71% 34.86
4 印度尼西亚 266,794,980 1.06% 140.08
5 巴西 210,867,954 0.75% 24.76
6 巴基斯坦 200,813,818 1.93% 227.70
7 尼日利亚 195,875,237 2.61% 212.04
8 孟加拉国 166,368,149 1.03% 1127.38
9 俄罗斯 143,964,709 0.00% 8.42
10 墨西哥 130,759,074 1.24% 66.57
11 日本 127,185,332 0.00% 336.53
12 埃塞俄比亚 106,672,306 2.46% 97.38
13 菲律宾 106,512,074 1.52% 311.12
14 埃及 99,375,741 1.87% 99.13
15 越南 96,491,146 0.99% 291.33
16 刚果民主共和国 82,643,624 3.09% 34.70
17 德国 82,293,457 0.22% 230.44
18 伊朗 82,011,735 1.05% 49.76
19 土耳其 81,916,871 1.45% 104.54
20 泰国 69,183,173 0.21% 134.83
21 英国 66,573,504 0.59% 274.08
22 法国 65,233,271 0.39% 118.24
23 意大利 60,482,200 0.00% 196.76
24 坦桑尼亚 59,091,392 3.11% 62.52
25 南非 57,398,421 1.20% 47.01
26 缅甸 53,855,735 0.91% 79.60
27 韩国 51,269,185 0.36% 510.57
28 肯尼亚 50,950,879 2.52% 87.79
29 哥伦比亚 49,464,683 0.81% 43.32
30 西班牙 46,397,452 0.09% 91.70
31 阿根廷 44,688,864 0.94% 16.07
32 乌干达 44,270,563 3.28% 183.28
33 乌克兰 44,009,214 0.00% 72.92
34 阿尔及利亚 42,008,054 1.67% 17.64
35 苏丹 41,511,526 2.41% 22.01
36 伊拉克 39,339,753 2.78% 89.75
37 波兰 38,104,832 0.00% 121.87
38 加拿大 36,953,765 0.90% 3.70
39 阿富汗 36,373,176 2.37% 55.77
40 摩洛哥 36,191,805 1.27% 81.05
41 沙特阿拉伯 33,554,343 1.87% 15.61
42 秘鲁 32,551,815 1.20% 25.33
43 委内瑞拉 32,381,221 1.26% 35.33
44 乌兹别克斯坦 32,364,996 1.42% 72.34
45 马来西亚 32,042,458 1.32% 96.86
46 安哥拉 30,774,205 3.32% 24.68
47 莫桑比克 30,528,673 2.90% 38.09
48 尼泊尔 29,624,035 1.09% 201.28
49 加纳 29,463,643 2.18% 123.52
50 也门 28,915,284 2.35% 54.77
51 马达加斯加 26,262,810 2.71% 44.60
52 朝鲜 25,663,011 0.47% 212.47
53 科特迪瓦 24,905,843 2.52% 77.24
54 澳大利亚 24,772,247 1.32% 3.22
55 喀麦隆 24,678,234 2.60% 51.91
57 尼日尔 22,311,375 3.88% 17.61
58 斯里兰卡 20,950,041 0.35% 319.31
59 布基纳法索 19,751,651 2.91% 72.36
60 罗马尼亚 19,580,634 0.00% 82.14
61 马拉维 19,164,728 2.91% 161.75
62 马里 19,107,706 3.05% 15.41
63 哈萨克斯坦 18,403,860 1.10% 6.75
64 叙利亚 18,284,407 0.08% 98.74
65 智利 18,197,209 0.79% 24.07
66 赞比亚 17,609,178 3.01% 23.40
67 危地马拉 17,245,346 1.96% 158.38
68 荷兰 17,084,459 0.28% 408.23
69 津巴布韦 16,913,261 2.32% 43.28
70 厄瓜多尔 16,863,425 1.44% 60.91
71 塞内加尔 16,294,270 2.80% 82.83
72 柬埔寨 16,245,729 1.50% 89.74
73 乍得 15,353,184 3.04% 11.96
74 索马里 15,181,925 2.98% 23.81
75 几内亚 13,052,608 2.64% 53.09
76 南苏丹 12,919,053 2.73% 20.85
77 卢旺达 12,501,156 2.40% 474.64
78 突尼斯 11,659,174 1.10% 71.26
79 比利时 11,498,519 0.61% 376.65
80 古巴 11,489,082 0.04% 104.56
81 贝宁 11,485,674 2.77% 101.98
82 布隆迪 11,216,450 3.24% 402.98
83 玻利维亚 11,215,674 1.48% 10.21
84 希腊 11,142,161 0.00% 84.42
85 海地 11,112,945 1.20% 400.47
86 多明尼加共和国 10,882,996 1.08% 223.60
87 捷克共和国 10,625,250 0.07% 134.73
88 葡萄牙 10,291,196 0.00% 111.75
89 瑞典 9,982,709 0.73% 22.17
90 阿塞拜疆 9,923,914 0.98% 114.59
91 约旦 9,903,802 2.08% 110.85
92 匈牙利 9,688,847 0.00% 104.15
93 阿联酋 9,541,615 1.50% 114.13
94 白俄罗斯 9,452,113 0.00% 45.53
95 洪都拉斯 9,417,167 1.64% 83.71
96 塔吉克斯坦 9,107,211 2.08% 63.64
97 塞尔维亚 8,762,027 0.00% 99.16
98 奥地利 8,751,820 0.19% 104.35
99 瑞士 8,544,034 0.80% 206.96
100 以色列 8,452,841 1.58% 406.97
101 巴布亚新几内亚 8,418,346 2.03% 18.19
102 多哥 7,990,926 2.48% 140.72
103 塞拉利昂 7,719,729 2.15% 107.61
105 保加利亚 7,036,848 0.00% 63.46
106 老挝 6,961,210 1.50% 29.40
107 巴拉圭 6,896,908 1.26% 16.96
108 利比亚 6,470,956 1.51% 3.68
109 萨尔瓦多 6,411,558 0.53% 304.72
110 尼加拉瓜 6,284,757 1.08% 48.21
111 吉尔吉斯斯坦 6,132,932 1.45% 30.67
112 黎巴嫩 6,093,509 0.18% 583.00
113 土库曼斯坦 5,851,466 1.62% 11.99
114 新加坡 5,791,901 1.45% 8157.61
115 丹麦 5,754,356 0.36% 133.53
116 芬兰 5,542,517 0.35% 16.38
117 斯洛伐克 5,449,816 0.04% 111.14
118 刚果共和国 5,399,895 2.11% 34.83
119 挪威 5,353,363 0.90% 16.53
120 厄立特里亚 5,187,948 2.35% 44.12
121 巴勒斯坦 5,052,776 2.68% 812.34
122 哥斯达黎加 4,953,199 0.97% 96.93
123 利比里亚 4,853,516 2.57% 43.58
124 阿曼 4,829,946 4.18% 15.61
125 爱尔兰 4,803,748 0.88% 68.36
126 新西兰 4,749,598 0.93% 17.56
127 中非共和国 4,737,423 1.68% 7.60
128 毛里塔尼亚 4,540,068 2.71% 4.40
129 科威特 4,197,128 1.46% 235.56
130 克罗地亚 4,164,783 0.00% 73.59
131 巴拿马 4,162,618 1.56% 55.19
132 摩尔多瓦 4,041,065 0.00% 119.40
133 格鲁吉亚 3,907,131 0.00% 56.06
134 波多黎各 3,659,007 0.00% 412.51
135 波黑 3,503,554 0.00% 68.42
136 乌拉圭 3,469,551 0.37% 19.17
137 蒙古 3,121,772 1.50% 2.00
138 阿尔巴尼亚 2,934,363 0.14% 102.07
139 亚美尼亚 2,934,152 0.13% 98.65
140 牙买加 2,898,677 0.29% 263.73
141 立陶宛 2,876,475 0.00% 44.05
142 卡塔尔 2,694,849 2.11% 232.60
143 纳米比亚 2,587,801 2.13% 3.13
144 博茨瓦纳 2,333,201 1.81% 4.01
145 莱索托 2,263,010 1.33% 74.55
146 冈比亚 2,163,765 3.01% 202.43
147 马其顿 2,085,051 0.09% 81.09
148 斯洛文尼亚 2,081,260 0.06% 102.66
149 加蓬 2,067,561 2.09% 7.72
150 拉脱维亚 1,929,938 0.00% 29.89
151 几内亚比绍 1,907,268 2.47% 52.80
152 巴林 1,566,993 4.99% 2048.36
153 斯威士兰 1,391,385 1.76% 80.13
154 特立尼达和多巴哥 1,372,598 0.25% 267.56
155 东帝汶 1,324,094 2.14% 89.02
156 赤道几内亚 1,313,894 3.64% 46.84
157 爱沙尼亚 1,306,788 0.00% 28.89
158 毛里求斯 1,268,315 0.25% 621.72
159 塞浦路斯 1,189,085 0.81% 128.54
160 吉布提 971,408 1.51% 41.87
161 斐济 912,241 0.74% 49.93
162 留尼旺岛 883,247 0.76% 351.75
163 科摩罗 832,347 2.26% 447.02
164 不丹 817,054 1.17% 21.28
165 圭亚那 782,225 0.56% 3.64
167 黑山 629,219 0.04% 45.56
168 所罗门群岛 623,281 1.95% 21.57
169 卢森堡 590,321 1.18% 228.28
170 苏里南 568,301 0.87% 3.47
171 西撒哈拉 567,421 2.68% 2.13
172 佛得角 553,335 1.27% 137.20
173 瓜德罗普岛 449,173 0.00% 275.90
174 马尔代夫 444,259 1.82% 1480.86
175 文莱 434,076 1.25% 75.30
176 马耳他 432,089 0.29% 1367.37
177 巴哈马 399,285 0.99% 28.64
178 马提尼克 385,065 0.04% 341.37
179 伯利兹 382,444 2.07% 16.65
180 冰岛 337,780 0.82% 3.28
181 法属圭亚那 289,763 2.49% 3.47
182 巴巴多斯 286,388 0.23% 666.02
183 法属玻利尼西亚 285,859 1.01% 68.60
184 瓦努阿图 282,117 2.13% 23.15
185 新喀里多尼亚 279,821 1.29% 15.06
186 马约特岛 259,682 2.62% 694.34
187 圣多美和普林西比 208,818 2.20% 216.62
188 萨摩亚 197,695 0.64% 69.56
189 圣露西亚 179,667 0.46% 291.67
190 关岛 165,718 0.91% 301.85
191 库拉索 161,577 0.65% 363.91
192 基里巴斯 118,414 1.73% 146.01
193 圣文森特和格林纳丁斯 110,200 0.28% 283.29
194 汤加 109,008 0.91% 145.93
195 格林纳达 108,339 0.48% 314.94
196 密克罗尼西亚 106,227 0.65% 151.32
197 阿鲁巴岛 105,670 0.39% 587.06
198 美属维尔京群岛 104,914 0.01% 302.35
199 安提瓜和巴布达 103,050 1.02% 233.14
200 塞舌尔 95,235 0.53% 210.70
201 马恩岛 84,831 0.65% 148.31
202 安道尔 76,953 0.00% 164.43
203 多米尼克 74,308 0.52% 98.95
204 开曼群岛 62,348 1.28% 236.17
205 百慕大 61,070 0.00% 1130.93
206 格陵兰岛 56,565 0.15% 0.03
207 圣基茨和尼维斯 55,850 0.91% 213.98
208 美属萨摩亚 55,679 0.07% 279.79
209 北马里亚纳群岛 55,194 0.09% 118.95
210 马绍尔群岛 53,167 0.08% 293.74
211 法罗群岛 49,489 0.40% 35.53
212 圣马丁岛 40,552 1.08% 1192.71
213 摩纳哥 38,897 0.52% 19255.94
214 列支敦士登 38,155 0.61% 238.47
215 特克斯和凯科斯群岛 35,963 1.46% 37.94
216 直布罗陀 34,733 0.47% 5788.83
217 圣马力诺 33,557 0.47% 550.11
218 英属维尔京群岛 31,719 1.68% 210.06
219 帕劳 21,964 1.08% 47.85
220 库克群岛 17,411 0.18% 73.78
221 安圭拉 15,045 0.91% 165.33
222 瓦利斯和富图纳群岛 11,683 0.00% 82.27
223 瑙鲁 11,312 0.00% 538.67
Process finished with exit code 0
总结
Selenium是一个WEB自动化工具，可以用来做自动化测试、也可以用来做数据采集(如爬虫)，简单而又好用。
已归档，2021年 12月 26日 星期日 06:08:33 CST。
扫一扫
2万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
