Selenium4 RemoteWebDriver使用DevTools

zzz9527me

已于 2022-03-16 14:40:10 修改

阅读量2.5k

点赞数 2

分类专栏：学习文章标签： selenium docker

于 2022-03-07 15:59:51 首次发布

本文链接：https://blog.csdn.net/zcanjie/article/details/123332136

版权

学习专栏收录该内容

2 篇文章 0 订阅

订阅专栏

一、部署Selenium Hub 和 Node

这里主要采用docker方式进行部署，需要安装docker和docker-compose。下面是docker-compose.yaml 文件, 在服务器上部署安装即可。

docker-compose up -d

docker-compose.yaml 文件

version: "3"
services:
  chrome:
    image: selenium/node-chrome:4.1.2-20220217
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_MAX_INSTANCES=3
      - SE_NODE_MAX_SESSIONS=16
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
      - SE_NODE_GRID_URL=http://172.33.0.18:4444
      - SE_NODE_SESSION_TIMEOUT=300

  edge:
    image: selenium/node-edge:4.1.2-20220217
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
      - SE_NODE_MAX_INSTANCES=3
      - SE_NODE_MAX_SESSIONS=16
      - SE_NODE_GRID_URL=http://172.33.0.18:4444
      - SE_NODE_SESSION_TIMEOUT=300

  firefox:
    image: selenium/node-firefox:4.1.2-20220217
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
      - SE_NODE_MAX_INSTANCES=3
      - SE_NODE_MAX_SESSIONS=16
      - SE_NODE_GRID_URL=http://172.33.0.18:4444
      - SE_NODE_SESSION_TIMEOUT=300

  selenium-hub:
    image: selenium/hub:4.1.2-20220217
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"
    networks:
      default:
        ipv4_address: 172.33.0.18

networks:
  default:
    driver: bridge
    ipam:
      config:
       - subnet: "172.33.0.0/16"

二、查看部署情况

docker-compose logs -f --tail=all

三、web ui

在浏览器访问下面地址, 看是否可以打开。

注: 下面地址换成自己服务器的地址

http://192.168.1.80:4444

四、新建工程

1、pom 添加如下依赖

<dependency>
   <groupId>org.seleniumhq.selenium</groupId>
   <artifactId>selenium-api</artifactId>
   <version>4.1.2</version>
</dependency>
<dependency>
   <groupId>org.seleniumhq.selenium</groupId>
   <artifactId>selenium-chrome-driver</artifactId>
   <version>4.1.2</version>
</dependency>
<dependency>
   <groupId>org.seleniumhq.selenium</groupId>
   <artifactId>selenium-remote-driver</artifactId>
   <version>4.1.2</version>
</dependency>
<dependency>
   <groupId>org.seleniumhq.selenium</groupId>
   <artifactId>selenium-java</artifactId>
   <version>4.1.2</version>
</dependency>
<dependency>
   <groupId>net.bytebuddy</groupId>
   <artifactId>byte-buddy</artifactId>
   <version>1.12.4</version>
</dependency>
<dependency>
   <groupId>io.netty</groupId>
   <artifactId>netty-handler</artifactId>
   <version>4.1.70.Final</version>
</dependency>

五、测试代码

public class MyApp {

    private static final String URL = "http://192.168.1.80:4444/wd/hub/";

    public static void main(String[] args) throws MalformedURLException, InterruptedException {

        ChromeOptions chromeOptions = new ChromeOptions();

        WebDriver driver = new RemoteWebDriver(new URL(URL), chromeOptions);

        driver = new Augmenter().augment(driver);

        DevTools devTools = ((HasDevTools) driver).getDevTools();
        devTools.createSession();

        devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));
        devTools.addListener(Network.responseReceived(), res -> {
            System.out.println(res.getResponse().getUrl());
        });

        driver.get("https://www.baidu.com");

        /*JavascriptExecutor js = (JavascriptExecutor) driver;
        js.executeScript("window.scrollTo(0, document.body.scrollHeight)");*/

        Thread.sleep(10000);


        driver.quit();

    }
}

六、拦截network 请求，获取ResponseBody

DevTools devTools = ((HasDevTools) driver).getDevTools();
devTools.createSessionIfThereIsNotOne();
devTools.send(Network.enable(Optional.empty(), Optional.empty(), Optional.empty()));


devTools.addListener(Network.responseReceived(), res -> {
   Response response = res.getResponse();
   String url = response.getUrl();

   // 获取network 请求的responseBody
   String responseBody = devTools.send(Network.getResponseBody(res.getRequestId())).getBody();
});

driver.get("https://www.baidu.com");

driver.wait(Duration.ofSeconds(1).toMillis());