多进程使用docker容器中的selenium

启动容器

docker run -it --rm --shm-size="4g" -p 4444:4444 selenium/standalone-chrome:3.141.59
适当增加--shm-size="4g"大小，默认为64M，会抛出以下错误

Traceback (most recent call last):
  File "C:\Program Files\Python38\lib\multiprocessing\process.py", line 315, in _bootstrap
    self.run()
  File "C:\Program Files\Python38\lib\multiprocessing\process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "D:\workspace\test\test_selenium.py", line 18, in test
    driver.get("https://www.baidu.com")
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 333, in get
    self.execute(Command.GET, {'url': url})
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Program Files\Python38\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
  (Session info: chrome=92.0.4515.107)

代码示例

import time

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from multiprocessing import Process

driver = webdriver.Remote(
    command_executor="http://localhost:4444/wd/hub",
    desired_capabilities=DesiredCapabilities.CHROME
)


def test(i):
    time.sleep(3)
    driver.get("https://www.baidu.com")
    print(driver.title, i)


if __name__ == '__main__':

    p_list = []
    for i in range(5):
        p_list.append(Process(target=test, args=(i,)))

    for p in p_list:
        p.start()

    for p in p_list:
        p.join()

    driver.close()