python调用selenium的get,是否可以在python中并行化Selenium Webdriver get_attribute调用？...

I am running this code

from multiprocessing.Pool import ThreadPool

from selenium import webdriver

driver = webdriver.Firefox()

driver.get(url)

elements = driver.find_elements_by_class_name("class-name")

pool = ThreadPool(4)

async = [pool.apply_async(fn_which_calls_get_attribute,(element,)) for element in elements]

results = [result.get() for result in async]

which works fine for some of the results, but throws an error of ResponseNotReady for other results. It runs as expected if I use "pool.apply" instead of the async version.

Is it a problem that I am making multiple calls to the selenium driver at once, and the error is because it cannot handle it? Or is something wrong with my parallelization?

解决方案

Just Hint that Selenium run in a single thread and in a single core system. So its not possible to exercise multi-threading over selenium webdriver . Yes you can create a separate instance and attach to another core of a multi core system.

I may not answer your question but if you are trying to do something similar good not to do.