控制selenium爬取网页的加载内容

对比：

1. 不加以限制措施

   start = time.perf_counter()
   from selenium import webdriver
   fp = webdriver.FirefoxProfile()
   driver = webdriver.Firefox(firefox_profile=fp, executable_path = r'C:\Users\Shinelon\Downloads\geckodriver-v0.26.0-win64\geckodriver.exe')
   driver.get("http://www.santostang.com/2018/07/04/hello-world/")
   end = time.perf_counter()
   interval = end - start
   print('time_common:', interval)
   
   time_common: 15.465049999998882
   

2. 控制CSS的加载

   start = time.perf_counter()
   from selenium import webdriver
   fp = webdriver.FirefoxProfile()
   fp.set_preference("permissions.default.stylesheet",2)
   driver = webdriver.Firefox(firefox_profile=fp, executable_path = r'C:\Users\Shinelon\Downloads\geckodriver-v0.26.0-win64\geckodriver.exe')
   driver.get("http://www.santostang.com/2018/07/04/hello-world/")
   end = time.perf_counter()
   interval = end - start
   print('cost_CSS:', interval)
   
   cost_CSS: 21.29137950000586
   

3. 控制图片文件的显示

   start = time.perf_counter()
   from selenium import webdriver
   fp = webdriver.FirefoxProfile()
   fp.set_preference("permissions.default.image",2)
   driver = webdriver.Firefox(firefox_profile=fp, executable_path = r'C:\Users\Shinelon\Downloads\geckodriver-v0.26.0-win64\geckodriver.exe')
   driver.get("http://www.santostang.com/2018/07/04/hello-world/")
   end = time.perf_counter()
   interval = end - start
   print('cost_image:', interval)
   
   cost_image: 15.086580300005153
   

4. 控制JavaScript的执行

   start = time.perf_counter()
   from selenium import webdriver
   fp = webdriver.FirefoxProfile()
   fp.set_preference("javascript.enabled", False)
   driver = webdriver.Firefox(firefox_profile=fp, executable_path = r'C:\Users\Shinelon\Downloads\geckodriver-v0.26.0-win64\geckodriver.exe')
   driver.get("http://www.santostang.com/2018/07/04/hello-world/")
   end = time.perf_counter()
   interval = end - start
   print('cost_JS:', interval)
   
   cost_JS: 22.82997449999675
   

5. CSS、图片、JavaScript均加以控制

   start = time.perf_counter()
   from selenium import webdriver
   fp = webdriver.FirefoxProfile()
   fp.set_preference("permissions.default.stylesheet",2)
   fp.set_preference("permissions.default.image",2)
   fp.set_preference("javascript.enabled", False)
   driver = webdriver.Firefox(firefox_profile=fp, executable_path = r'C:\Users\Shinelon\Downloads\geckodriver-v0.26.0-win64\geckodriver.exe')
   driver.get("http://www.santostang.com/2018/07/04/hello-world/")
   end = time.perf_counter()
   interval = end - start
   print('cost_all:', interval)
   
   cost_all: 17.23865569999907
   

尝试加载该博客50次的结果：