linux centos7 关于Python模拟浏览器踩坑（转载）

https://cloud.tencent.com/developer/article/1404558

1、腾讯云服务器

2、Linux VM_0_10_centos 3.10.0-514.26.2.el7.x86_64 #1 SMP Tue Jul 4 15:04:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

待运行python脚本：

from selenium import webdriver

option = webdriver.ChromeOptions()
option.add_argument(‘headless’)
browser = webdriver.Chrome(chrome_options=option)

browser.get(‘http://www.baidu.com/’)
print(browser.title)
exit();
安装详细过程：

cd /usr/local/share
wget -N http://chromedriver.storage.googleapis.com/2.26/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
chmod +x chromedriver
ln -s /usr/local/share/chromedriver /usr/local/bin/chromedriver
ln -s /usr/local/share/chromedriver /usr/bin/chromedriver

yum install python-pip

pip install selenium
执行，python脚本，报错：

[root@VM_0_10_centos pythonproj]# python webdriver.py
Traceback (most recent call last):
File “webdriver.py”, line 3, in
browser = webdriver.Chrome()
File “/usr/lib/python2.7/site-packages/selenium/webdriver/chrome/webdriver.py”, line 73, in init
self.service.start()
File “/usr/lib/python2.7/site-packages/selenium/webdriver/common/service.py”, line 98, in start
self.assert_process_still_running()
File “/usr/lib/python2.7/site-packages/selenium/webdriver/common/service.py”, line 111, in assert_process_still_running
% (self.path, return_code)
selenium.common.exceptions.WebDriverException: Message: Service chromedriver unexpectedly exited. Status code was: 127
手动查看 chromedriver 版本

[root@VM_0_10_centos pythonproj]# chromedriver --version
chromedriver: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory
安装chrome浏览器：

yum install chromium
执行，还是报错：

[root@VM_0_10_centos ~]# chromedriver
chromedriver: error while loading shared libraries: libgconf-2.so.4: cannot open shared object file: No such file or directory
搜索，发现这个so文件问题，是少了这个库GConf2，http://landcareweb.com/questions/25967/que-shao-gong-xiang-ku-libgconf-2-so-4

安装这个库：

yum install GConf2
再执行 webdriver，搞定！

[root@VM_0_10_centos ~]# chromedriver
Starting ChromeDriver 2.26.436382 (70eb799287ce4c2208441fc057053a5b07ceabac) on port 9515
Only local connections are allowed.
再执行pathon脚本，报错：

selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.26.436382 (70eb799287ce4c2208441fc057053a5b07ceabac),platform=Linux 3.10.0-514.26.2.el7.x86_64 x86_64)
查看版本问题：

[root@VM_0_10_centos pythonproj]# yum list installed | grep chro
chromium.x86_64 71.0.3578.98-2.el7 @epel
[root@VM_0_10_centos pythonproj]# chromedriver --version
ChromeDriver 2.26.436382 (70eb799287ce4c2208441fc057053a5b07ceabac)
好像是版本不一致的问题，看这个：https://www.jianshu.com/p/40027de48c5b

ChromeDriver的版本太低了，跟安装的浏览器版本搭不上。

重新安装下 ChromeDriver版本 ：v2.45

cd /usr/local/share
wget -N http://chromedriver.storage.googleapis.com/2.45/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
再执行脚本，依旧报错：

selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn’t exist)
(The process started from chrome location /usr/bin/chromium-browser is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
(Driver info: chromedriver=2.45.615279 (12b89733300bd268cff3b78fc76cb8f3a7cc44e5),platform=Linux 3.10.0-514.26.2.el7.x86_64 x86_64)
不过，这次错误明显好很多。

搜一下，找到解决办法：https://github.com/heroku/heroku-buildpack-google-chrome/issues/46

Don’t think it’s has a heroku issue. Had a similar issue with docker container as well. Here is my fix that worked in docker -

`
chrome_options = Options()
chrome_options.add_argument(’–headless’)
chrome_options.add_argument(’–no-sandbox’)
chrome_options.add_argument(’–disable-dev-shm-usage’)
driver = webdriver.Chrome(’/path/to/your_chrome_driver_dir/chromedriver’,chrome_options=chrome_options)
修改python脚本如下：

from selenium import webdriver

option = webdriver.ChromeOptions()
option.add_argument(‘headless’)
option.add_argument(‘no-sandbox’)
option.add_argument(‘disable-dev-shm-usage’)
browser = webdriver.Chrome(’/usr/local/bin/chromedriver’,chrome_options=option)

browser.get(‘http://www.baidu.com/’)
print(browser.title)
exit();
再执行，成功！

运行一段时候后，又报错：

selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn’t exist
(Driver info: chromedriver=2.45.615279 (12b89733300bd268cff3b78fc76cb8f3a7cc44e5),platform=Linux 3.10.0-514.26.2.el7.x86_64 x86_64)
查看进程：

[root@VM_0_10_centos ~]#
[root@VM_0_10_centos ~]#
[root@VM_0_10_centos ~]# ps -ef | grep chrome
root 2947 1 0 12:51 ? 00:00:00 /usr/local/bin/chromedriver --port=40245
root 3615 1 0 12:55 ? 00:00:00 /usr/local/bin/chromedriver --port=45273
root 5809 1 0 13:10 pts/0 00:00:01 /usr/local/bin/chromedriver --port=44337
root 8637 5268 0 13:30 pts/0 00:00:00 grep --color=auto chrome
root 21378 1 0 11:11 ? 00:00:01 /usr/local/bin/chromedriver --port=47677
root 25024 1 0 Mar18 ? 00:00:00 /usr/local/bin/chromedriver --port=38096
root 25346 1 0 Mar18 ? 00:00:00 /usr/local/bin/chromedriver --port=42048
[root@VM_0_10_centos ~]#
[root@VM_0_10_centos ~]#
[root@VM_0_10_centos ~]# killall chromedriver
[root@VM_0_10_centos ~]# ps -ef | grep chrome
root 8674 5268 0 13:31 pts/0 00:00:00 grep --color=auto chrome
不知道是不是这影响，，，代码里面没有browser.quit()，如上，kill chromedriver 掉再试试。

还是报错，继续查，

[root@VM_0_10_centos ~]# ps -ef | grep driver
root 2956 1 0 12:51 ? 00:00:02 /usr/bin/chromium-browser --enable-plugins --enable-extensions --enable-user-scripts --enable-printing --enable-gpu-rasterization --enable-sync --auto-ssl-client-auth --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-dev-shm-usage --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-web-resources --enable-automation --enable-logging --force-fieldtrials=SiteIsolationExtensions/Control --headless --ignore-certificate-errors --load-extension=/tmp/.org.chromium.Chromium.dteWvh/internal --log-level=0 --metrics-recording-only --no-first-run --no-sandbox --password-store=basic --remote-debugging-port=0 --test-type=webdriver --use-mock-keychain --user-data-dir=/tmp/.org.chromium.Chromium.tOyVfN data:,
… 一大堆这个进程
[root@VM_0_10_centos ~]# killall chromium-browser
再重试，好了。

[root@VM_0_10_centos ~]# killall chromedriver
[root@VM_0_10_centos ~]# killall chromium-browser
[root@VM_0_10_centos ~]#
[root@VM_0_10_centos ~]#
[root@VM_0_10_centos ~]# ps -ef | grep chro
root 13735 13285 0 14:08 pts/0 00:00:00 grep --color=auto chro
脚本退出时，一定要主动调用 driver.quit !!!

修改脚本：

from selenium import webdriver

option = webdriver.ChromeOptions()
option.add_argument(‘headless’)
option.add_argument(‘no-sandbox’)
option.add_argument(‘disable-dev-shm-usage’)
browser = webdriver.Chrome(’/usr/local/bin/chromedriver’,chrome_options=option)

browser.get(‘http://www.baidu.com/’)
print(browser.title)
browser.quit() # 这个一定要加。加了就没上面的问题了。