Linux机器上Selenium+Python3+Chrome使用driver.get()只能获取到标签而没有内容的解决方法

最新推荐文章于 2024-04-15 08:58:53 发布

dcf869009372

最新推荐文章于 2024-04-15 08:58:53 发布

阅读量3.5k

点赞数 1

分类专栏： Python3 Selenium Linux 文章标签： linux selenium python google chrome google drive

本文链接：https://blog.csdn.net/dcf869009372/article/details/106271228

版权

在Linux环境中使用Selenium+Python3+Chrome，遇到driver.get()只能获取到HTML标签但无内容的问题。本文将介绍问题的现象及提供解决方案。

摘要由CSDN通过智能技术生成

代码：

#!/usr/bin/python3
#coding=utf8
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(chrome_options=chrome_options)

driver.get("http://toutiao.com/group/6552087122092753412") #获取头条网页源代码
print(driver.page_source)

driver.quit()

问题现象：

[root@qiufeng workspace]# python3 test.py
<html><head></head><body></body></html>
[root@qiufeng workspace]#

解决方法：

# 注释headless
# chrome_options.add_argument('--headless')
# 添加header
chrome_options.add_argument('user-agent="Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"')

结果：

[root@qiufeng workspace]# python3 test.py
<html><head><meta charset="utf-8"><title>成都街拍，夏季时尚穿搭参考，你更喜欢哪一款？</title><meta http-equiv="x-dns-prefetch-control" content="on"><link rel="dns-prefetch" href="//s3.pstatp.com/"><link rel="dns-prefetch" href="//s3a.pstatp.com/"><link rel="dns-prefetch" href="//s3b.pstatp.com"><link rel="dns-prefetch" href="//p1.pstatp.com/"><link rel="dns-prefetch" href="//p3.pstatp.com/"><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1"><meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,minimum-scale=1,user-scalable=no,minimal-ui"><meta name="360-site-verification" content="b96e1758dfc9156a410a4fb9520c5956"><meta name="360_ssp_verify" content="2ae4ad39552c45425bddb738efda3dbb"><meta name="google-site-verification" content="3PYTTW0s7IAfkReV8wAECfjIdKY-bQeSkVTyJNZpBKE"><meta name="shenma-site-verification" content="34c05607e2a9430ad4249ed48faaf7cb_1432711730"><meta name="baidu_union_verify" content="b88dd3920f970845bad8ad9f90d687f7"><meta name="domain_verify" content="pmrgi33nmfuw4ir2ej2g65lunfqw6ltdn5wselbcm52wszbchirdqyztge3tenrsgq3dknjume2tayrvmqytemlfmiydimddgu4gcnzcfqrhi2lnmvjwc5tfei5dcnbwhazdcobuhe2dqobrpu"><link rel="shortcut icon" href="//s3a.pstatp.com/toutiao/resource/ntoutiao_web/static/image/favicon_5995b44.ico" type="image/x-icon"><!--[if lt IE 9]>
  <p>您的浏览器版本过低，请<a href="http://browsehappy.com/">升级浏览器</a></p>
<![endif]--><script async="" src="https://i.snssdk.com/slardar/sdk.js?bid=toutiao_pc"></script><script>(function (i, s, o, g, r, a, m) { i["SlardarMonitorObject"] = r; i[r] = i[r] || function () { (i[r].q = i[r].q || []).push(arguments) }, i[r].l = 1 * new Date; a = s.createElement(o), m = s.getElementsByTagName(o)[0]; a.async = 1; a.src = g; m.parentNode.insertBefore(a, m); i[r].globalPreCollectError = function () { i[r]('precollect', 'error', arguments); }; if (typeof i.addEventListener === 'function') { i.addEventListener('error', i[r].globalPreCollectError, true) } })(window, document, "script", "https://i.snssdk.com/slardar/sdk.js?bid=toutiao_pc", "Slardar");</script><script>window.Slardar && window.Slardar('config', {
    sampleRate: 1,
    bid: 'toutiao_pc',
    pid: 'image_detail_new',
    ignoreAjax: [/\/action_log\//],
    ignoreStatic: [/\.tanx\.com\//, /\.alicdn\.com\//, /\.mediav\.com/]
  });</script><meta http-equiv="Content-Security-Policy" content="upgrade-insecure-requests"><meta name="pathname" content="toutiao_pc_image_detail_new"><meta name="keywords" content="今日头条，头条，头条网，头条新闻，今日头条官网"><meta name="description" content="《今日头条》(www.toutiao.com)是一款基于数据挖掘的推荐引擎产品，它为用户推荐有价值的、个性化的信息，提供连接人与信息的新型服务，是国内移动互联网领域成长最快的产品服务之一。"><link rel="stylesheet" href="//s3a.pstatp.com/toutiao/static/css/page/index_node/index.bffa4540d1972dd0bb44948d360bd13a.css"><script>!function(e){function t(a){if(o[a])return o[a].exports;var r=o[a]={exports:{},id:a,loaded:!1};return e[a].call(r.exports,r,r.exports,t),r.loaded=!0,r.exports}var a=window.webpackJsonp;window.webpackJsonp=function(n,c){for(var p,s,l=0,i=[];l<n.length;l++)s=n[l],r[s]&&i.push.apply(i,r[s]),r[s]=0;for(p in c)Object.prototype.hasOwnProperty.call(c,p)&&(e[p]=c[p]);for(a&&a(n,c);i.length;)i.shift().call(null,t);if(c[0])return o[0]=0,t(0)};var o={},r={0:0};t.e=function(e,a){if(0===r[e])return a.call(null,t);if(void 0!==r[e])r[e].push(a);else{r[e]=[a];var o=document.getElementsByTagName("head")[0],n=document.createElement("script");n.type="text/javascript",n.charset="utf-8",n.async=!0,n.src=t.p+"static/js/"+e+"."+{1:"8bc7f753ee0f99c80e55",2:"e982b8b6aaed0d57ade4",3:"29c993fa5751f3c931e7",4:"20e52a329a227ef9a27e"}[e]+".js",o.appendChild(n)}},t.m=e,t.c=o,t.p="/toutiao/",t.p="//s3.pstatp.com/toutiao/"}([]);</script><script src="//unpkg.pstatp.com/byted/sec_sdk_build/2.0.2/dist/captcha.js"></script><script src="//sf1-ttcdn-tos.pstatp.com/obj/ttfe/rc/acrawler.js"></script><script>window.byted_acrawler.init({
      aid: 24,
      dfp: true
    });</script><script>;(function () {
      var currentDate = new Date();
      var year = currentDate.getFullYear();
      var month = currentDate.getMonth() + 1;
      var date = currentDate.getDate();

      if (year === 2020 && month === 4 && date === 4) {
        try {
          document.documentElement.style.cssText += 'filter:progid:DXImageTransform.Microsoft.BasicImage(grayscale=1);-webkit-filter: grayscale(100%);';
        } catch (error) {
          console.log(error);
        }
      }
    })();</script><script src="https://unpkg.pstatp.com/byted/secsdk-captcha/2.8.13/build/captcha.js"></script><style type="text/css">@keyframes resizeanim { from { opacity: 0; } to { opacity: 0; } }
      .resize-triggers { animation: 1ms resizeanim; visibility: hidden; opacity: 0; }
      .resize-triggers, .resize-triggers > div, .contract-trigger:before { content: " "; display: block; position: absolute; top: 0; left: 0; height: 100%; width: 100%; overflow: hidden; }
      .resize-triggers > div { background: #eee; overflow: auto; }
      .contract-trigger:before { width: 200%; height: 200%; }</style><script src="https://s3a.pstatp.com/toutiao/slardar/sdk/plugins/browser/monitors.3.0.11.js" async="" crossorigin="anonymous"></script><script src="https://s3a.pstatp.com/toutiao/slardar/sdk/plugins/browser/sentry.3.0.11.js" async="" crossorigin="anonymous"></script><script charset="utf-8" src="https://unpkg.pstatp.com/byted/secsdk-captcha/2.8.13/build/static/js/2.js"></script><style data-styled="" data-styled-version="4.2.0"></style></head><body><div data-v-c9895306=""><div data-v-c9895306="" class="toutiao-header"><div class="topbar"><!----> <div class="bui-left clearfix"></div> <div class="bui-right"><ul class="user-nav-list clearfix"><!----> <!----> <li class="nav-login"><a ga_event="nav_login" class="tb-link">登录</a></li> <!----> <li><a href="//www.toutiao.com/complain/" ga_event="mh_nav_complain" target="blank" class="tb-link">侵权投诉</a></li> <li><div class="tt-dropdown"><a href="javascript:;" class="tt-dropdown-link tb-link">头条产品</a> <ul class="tt-dropdown-menu" style="display: none;"><li class="tt-dropdown-menu__item"><a href="https://www.wukong.com/" target="_blank" ga_event="mh_nav_others" class="layer-item">问答</a></li> <li class="tt-dropdown-menu__item"><a href="https://mp.toutiao.com/" target="_blank" ga_event="mh_nav_others" class="layer-item">头条号</a></li> <li class="tt-dropdown-menu__item"><a href="https://tuchong.com?utm_source=toutiao&amp;utm_medium=pc_header" target="_blank" ga_event="mh_nav_others" class="layer-item">图虫</a></li> <li class="tt-dropdown-menu__item"><a href="https://stock.tuchong.com/?source=ttweb" target="_blank" ga_event="mh_nav_others" class="layer-item">正版图库</a></li> <li class="tt-dropdown-menu__item"><a href="https://www.oceanengine.com/?source=pchometop" target="_blank" ga_event="mh_nav_ad" class="layer-item">广告投放</a></li> <li class="tt-dropdown-menu__item"><a href="https://www.dcdapp.com/?zt=tt_pc_home_top_bar" target="_blank" ga_event="mh_nav_others" class="layer-item">懂车帝</a></li></ul></div></li></ul></div></div> <div class="middlebar"><div class="middlebar-inner clearfix" style="width: 1120px;"><div class="bui-left logo-box"><a href="/" ga_event="go_home" class="logo-link"><img src="//s3.pstatp.com/toutiao/static/img/logo.271e845.png" class="logo"></a></div> <div class="bui-left chinese-tag"><a href="/" target="_blank" ga_event="click_index">首页</a>
        /
        <a href="/ch/news_image/" target="_blank" ga_event="click_channel">图片</a>
        /
        <span class="text">正文</span></div> <div ga_event="middlebar_search" class="bui-right"><div class="search-wrap"><div class="tt-autocomplete"><div class="tt-input tt-input-group tt-input-group--append"><!----> <!----> <input type="text" placeholder="搜索站内资讯、视频或用户" autocomplete="off" class="tt-input__inner"> <!----> <div class="tt-input-group__append"><button type="button" class="tt-button tt-button--default"><!----> <!----> <span>搜索</span></button></div></div> <div class="tt-autocomplete-suggestion" style="display: none;"><div class="tt-scrollbar"><div class="tt-autocomplete-suggestion__wrap tt-scrollbar__wrap" style="margin-bottom: -15px; margin-right: -15px;"><ul class="tt-scrollbar__view tt-autocomplete-suggestion__list" style="position: relative;"><div class="resize-triggers"><div class="expand-trigger"><div style="width: 1px; height: 1px;"></div></div><div class="contract-trigger"></div></div></ul></div><div class="tt-scrollbar__bar is-horizontal"><div class="tt-scrollbar__thumb" style="transform: translateX(0%);"></div></div><div class="tt-scrollbar__bar is-vertical"><div class="tt-scrollbar__thumb" style="transform: translateY(0%);"></div></div></div></div></div></div></div></div></div></div> <div data-v-c9895306="" class="bui-box container"><div data-v-c9895306="" class="galleryBox"><div class="gallery" style="width: 1000px;"><div class="bui-box gallery-inner"><div class="bui-left image-box" style="width: 658px;"><div class="J_imageList"><div class="imageList"><ul class="image-list" style="width: 5922px; margin-left: 0px;"><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 336px; height: 598px;"><img src="http://p1.pstatp.com/origin/pgc-image/1525526621497af92b9b23b" data-src="http://p1.pstatp.com/origin/pgc-image/1525526621497af92b9b23b" alt=""> <a href="http://p1.pstatp.com/origin/pgc-image/1525526621497af92b9b23b" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 397px; height: 598px;"><img src="http://p9.pstatp.com/origin/pgc-image/1525526623931c9d1b5124b" data-src="http://p9.pstatp.com/origin/pgc-image/1525526623931c9d1b5124b" alt=""> <a href="http://p9.pstatp.com/origin/pgc-image/1525526623931c9d1b5124b" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 336px; height: 598px;"><img data-src="http://p3.pstatp.com/origin/pgc-image/15255266244311e7a278379" alt=""> <a href="http://p3.pstatp.com/origin/pgc-image/15255266244311e7a278379" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 336px; height: 598px;"><img data-src="http://p3.pstatp.com/origin/pgc-image/15255266272593b9664a5c3" alt=""> <a href="http://p3.pstatp.com/origin/pgc-image/15255266272593b9664a5c3" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 336px; height: 598px;"><img data-src="http://p1.pstatp.com/origin/pgc-image/15255266276860e08ca59e4" alt=""> <a href="http://p1.pstatp.com/origin/pgc-image/15255266276860e08ca59e4" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 336px; height: 598px;"><img data-src="http://p3.pstatp.com/origin/pgc-image/1525526630962d63c09be26" alt=""> <a href="http://p3.pstatp.com/origin/pgc-image/1525526630962d63c09be26" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 397px; height: 598px;"><img data-src="http://p9.pstatp.com/origin/pgc-image/1525526634528ba6ff9d230" alt=""> <a href="http://p9.pstatp.com/origin/pgc-image/1525526634528ba6ff9d230" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="width: 336px; height: 598px;"><img data-src="http://p1.pstatp.com/origin/pgc-image/1525526638999d8508519cb" alt=""> <a href="http://p1.pstatp.com/origin/pgc-image/1525526638999d8508519cb" title="查看原图" target="_blank" ga_event="view_original_photo" class="image-origin"><i class="bui-icon icon-enlarge" style="font-size: 14px; color: rgb(255, 255, 255);"></i></a></div></li><li class="image-item" style="width: 658px;"><div class="image-item-inner" style="w

最低0.47元/天解锁文章

dcf869009372

关注

1
点赞
踩
1

收藏

觉得还不错? 一键收藏
1
评论
Linux机器上Selenium+Python3+Chrome使用driver.get()只能获取到标签而没有内容的解决方法

Linux机器上Selenium+Python3+Chrome使用driver.get()只能获取到标签<html><head></head><body></body></html>而没有内容的解决方法
复制链接

扫一扫