python爬虫,发送请求需要携带cookies

此网站,需要先注册登录!!!

页面源代码:


<!DOCTYPE html><html lang="zh-Hans"><head><meta charset="UTF-8"><meta http-equiv="x-ua-compatible" content="IE=edge"><meta name="viewport" content="width=device-width, initial-scale=1"><meta name="renderer" content="webkit"><meta name="keywords" content="股票,炒股,股票资讯,行情,财经,证券,投资,金融,港股,美股,基金,债券,期货,外汇,科创板,保险,雪球财经,雪球,雪球股票,投资社区,雪球投资"><meta name="description" content="雪球,聪明的投资者都在这里 - 4300万投资者都在用的投资社区和财富管理平台,沪深、港股、美股全球市场实时行情,公募私募股票基金债券免费热点资讯,与投资高手实战交流。支持股票基金在线开户,炒股、投资理财低佣金,交易安全、方便、快捷。提供选基工具、基金估值工具、基金定投、基金排名、指数估值、投资组合供投资者使用参考。"><meta name="format-detection" content="telephone=no"><meta name="baidu-site-verification" content="codeva-Q0pduO3YRC"><meta content="always" name="referrer"><link href="//assets.imedao.com/ugc/js/vue-common-6118e8a4ff.js" rel="preload" as="script"><link href="//assets.imedao.com/ugc/js/vue-home-1033b4aeaf.js" rel="preload" as="script"><link href="//xqdoc.imedao.com/17aebcfb84a145d33fc18679.ico" rel="shortcut icon"><link rel="stylesheet" href="//assets.imedao.com/ugc/css/component/index-6a2a1e3e87.css"><title>我的首页&nbsp;-&nbsp;雪球</title><link rel="stylesheet" href="//assets.imedao.com/ugc/css/home-99631a21fd.css"><link rel="stylesheet" href="//assets.imedao.com/ugc/css/im-ca8ee8fac5.css"><script>window.SNOWMAN_USER = {"id":6608945068,"name":null,"province":null,"city":null,"location":null,"description":null,"url":null,"domain":null,"gender":"n","verified":false,"type":"1","step":"null","profile":"/6608945068","recommend":null,"intro":null,"status":1,"following":false,"blocking":false,"subscribeable":false,"remark":null,"constrained":0,"screen_name":"用户6608945068","created_at":1705136454711,"followers_count":0,"friends_count":0,"status_count":0,"last_status_id":0,"blog_description":null,"st_color":"1","stocks_count":0,"cube_count":0,"donate_count":0,"verified_type":0,"verified_description":null,"verified_realname":false,"stock_status_count":null,"follow_me":false,"allow_all_stock":false,"name_pinyin":null,"screenname_pinyin":null,"group_ids":null,"common_count":0,"recommend_reason":null,"verified_infos":[],"select_company_background":null,"select_company_banners":null,"privacy_agreement":null,"ip_location":"河南","reg_time":1705136454722,"photo_domain":"//xavatar.imedao.com/","profile_image_url":"community/default/avatar.png,community/default/avatar.png!180x180.png,community/default/avatar.png!50x50.png,community/default/avatar.png!30x30.png"};</script><script>window.SNOWMAN_LOGIN = 1;
//- fix: 登录但是uid 为空问题
if(window.SNOWMAN_USER && !window.SNOWMAN_USER.id){
  window.location.reload();
}</script><script>window.SNB = {
  staticDomain: 'https://assets.imedao.com/'
};</script></head><body><div id="app"><div class="nav__placeholder"></div><nav class="nav stickyFixed"><div class="container signed_container"><div class="nav__lf"><a class="nav__logo" href="/" data-analytics="31" data-analytics-data="{type: '雪球图标'}"></a><div class="nav__menu"><a class="nav__menu__item active" href="/" data-analytics="31" data-analytics-data="{type: '首页'}">首页</a><nav-essence></nav-essence><!--a.nav__menu__item(href="/ask/square", data-analytics='31', data-analytics-data="{type: '问答'}") 问答--><div class="nav__menu__item">行情<i></i><div class="nav__dropdown__container"><i></i><div class="nav__dropdown"><a href="/hq">行情中心</a><a href="/stock/screener">筛选器</a><a href="/hq/cnew?market=CN&amp;first_name=0&amp;second_name=4">新股上市</a><a href="/p/discover">买什么</a></div></div></div><div class="nav__menu__item">交易<i></i><div class="nav__dropdown__container"><i></i><div class="nav__dropdown"><a href="https://broker.xueqiu.com" target="_blank">A股交易</a><a href="https://danjuanfunds.com" target="_blank">基金交易</a><a href="https://danjuanfunds.com/rn/pf-shelves/home?login_type=1" target="_blank">私募中心</a></div></div></div></div><search></search></div><div class="nav__rt"><div class="nav__user-info__main"><div class="nav__user-info__username"><div class="nav__user-info__base"><a href="javascript:;"><i class="iconfont">&#xe6f6;</i><span class="user-name">用户6608945068</span><nav-msg></nav-msg><cs-button :isLogin="1"></cs-button></a></div></div><nav-info></nav-info></div><nav-publish></nav-publish></div></div></nav><div class="home__main"><div class="user__container"><div class="user__col--lf"><a class="avatar" href="/u/6608945068" target="_blank" data-tooltip="false"><img src="//xavatar.imedao.com/community/default/avatar.png!180x180.png"></a><div class="user__col--name">用户6608945068</div><table class="user__col--friends"><td><a href="/center/#/friends"><strong>0</strong><br><span>关注</span></a></td><td><a href="/center/#/fans"><strong>0</strong><br><span>粉丝</span></a></td></table><ul class="user__control__pannel"><li><i class="iconfont">&#xe659;</i><a href="/">首页</a></li><li><i class="iconfont">&#xe65a;</i><a href="/center/#/status">讨论</a></li><li class="performance"><i class="iconfont">&#xe651;</i><a href="/performance">持仓盈亏</a></li><li><i class="iconfont">&#xe650;</i><a href="/center/#/fav">收藏</a></li></ul><ul class="user__control__pannel"><li><i class="iconfont">&#xe653;</i><a href="/center/#/likes">赞我的</a></li><li><i class="iconfont">&#xe655;</i><a href="/center/#/comments">讨论我的</a></li><li><i class="iconfont">&#xe657;</i><a href="/center/#/mention">提到我的</a></li><li><i class="iconfont">&#xe654;</i><a href="/center/#/quiz">我收到的问题</a></li></ul><ul class="user__control__pannel"><li><i class="iconfont">&#xe669;</i><a href="/6608945068/column">我的专栏</a></li><li><i class="iconfont">&#xe64f;</i><a href="/center/#/wallet">我的钱包</a></li></ul><ul class="user__control__pannel"><li><i class="iconfont">&#xe60d;</i><a href="/center/#/report">投诉中心</a></li></ul><ul class="xueqiu__menu"><li><i class="iconfont">&#xe652;</i><a href="/about" target="_blank">关于雪球</a></li><li><i class="iconfont">&#xe658;</i><a href="/dh" target="_blank">财经导航</a></li><li><i class="iconfont">&#xe648;</i><a href="/invest-edu" target="_blank">投资者教育</a></li><li><img class="iconfont_img" src="https://xqimg.imedao.com/1887627038f25f273fe5dafc.png"><a href="/about/faq#5_0" target="_blank">风险提示</a></li></ul></div><div class="user__col--middle"><div class="home__stock-index"></div><div class="snb_skeleton" id="skeleton__home__stock-index"><div class="home__stock-index__box"><div class="home__stock-index__item"><p>上证指数</p><div class="info"><div class="current">————</div><div class="growth">————</div></div><a class="chart"></a></div><div class="home__stock-index__item"><p>深证成指</p><div class="info"><div class="current">————</div><div class="growth">————</div></div><a class="chart"></a></div><div class="home__stock-index__item"><p>创业板指</p><div class="info"><div class="current">————</div><div class="growth">————</div></div><a class="chart"></a></div></div></div><index-svg></index-svg><div class="optional optional"></div><div class="snb_skeleton" id="skeleton__optional"><div class="optional__tabs__controls"><a class="tabs__control">自选股票</a><a class="tabs__control">自选基金</a><a class="tabs__control">自选组合</a></div><div class="optional__tabs__contents"><table class="optional__tb optional_stocks"><tr><th class="optional__category"></th></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr><tr> <td></td></tr></table></div></div><optional></optional><div class="editor-container"><tiny-editor :config="{storage : true}"></tiny-editor></div><div class="snb_skeleton" id="skeleton__timeline"><div class="home-timeline"><div class="home-timeline-tabs"><a>全部</a><a>讨论</a><a>交易</a><a>新闻</a><a>公告</a></div><div class="status-list"><div class="timeline__item"><a class="avatar avatar-md"></a><div class="timeline__item__main"><div class="timeline__item__info"><div><a class="user-name">--------</a></div><a class="date-and-source">1分钟前 · 来自雪球</a></div><div class="timeline__item__bd"><div class="content content--description">------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div class="timeline__item__ft"><div class="timeline__item__control"></div></div></div></div><div class="timeline__item"><a class="avatar avatar-md"></a><div class="timeline__item__main"><div class="timeline__item__info"><div><a class="user-name">--------</a></div><a class="date-and-source">2分钟前 · 来自雪球</a></div><div class="timeline__item__bd"><div class="content content--description">------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div class="timeline__item__ft"><div class="timeline__item__control"></div></div></div></div><div class="timeline__item"><a class="avatar avatar-md"></a><div class="timeline__item__main"><div class="timeline__item__info"><div><a class="user-name">--------</a></div><a class="date-and-source">3分钟前 · 来自雪球</a></div><div class="timeline__item__bd"><div class="content content--description">------------------------------------------------------------------------------------------------------------------------------------------------------</div></div><div class="timeline__item__ft"><div class="timeline__item__control"></div></div></div></div></div></div></div><home-timeline></home-timeline></div><div class="home__col--rt"><ad-right-aside pos-id="right_top" position-id="10323020601" platform-id="0x05" page-id="0x05" mod-id="01"></ad-right-aside><side-nav></side-nav><today-topic-list></today-topic-list><talks-list></talks-list><stock-hot-list></stock-hot-list><most-profitable-list></most-profitable-list><recommend-user :classname="['recommend-user__container']"></recommend-user><div class="home__reward__entry board"><h3>悬赏提问<a class="more" href="/law/prize/faq" target="_blank">常见问题</a></h3><div class="home__reward__entry__bd"><p>有问题不知道问谁?<br>试试悬赏提问</p><a class="button button-outline-gold" href="">发起悬赏</a></div></div><div class="other-service__container board"><h3>其他服务</h3><table class="footer-right_list"><td><a href="/broker/open/pingan?from=foot" target="_blank">开户指南</a></td><td><a href="/operation/guide" target="_blank">使用指南</a></td><td><a href="/about/faq" target="_blank">常见问题</a></td><td><a href="/verify/index" target="_blank">认证申请</a></td></table><p>其他建议反馈欢迎点击&nbsp;<a href="https://xueqiu.com/hashtag/I-aIkee7membqueQg-aPkOW7uuiuriM=" target="_blank">#我给雪球提建议#</a></p><p>如受到欺诈信息骚扰请至&nbsp;<a href="https://xueqiu.com/hashtag/I-mYsuiviOmql-S4vuaKpeS4k-WMuiM=" target="_blank">#防诈骗举报专区#</a></p></div><side-report-entrance></side-report-entrance><ad-right-aside pos-id="right_bottom" position-id="10323020602" platform-id="0x05" page-id="0x05" mod-id="01"></ad-right-aside></div></div></div><page-footer></page-footer><div class="modals dimmer"><modals></modals><confirms></confirms><alert></alert><alert-pay-success></alert-pay-success></div><div class="toast"><i class="iconfont">&#xe630;</i><i class="iconfont error-icon">&#xe631;</i><i class="iconfont warn-icon">&#xe632;</i><i class="iconfont loading-icon">&#xe64a;</i><span></span></div><tooltip></tooltip><top :target="'.user__col--middle'" :reference="'.home__col--rt'"></top><upgrade-tip></upgrade-tip><widget-download-app></widget-download-app></div><script src="//assets.imedao.com/ugc/js/jquery-3-bf5be4b4bc.1.1.js" rel="preload" as="script"></script><script src="//assets.imedao.com/ugc/js/polyfill-7-1a461e6e65.4.4.js" rel="preload" as="script"></script><script src="//assets.imedao.com/ugc/js/raven-f24a7020e8.min.js"></script><script>Raven.config('https://9f2e0acfac5e4a3ba8fcc2cf22b69b11@sentry.xueqiu.com/29', {sampleRate: 0.2}).install()
Raven.setUserContext({
  id: SNOWMAN_USER.id || -1,
  username: SNOWMAN_USER.screen_name || '',
  ip_address: '123.149.30.241'
});</script><script src="//assets.imedao.com/ugc/js/medium-editor-bd85ca59df.js"></script><script src="//assets.imedao.com/ugc/js/page-b3a1971449.js"></script><script>var _hmt = _hmt || [];
(function() {
  var hm = document.createElement("script");
  hm.src = "//hm.baidu.com/hm.js?1db88642e346389874251b5a1eded6e3";
  var s = document.getElementsByTagName("script")[0];
  s.parentNode.insertBefore(hm, s);
})();</script><script src="//assets.imedao.com/ugc/js/apm-4-df4081db31.1.js"></script><script src="//assets.imedao.com/ugc/js/env-48c33adc63.js"></script><script>if (window.localStorage.getItem('snb-logout-flag') && window.needAutoOpenLoginModal()) {
  $.subscribe('vue-ready', function(e, bus){
    if (!window.SNOWMAN_LOGIN && !JSON.parse(window.localStorage.getItem('snb-modal-login-show') || 'false')) {
      bus.$emit('modal-login', 1, {
        trigger: 'quit'
      });
    }
  });
  window.localStorage.setItem('snb-logout-flag', false)
};
if(window && window.elasticApm){
  var apm = window.elasticApm.init({
    serviceName: 'snowman-web',
    serverUrl: 'https://open.xueqiu.com/es-apm',
    transactionSampleRate: 0.1,
  })
  var apmTransactionName = location.pathname;

  if (apmTransactionName.match(/^\/S\//i)) {
    //- 个股页
    apmTransactionName = 'stock pages';
  } else if (apmTransactionName.match(/^\/$/)){
    //- 首页
    apmTransactionName = 'home page';
  } else if (apmTransactionName.match(/^\/[0-9]{10}\/[0-9]{8}/)) {
    //- 文章页
    apmTransactionName = 'article pages';
  } else if (apmTransactionName.match(/^\/(u\/|[0-9]{10})/)) {
    //- 个人页
    apmTransactionName = 'profile pages';
  } else {
    //- 其他页面
    apmTransactionName = 'other pages';
  }
  apm.setInitialPageLoadName(apmTransactionName);
  if (window.SNOWMAN_LOGIN) {
    apm.setUserContext({
      id: window.SNOWMAN_USER.id,
      username: window.SNOWMAN_USER.screen_name
    })
  }
}</script><script src="//assets.imedao.com/ugc/js/vue-common-6118e8a4ff.js"></script><script src="//assets.imedao.com/ugc/js/vue-home-1033b4aeaf.js"></script><script src="//assets.imedao.com/ugc/js/underscore-dbf542885f.js"></script><script src="//assets.imedao.com/ugc/js/backbone-9c5159b315.js"></script><script src="//assets.imedao.com/ugc/js/im-8fcce4c918.js"></script><script>$.ajax('/snowman/f2e/statistics?from=snowman');
// 进入埋点
try {
  $(document).trigger('analytics', {
    page: 500,
    type: 0
  });
} catch (error) {
}</script></body></html>

python代码:

import requests

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36',
    'Host': 'xueqiu.com',
    'Referer': 'https://xueqiu.com/',
    'Cookie': 'cookiesu=411700482632361; device_id=29270365565b168f60a35e878597d7a3; s=am11lj11rm; __utmz=1.1700482786.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);acw_tc=2760825d17051362899667830e47536d0c8ad57ed00a73bbefb6c4b513e5ee; Hm_lvt_1db88642e346389874251b5a1eded6e3=1705136319; u=6608945068; snbim_minify=true; bid=4e57f453e306324b754929f18b8e3530_lrbu8ecs; __utma=1.2074947920.1700482786.1700482786.1705136477.2; __utmc=1; __utmt=1; remember=1; xq_a_token=34346c84cf49fc520a74b727e01a599d017d6960;xqat=34346c84cf49fc520a74b727e01a599d017d6960;xq_id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1aWQiOjY2MDg5NDUwNjgsImlzcyI6InVjIiwiZXhwIjoxNzA3NzI4NjM2LCJjdG0iOjE3MDUxMzY2MzYzNDYsImNpZCI6ImQ5ZDBuNEFadXAifQ.SEcNWjvbF4GwO-dVZY-AJxXJ3TBTqGqV_Xac5S034vvnx5cYicltppGpv4vfXjzFSdgMYhby1cLR0CzxAKjtbItJre1ZIz_814TC8v4524L0LajorvNNujiF_9FccrZ6Z1PT-GZ6l1Q8n22s3KxafEGM64Hr2VK6KoWJpwTzvSywD3SUwUoNTuHLV0Pm2fn9esdB9mcnCTZkRA3HiRlVh7XcBY7K8T9frLbMTG9ssXZ1JzDjaGwB07fAxRLteblFWWbelk8db77ZK066q7OlZ1bVexVBHdqkXu_vh2Snfud9f8x3_Sjj7a_byBgL5A5L5MJWpmrmZHqJlyKCXiLGQ; xq_r_token=2f87a9dd64211e24c310cb58f2169f0f6e91723c; xq_is_login=1; Hm_lpvt_1db88642e346389874251b5a1eded6e3=1705136778; __utmb=1.9.10.1705136477; is_overseas=0',
}
url = 'https://xueqiu.com/statuses/hot/listV2.json?since_id=-1&max_id=366242&size=15'

response = requests.get(url, headers=headers)
print(response.json())

运行结果:有数据

{'next_max_id': -1, 'items': [], 'next_id': -1}

此实例只是为了验证,某些网站反爬必须需要携带cookies。

  • 6
    点赞
  • 10
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
Python中进行爬虫时,可以使用requests库来携带cookie。首先,需要导入requests库,然后创建一个Session对象来处理cookie。可以通过将cookie存储到字典类型的变量中,并使用session的cookies属性进行设置。接下来,可以使用session对象发送POST请求携带cookie。可以使用session对象的post方法,将POST请求数据通过data参数传递进去。以下是一个示例代码: ``` import requests # 创建Session对象 session = requests.Session() # 设置cookie cookies = {'key1': 'value1', 'key2': 'value2'} session.cookies.update(cookies) # 发送POST请求携带cookie url = 'http://www.example.com/post' post_data = {'key1': 'value1', 'key2': 'value2'} response = session.post(url, data=post_data) ``` 通过以上步骤,你可以在Python爬虫携带cookie进行POST请求。<span class="em">1</span><span class="em">2</span><span class="em">3</span> #### 引用[.reference_title] - *1* *3* [Python | 一文解决如何使用Requests爬虫携带cookie发送post请求](https://blog.csdn.net/weixin_46264660/article/details/130206777)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT0_1"}}] [.reference_item style="max-width: 50%"] - *2* [python爬虫携带cookie访问QQ空间](https://blog.csdn.net/weixin_48964486/article/details/122408224)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v92^chatsearchT0_1"}}] [.reference_item style="max-width: 50%"] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值