import requests
url = "https://www.cnblogs.com/AggSite/AggSitePostList"
data = { "CategoryType" : "SiteHome" ,
"ParentCategoryId" : 0 ,
"CategoryId" : 808 ,
"PageIndex" : 1 ,
"TotalPostCount" : 4000 ,
"ItemListActionName" : "AggSitePostList" }
headers = {
"content-type" : "application/json; charset=UTF-8" ,
"user-agent" : "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36" ,
}
import json
resp = requests. post( url, data= json. dumps( data) , headers= headers)
resp
<Response [200]>
resp. headers
{'Date': 'Tue, 15 Mar 2022 11:31:22 GMT', 'Content-Type': 'text/html; charset=utf-8', 'Transfer-Encoding': 'chunked', 'Connection': 'keep-alive', 'Vary': 'Accept-Encoding', 'Strict-Transport-Security': 'max-age=2592000; includeSubDomains; preload', 'Content-Encoding': 'gzip'}
html = resp. text
from bs4 import BeautifulSoup
soup = BeautifulSoup( html, "html.parser" )
articles = soup. find_all( "article" , class_= "post-item" )
for article in articles:
link = article. find( "a" , class_= "post-item-title" )
title = link. get_text( )
href = link. get( "href" )
a_links = article. find_all( "a" )
icon_views = 0
icon_comment = 0
icon_digg = 0
for a_link in a_links:
if "icon_views" in str ( a_link) :
icon_views = a_link. find( "span" ) . get_text( )
if "icon_comment" in str ( a_link) :
icon_comment = a_link. find( "span" ) . get_text( )
if "icon_digg" in str ( a_link) :
icon_digg = a_link. find( "span" ) . get_text( )
print ( title, href, icon_views, icon_comment)
自己动手写Vector【Cherno C++教程】 https://www.cnblogs.com/zhangyi1357/p/16009968.html 0 0
Seastar 教程(三) https://www.cnblogs.com/morningli/p/15961636.html 9 0
.NET CORE 授权 https://www.cnblogs.com/yuxl01/p/15991851.html 47 0
对前端“价值”的理解 https://www.cnblogs.com/HuiTaiLang1216/p/16009474.html 95 0
浅析XML https://www.cnblogs.com/hjk-airl/p/16009344.html 44 0
[翻译] TensorFlow 分布式之论文篇 "Implementation of Control Flow in TensorFlow" https://www.cnblogs.com/rossiXYZ/p/16009379.html 21 0
iCloud开发: key-value Storage,CloudKit,iCloud Documents https://www.cnblogs.com/songliquan/p/16009342.html 27 0
微服务7:通信之RPC https://www.cnblogs.com/wzh2010/p/15642251.html 53 0
Lua中如何实现类似gdb的断点调试—08支持通过包名称添加断点 https://www.cnblogs.com/logchen/p/16009256.html 29 0
spring源码干货分享-对象创建详细解析(set注入和初始化) https://www.cnblogs.com/w001/p/16009244.html 17 0
sql server数据库如何存储数组,int[]float[]double[]数组存储到数据库方法 https://www.cnblogs.com/zhaimaojun/p/16009113.html 77 2
【图文并茂】 做开发这么久了,还不会搭建服务器Maven私有仓库?这也太Low了吧 https://www.cnblogs.com/binghe001/p/16008737.html 243 0
关于分布式锁的那些事儿 https://www.cnblogs.com/zhaorongbiao/p/16005116.html 282 2
一比一还原axios源码(零)—— 概要 https://www.cnblogs.com/zaking/p/15876002.html 175 0
二进制部署1.23.4版本k8s集群-6-部署Node节点服务 https://www.cnblogs.com/wgh2008/p/16008188.html 58 0
字节一面:go的协程相比线程,轻量在哪? https://www.cnblogs.com/JulianHuang/p/16008107.html 203 1
Android 12(S) 图形显示系统 - BufferQueue/BLASTBufferQueue之初识(六) https://www.cnblogs.com/roger-yu/p/15773008.html 50 1
HBase海量数据高效入仓解决方案 https://www.cnblogs.com/vivotech/p/16007922.html 178 1
从刘维尔方程到Velocity-Verlet算法 https://www.cnblogs.com/dechinphy/p/liouville.html 83 0
Spring Bean生命周期,好像人的一生。。 https://www.cnblogs.com/three-fighter/p/16007800.html 244 2