腾讯新闻爬取

最新推荐文章于 2024-08-03 15:58:00 发布

pickle Rick

最新推荐文章于 2024-08-03 15:58:00 发布

阅读量801

点赞数

分类专栏： Python爬虫文章标签： python html

本文链接：https://blog.csdn.net/weixin_44654458/article/details/105795666

版权

Python爬虫专栏收录该内容

4 篇文章 0 订阅

订阅专栏

腾讯新闻爬取

import time
from  selenium import webdriver
driver=webdriver.Chrome(executable_path="D:\chromedriver\chromedriver.exe")
driver.get("https://news.qq.com")
for i in range(1,100):
    time.sleep(2)
    driver.execute_script("window.scrollTo(window.scrollX, %d);"%(i*200))

from bs4 import BeautifulSoup
html=driver.page_source
bsObj=BeautifulSoup(html,"lxml")

jxtits=bsObj.find_all("div",{"class":"jx-tit"})[0].find_next_sibling().find_all("li")

print("index",",","title",",","url")
for i,jxtit in enumerate(jxtits):
     print(jxtit)
    
    try:
        text=jxtit.find_all("img")[0]["alt"]
    except:
        text=jxtit.find_all("div",{"class":"lazyload-placeholder"})[0].text
    try:
        url=jxtit.find_all("a")[0]["href"]
    except:
        print(jxtit)
    print(i+1,",",text,",",url)