python爬取世界大学排行榜

最新推荐文章于 2023-05-02 17:00:24 发布

Emilyzhai

最新推荐文章于 2023-05-02 17:00:24 发布

阅读量867

点赞数

文章标签： python 爬虫数据分析

本文链接：https://blog.csdn.net/zhaipupu/article/details/122212045

版权

#python爬取世界大学排行榜，使用bs4解析页面并写进csv文件
import requests
from bs4 import BeautifulSoup
import csv
f = open("data2.csv",'w',encoding="utf-8")
csvwriter = csv.writer(f)
url= 'https://www.maigoo.com/news/603854.html'
headers = {
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
}
res = requests.get(url,verify=False,headers=headers)#取消安全验证

page = BeautifulSoup(res.text,"html.parser") #指定html解析器
#就两个函数
#find(标签，属性=值)
#find_all(标签，属性=值)
table = page.find("table",attrs={"class":"mod_table table1 fcolor30"})
trs = table.find_all('tr')[1:]
for tr in trs:
    tds = tr.find_all('td')
    rank = tds[0].text
    name = tds[1].text
    country = tds[2].text
    csvwriter.writerow([rank,name,country])

    print(rank,name,country)
f.close()