001 爬取百度贴吧并保存

最新推荐文章于 2021-11-30 16:06:40 发布

Captain_Data

最新推荐文章于 2021-11-30 16:06:40 发布

阅读量245

点赞数

分类专栏： sql+python项目专栏

本文链接：https://blog.csdn.net/Captain_DUDU/article/details/102910070

版权

sql+python项目专栏专栏收录该内容

11 篇文章 0 订阅

订阅专栏

import requests
import time
def GetHtml(keywords,pages,timeSleep):





    url="http://tieba.baidu.com/f"
    kv={
        'User-Agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.132 Mobile Safari/537.36'
    }
    for i in range(0,pages):
        File_name = '贴吧第'+str(i+1)+'页' + '.html'
        i=i*50
        kv2={'kw' : keywords , 'pn' : i}
        time.sleep(timeSleep)
        r=requests.get(url,params=kv2,headers=kv)
        print(r.url)
        r.encoding="utf8"
        with open(File_name,'w',encoding="utf8") as baiduFile:
            baiduFile.write(r.text)
if __name__ == '__main__':
    while True:
        keywords=input("请输入搜索关键字：")
        pages=int(input("请输入需要获得的页数："))
        timeSleep=int(input('请输入每次爬取的间隔时间：'))
        GetHtml(keywords, pages, timeSleep)
        print("成功")