贴吧爬虫
import requests
class TiebaSpider:
def __init__(self,tieba_name):
self.tieba_name = tieba_name
# pass
self.url_temp="http://tieba.baidu.com/f?kw="+tieba_name+"&ie=utf-8&pn={}" # 需下方选择第1页,url修改tieba_name与页码改为{}
self.headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36"}
def get_url_list(self): # 1 构造url列表
url_list=[]
for i in range(10): #设置爬取个数
url_list.append(self.url_temp.format(i*50))
return url_list
def parse_url(self,url): # 发送请求,获取响应
print(url)
response = requests.get(url,headers=self.headers)
return response.content.decode()
def save_html(self,html_str,page_num):
file_path = "{}-第{}页.html".format(self.tieba_name,page_num)
with open (file_path,"w", encoding="utf-8") as f:
f.write(html_str)
def run(self): # 实现主要逻辑
# 1 构造url列表;
url_list = self.get_url_list()
# 2 遍历,发送请求,获取响应;
for url in url_list:
html_str = self.parse_url(url) # 相应html字符串
# 3 保存
page_num = url_list.index(url)+1 # 页码数
self.save_html(html_str,page_num)
if __name__ == "__main__":
tieba_spider = TiebaSpider("lol") #类的实例化(设置传入的参数)
tieba_spider.run()
运行结果:
附作者相关研究:
Python:
Python实现线性回归、逻辑回归、KNN、SVM、朴素贝叶斯、决策树、K-Means7种机器学习算法的经典案例——亲测可用,链接
https://download.csdn.net/download/qq_35778860/20715889
Python实现飞机大战的完整代码——亲测可用,链接
https://download.csdn.net/download/qq_35778860/57165535
Python爬虫获取豆瓣网评分Top200的电影——亲测可用,链接
https://download.csdn.net/download/qq_35778860/56379309
Python根据下拉选项绘制雷达图和柱形图(异常预控平台)——亲测可用,链接
https://download.csdn.net/download/qq_35778860/20715210
Python根据下拉选项绘制雷达图和柱形图(异常预控平台)——亲测可用,链接
https://download.csdn.net/download/qq_35778860/20715210
python实现雷达图——亲测可用,链接
https://download.csdn.net/download/qq_35778860/20695215
基于python的北京房屋出租数据可视化分析与3D展示——亲测可用,链接
https://download.csdn.net/download/qq_35778860/20675051
基于Python设计的web接口聊天机器人
https://download.csdn.net/download/qq_35778860/66662593
用Python进行自然语言处理(中文).rar
https://download.csdn.net/download/qq_35778860/63652414
《机器学习实战》全书python代码——很全很完整
https://download.csdn.net/download/qq_35778860/62203534
Python3实现十大排序算法
https://download.csdn.net/download/qq_35778860/61630851
C#
成绩管理系统(C/S结构的应用系统,含access数据库)——C#实现
https://download.csdn.net/download/qq_35778860/66967387
C#实现学生信息管理系统(包括增删改查功能)——亲测可用
https://download.csdn.net/download/qq_35778860/61620149
Matlab
极限学习机ELM+OSELM+KELM+半监督SSELM+USELM的matlab程序(附完整代码)
https://download.csdn.net/download/qq_35778860/56378533
P+近邻KNN+LS最小二乘算法的matlab代码——亲测可用(含实例)
https://download.csdn.net/download/qq_35778860/56378309
基于matlab的卷积神经网络实现手写数字识别
https://download.csdn.net/download/qq_35778860/66667747
基于MATLAB的人工蜂群算法
https://download.csdn.net/download/qq_35778860/66665944
利用MATLABsilulink搭建有源电力滤波器仿真.zip
https://download.csdn.net/download/qq_35778860/63659150
动态规划MATLAB程序.zip
https://download.csdn.net/download/qq_35778860/63658838
基于matlab的模式识别基础实例源代码.zip
https://download.csdn.net/download/qq_35778860/63658486
Matlab_动态窗口法实现机器人在障碍环境下的模拟避碰仿真.zip
https://download.csdn.net/download/qq_35778860/63658232
bp神经网络整定pid参数matlab应用程序.rar
https://download.csdn.net/download/qq_35778860/63656174
利用深度学习的matlab程序编码仿真实现图像分割.zip
https://download.csdn.net/download/qq_35778860/63655267
Java
基于java的信息管理系统
https://download.csdn.net/download/qq_35778860/66658535
Java swing学生成绩系统(源码+数据库脚本).rar
https://download.csdn.net/download/qq_35778860/63653951
Jsp
jsp学生信息管理系统(源码+数据库脚本).zip
https://download.csdn.net/download/qq_35778860/63653655
SCN
随机配置网络SCN实现的matlab代码——亲测可用
https://download.csdn.net/download/qq_35778860/61636684
BLS
宽度学习BLS的matlab代码+Mnist数据集
https://download.csdn.net/download/qq_35778860/61635045
Mysql
Mysql增删改查代码操作,很全很完整——亲测可用
https://download.csdn.net/download/qq_35778860/61626574
C
基于单片机的交通灯控制系统设计(含代码,原理图)——C语言实现
https://download.csdn.net/download/qq_35778860/60383113
基于单片机的温度控制系统设计代码(含代码,原理图)——C语言实现
https://download.csdn.net/download/qq_35778860/60379923
郭天祥51单片机书中400例程——亲测可用
https://download.csdn.net/download/qq_35778860/58014293
其他
微信转盘抽奖小程序源码.rar
https://download.csdn.net/download/qq_35778860/63652890