今天爬取的网页为:熊猫TV LOL栏
一,分析网页:
一共6页,我们要想办法获取所有页的信息,所以F12查看网络请求,点击第二页:
发现了这个请求
这就是我们需要的东西
所以推测
pageno参数为第几页
所以代码就好写了
二,代码实现
将需要的东西封装成类
class InfoClass:
def __init__(self, id, name, lookpeople):
self.name = name
self.id = id
self.lookpeople = int(lookpeople)
def __repr__(self):
return repr((self.id, self.name, self.lookpeople))
存所有主播信息:
allpeople = []
根据pagenum获取主播信息:
def getAll(pagenum):
url = "https://www.panda.tv/ajax_sort?token=&pageno=" + str(
pagenum) + "&pagenum=120&classification=lol&order=top&_=1542530305175"
html = requests.get(url)
#获取json数据
mjson = html.json()
#打包成字典
mdic = dict(mjson)
mlist = mdic['data']['items']
#遍历list加入所有主播信息
for item in mlist:
id = item['id']
name = item['name']
personNum = int(item['person_num'])
newP = InfoClass(id, name, personNum)
allpeople.append(newP)
获取每一页信息:
print("%-10s %-40s %-10s" % ("id房间号", "姓名", "观看人数"))
for pagenum in range(6):
getAll(pagenum+1)
排序:
#根据lookpeople降序排序
li = sorted(allpeople, key=lambda InfoClass: InfoClass.lookpeople, reverse=True)
for item in li:
print("%-10s %-40s %-10s" % (item.id, item.name, item.lookpeople))
至此代码完成。
完整代码:
#Create By ZZK
import requests
class InfoClass:
def __init__(self, id, name, lookpeople):
self.name = name
self.id = id
self.lookpeople = int(lookpeople)
def __repr__(self):
return repr((self.id, self.name, self.lookpeople))
allpeople = []
def getAll(pagenum):
url = "https://www.panda.tv/ajax_sort?token=&pageno=" + str(
pagenum) + "&pagenum=120&classification=lol&order=top&_=1542530305175"
html = requests.get(url)
# 获取json数据
mjson = html.json()
# 打包成字典
mdic = dict(mjson)
mlist = mdic['data']['items']
# 遍历list加入所有主播信息
for item in mlist:
id = item['id']
name = item['name']
personNum = int(item['person_num'])
newP = InfoClass(id, name, personNum)
allpeople.append(newP)
print("%-10s %-40s %-10s" % ("id房间号", "姓名", "观看人数"))
for pagenum in range(6):
getAll(pagenum + 1)
# 根据lookpeople降序排序
li = sorted(allpeople, key=lambda InfoClass: InfoClass.lookpeople, reverse=True)
for item in li:
print("%-10s %-40s %-10s" % (item.id, item.name, item.lookpeople))
运行结果GIF如下: