网易云爬虫
今日发现qq空间里面的长按评论自动生成网易云评论的功能么得了,很可惜,觉得有必要写一个网易云音乐评论的爬虫。
之前的爬虫大部分失效,只能自力更生,改进了部分之前失效的api可以沿用的就继续使用。
import requests
from pyquery import PyQuery as pq
import pandas as pd
import random
import time
from lxml import etree
import json
from pandas.core.frame import DataFrame
headers = {
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36'}
def scrape_index(url):
url = 'https://music.163.com/discover/playlist/?order=hot&cat=%E5%8D%8E%E8%AF%AD&limit=35&offset=1'
print(url)
response = requests.get(url,headers = headers)
html = etree.HTML(response.content)
name_list = html.xpath(