爬虫抓取网易云音乐民谣歌单，收听量排行

最新推荐文章于 2024-04-17 13:19:57 发布

th是个小屁孩

最新推荐文章于 2024-04-17 13:19:57 发布

阅读量6.1k

点赞数 5

分类专栏：爬虫文章标签：爬虫云音乐

本文链接：https://blog.csdn.net/u013775952/article/details/53584090

版权

爬虫专栏收录该内容

2 篇文章 0 订阅

订阅专栏

今天花了一点时间抓取了网易云音乐的热门民谣歌单，共1500热门民谣歌单，后续有时间会爬取其他分类。文末有源代码下载。

下面记录一下过程。

爬虫过程

1.首先抓取各个歌单的url与标题

public static void DoPachong( String url_str, String charset) throws ClientProtocolException, IOException{
            HttpClient hc = new DefaultHttpClient();
            HttpGet hg = new HttpGet(url_str);
            HttpResponse response = hc.execute(hg);
            HttpEntity entity = response.getEntity();


            InputStream htm_in = null;

            if(entity != null){
                htm_in = entity.getContent();
                String htm_str = InputStream2String(htm_in,charset);
                Document  doc =  Jsoup.parse(htm_str);
                Elements links= doc.select("div[class=g-bd]").select("div[class=g-wrap p-pl f-pr]").select("ul[class=m-cvrlst f-cb]").select("div[class=u-cover u-cover-1");
                for (Element link : links) {
                        Elements lin = link.select("a");  
                        String re_url = lin.attr("href");
                        String re_title = lin.attr("title");
                        re_url = "http://music.163.com"+re_url;
                        System.out.print(re_title+"       ");
                        System.out.print(re_url+"       ");
                        SecondPaChong(re_url,charset);
                }
            }
    }

2.根据抓取的url进一步用jsoup解析收听量

    public static void SecondPaChong( String url_str, String charset) throws ClientProtocolException, IOException{
        HttpClient hc = new DefaultHttpClient();
        HttpGet hg = new HttpGet(url_str);
        HttpResponse response = hc.execute(hg);
        HttpEntity entity = response.getEntity();

        InputStream htm_in = null;

        if(entity != null){
            htm_in = entity.getContent();
            String htm_str = InputStream2String(htm_in,charset);
            Document  doc =  Jsoup.parse(htm_str);
            String links= doc.select("div[class=u-title u-title-1 f-cb]").select("div[class=more s-fc3]").select("strong").text();
            System.out.println(links);

        }
}