Java爬虫csdn主页文章列表

馆长先生

已于 2023-08-18 10:28:12 修改

阅读量553

点赞数

文章标签： java 爬虫开发语言

于 2023-08-17 19:55:13 首次发布

本文链接：https://blog.csdn.net/weixin_44668443/article/details/132348134

版权

引入依赖

        <dependency>
            <groupId>org.jsoup</groupId>
            <artifactId>jsoup</artifactId>
            <version>1.14.3</version>
        </dependency>

创建Java类

    private static final String URL = "https://blog.csdn.net/weixin_44668443?type=blog";

    /**
     * 爬取CSDN博客列表
     *
     * @param args
     */
    public static void main(String[] args){
        try {
            Document document = Jsoup.connect(URL).get();
            Elements articleElements = document.select("article.blog-list-box[data-v-6fe2b6a7]");
            articleElements.forEach(articleElement -> {
                Element h4Element = articleElement.selectFirst("div.blog-list-box-top h4[data-v-6fe2b6a7]");
                Element linkElement = articleElement.selectFirst("a[data-v-6fe2b6a7]");
                if (h4Element != null && linkElement != null) {
                    System.out.println("title: " + h4Element.text());
                    System.out.println("url: " + linkElement.attr("href"));
                }
            });
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }