爬虫

最新推荐文章于 2024-10-06 18:19:52 发布

qq_34461875

最新推荐文章于 2024-10-06 18:19:52 发布

阅读量109

点赞数

分类专栏：爬虫文章标签：爬虫

本文链接：https://blog.csdn.net/qq_34461875/article/details/112117763

版权

爬虫专栏收录该内容

1 篇文章 0 订阅

订阅专栏

资料来源：B站狂神说

Java爬取网页

jar包

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.10.2</version>
</dependency>

代码

private List parse(String key) throws IOException {
    String url = "https://search.jd.com/Search?keyword=" + key + "&enc=utf-8&wq=java&pvid=c4b49ddc8c4543e48c2028e565c83a2d";
    Document document = Jsoup.parse(new URL(url), 30000);
    Element element = document.getElementById("J_goodsList");
    //System.out.println(element.html());
    Elements tags = element.getElementsByTag("li");
    List<Content> goods = new ArrayList<>();
    for (Element tag : tags) {
        String img = tag.getElementsByTag("img").eq(0).attr("data-lazy-img");
        String price = tag.getElementsByClass("p-price").eq(0).text();
        String name = tag.getElementsByClass("p-name").eq(0).text();
        goods.add(new Content(name, price, img));
    }
    return goods;
}