python3利用beautifulsoup4库实现【html爬虫】处理汽车品牌大全

最新推荐文章于 2022-06-25 21:50:19 发布

lukezhang-123

最新推荐文章于 2022-06-25 21:50:19 发布

阅读量217

点赞数

分类专栏： python 文章标签： python3 beautifulsoup4 bs4

本文链接：https://blog.csdn.net/c5113620/article/details/102409490

版权

python 专栏收录该内容

8 篇文章 1 订阅

订阅专栏

# windonws
set http_proxy=http://192.168.1.11:1080
set https_proxy=http://192.168.1.11:1080

py -m pip install beautifulsoup4

import urllib.request
from bs4 import BeautifulSoup

with urllib.request.urlopen('http://car.bitauto.com/qichepinpai/') as url:
    s = url.read()

soup = BeautifulSoup(s,features='html.parser')
# print(soup.prettify())

uls = soup.find_all('ul',{'class':'list_pic'})
for ul in uls:
    lis = ul.find_all('li')
    for li in lis:
        print(li.find('a')['title'])
        htmlstr = urllib.request.urlopen(li.find('a')['href']).read()
        soup = BeautifulSoup(htmlstr,features='html.parser')
        div = soup.find('div',{'id':'data_table_MasterSerialList_0'})
        lis = div.find_all('li',{'class','name'})
        for li in lis:
            print(li.find('a').getText()+'  ',end='')
        print()