数据抓取 --Beautiful Soup库的使用问题（4）使用 TRY EXCEPT 时的踩过的坑！

最新推荐文章于 2023-09-21 10:07:20 发布

evan心诺在

最新推荐文章于 2023-09-21 10:07:20 发布

阅读量508

点赞数 2

分类专栏： Python 爬虫文章标签：爬虫 PYTHON TRY和EXCEPT

本文链接：https://blog.csdn.net/weixin_42555401/article/details/99291762

版权

爬虫同时被 2 个专栏收录

13 篇文章 0 订阅

订阅专栏

Python

10 篇文章 1 订阅

订阅专栏

下面是try和except 的使用的基本逻辑图。（这边不多解释基础知识）

这里有个注意点：

使用try 和 except 的里面的 ‘’‘新 ’‘’ 定义的列表，元组，字符串是不共享的。新的变量在try 和except的语句里面是并列关系的。

举例：

最底下边是爬取一个网站商品的SKU的库存的部分代码，使用的逻辑是，

如果是多尺码，那么选择 try, 否则 except.

这里很多人估计都踩过坑，就是COLOR_XPATH列表为什么要在 try里面放完后，在except里面再搁置一个。

如果没有搁置会出现什么结果，（跳过代码往下看）

try:
    # 多尺码,如果是多尺码，那么选择 try, 否则 except.
    pattern = re.compile('尺码')
    f = soup.find('span', text=pattern).get_text()
    type_of_size ='多尺码'
    print(type_of_size)

    # 颜色 XPATH
    COLOR_XPATH = ['//*[@name="goods_spec[颜色]"][1]/following-sibling::a[1]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[2]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[3]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[4]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[5]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[6]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[7]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[8]',
                   '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[9]'
                   ]
    # 尺码 XPATH
    SIZE_XPATH = ['//*[@name="goods_spec[尺码]"][1]/following-sibling::a[1]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[2]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[3]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[4]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[5]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[6]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[7]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[8]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[9]',
                  '//*[@name="goods_spec[尺码]"][1]/following-sibling::a[10]'
                 ]

    # 判断是否是多颜色，如果是多颜色，那么选择 try, 否则 except.
    panduanyanse = soup.find_all('input',{'name':'goods_spec[颜色]'})
    try:
        # 多颜色
        panduanyanse = panduanyanse[1]
        print(panduanyanse)

        # 点击 颜色 然后 点击尺码
        # 点击颜色
        for x_path in COLOR_XPATH:
            try:
                color_click = browser.find_element_by_xpath(x_path)
                color_click.click()
                time.sleep(3)

                # 点击尺码,提取尺码和库存
                size_list = soup.find_all('input', {'name': 'goods_spec[尺码]'})
                num = 0

                for x_size in SIZE_XPATH:
                    try:
                        size_click = browser.find_element_by_xpath(x_size)
                        size_click.click()
                        size = size_list[num].get('rel')
                        num = num+1
                        print('16.尺寸:', size)

                        # 抓取库存,需要给两秒时间加载数据
                        time.sleep(2)

                        cli = browser.find_element_by_xpath('//*[@id="buy_goods_form"]/div/div[6]/ul/li[2]/div[1]/a[2]')

                        time.sleep(2)

                        m = browser.find_element_by_xpath('//*[@id="store_count"]')
                        m.click()
                        size_num = soup.find('span', store_count='store_count')
                        print('17.尺寸库存',size_num)
                    except:
                        continue

            except:
                continue


    except:
        # 单颜色
        ''

# 单尺码
except:
    # 均码
    print('均码')
    type_of_size = '均码'

    # 判断是否是多颜色，如果是多颜色，那么选择 try, 否则 except.
    panduanyanse = soup.find_all('input',{'name':'goods_spec[颜色]'})
    try:
        # 多颜色
        panduanyanse = panduanyanse[1]
        print('多颜色')


        # 点击 颜色 然后 点击尺码
        # 点击颜色
        COLOR_XPATH = ['//*[@name="goods_spec[颜色]"][1]/following-sibling::a[1]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[2]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[3]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[4]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[5]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[6]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[7]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[8]',
                       '//*[@name="goods_spec[颜色]"][1]/following-sibling::a[9]'
                       ]
        for x_path in COLOR_XPATH:
            try:
                color_click = browser.find_element_by_xpath(x_path)
                color_click.click()
                time.sleep(3)

                soup = BeautifulSoup(browser.page_source, 'lxml')
                size_num = soup.find('span', id='store_count').get_text()
                print('17.尺寸库存', size_num)

            except:
                break

    except:
        print('单颜色')
        ''
        # 单颜色
        # size_num = soup.find('span', id='store_count').get_text()
        # print('17.尺寸库存:',size_num)

这是没有放置 COLOR_XPATH列表在except的结果。这是有放置 COLOR_XPATH列表在except的结果。