pickle dump 报错: maximum recursion depth exceeded while getting the str of an object

当执行pickle dump时,报错:

maximum recursion depth exceeded while getting the str of an object

我的代码如下:

def get_data_from_html(products, ods_path, dwd_path):
    all_result = {}
    for pd_part in tqdm(products, desc='Processing'):
        print(f"It time to handle product, {pd_part}")
        pd_html_path = os.path.join(ods_path, pd_part)
        pd_name = pd_part.strip().strip('.html')
        csv_name = os.path.join(dwd_path, f'{pd_name}.csv')
        result = []
        with open(pd_html_path, 'r') as f_obj:
            data = f_obj.read()
        soup = BeautifulSoup(data,'lxml')
        tables = soup.find_all('table')
        with open(csv_name, 'wb+') as csv_obj:
            for table in tables:
                trs = table.find_all('tr')
                for index, tr in enumerate(trs):
        #             print(f"{index} -- {tr}")
                    if index == 0:
                        continue
                    tmp = []
                    for td in tr.find_all('td'):
                        tmp.append(td.string)
                        csv_obj.write(','.join(tmp).encode('utf-8'))
                        csv_obj.write('/n'.encode('utf-8'))
                    result.append(tmp)
        all_result[pd_name] = result
    print(f'write all result to dump file by pickle')
    dump_path = os.path.join(dwd_path, 'all_result.dump')
    print(dump_path)
    with open(dump_path, 'wb') as pk_obj:
        pickle.dump(all_result, pk_obj)
    return all_result

 解决办法

经过搜索发现这个问题https://www.coder.work/article/1272776和BeautifulSoup有关系,

然后我将我的代码中的 tmp.append(td.string)修改为tmp.append(str(td.string)), 然后程序就能正常执行了。
 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值