pyppeteer有关协程Task was destroyed but it is pending的问题

再打量并发运行pypeteer时会出现Task was destroyed but it is pending!的问题,发现是因为协程的原因,最好的办法就是在await完成后自动将task任务清除。

async def GetStream(stream_id):
    # proxy = 'http://211.127.181.30:40414'
    # proxy = 'http://210.196.244.82:53825'
    # proxy='http://120.40.214.216:9999'
    # proxy = 'http://106.180.225.71:34585'

    # headless参数设为False,则变成有头模式
    if '--enable-automation' in launcher.DEFAULT_ARGS:
        launcher.DEFAULT_ARGS.remove("--enable-automation")
    # browser = await launch(
    #     {'handleSIGINT': False, 'handleSIGTERM': False, 'handleSIGHUP': False},
    #     headless=False, dumpio=True, userDataDir=r'./text', autoClose=False,
    #     args=['--proxy-server={}'.format(proxy),'--no-sandbox', '--disable-setuid-sandbox', '--disable-blink-features=AutomationControlled'])

    browser = await launch(
        {'handleSIGINT': False, 'handleSIGTERM': False, 'handleSIGHUP': False},
        headless=False, dumpio=True, userDataDir=r'./text', autoClose=False,
        args=['--no-sandbox', '--disable-setuid-sandbox',
              '--disable-blink-features=AutomationControlled'])

    page = await browser.newPage()
    # await page.authenticate({
    #     'username': '17475066',
    #     'password': '1icmm7xk'
    # })
    await page.setViewport({
        "width": 1068,
        "height": 480
    })
    await page.setUserAgent("Linux/Deepin")
    cookies = get_redis('cookies')
    await page.setCookie(*cookies)
    await page.setRequestInterception(True)
    page.on('request', lambda req: asyncio.ensure_future(intercept_request(req, stream_id)))
    responseBody = ''
    try:
        response = await page.goto('https://www.365sb.com/streamingapi/v2/stream')
        responseBody = await response.text()
        await page.waitFor(2000)
    except Exception as e:
        await page.close()
        await browser.close()
#这里就是在出现异常无法自动关闭task时进行清除
        for i in asyncio.Task.all_tasks():
            flag = i.cancel()
        print('ERROR: ', e)
    finally:
        await page.close()
        await browser.close()
        return responseBody


def main(id, servers):
    data = {
        'state': False,  # 结果标记
        'vtype': '',  # 0: 正常, 1:音视频分离, 2:mpd'
        'video': '',  # 空
        'headers': 'https://www.365sb.com/',  # 空
        'user_agent': 'Linux/Deepin',
        'error_content': ''
    }
    stream_id = result['stream']
    print(stream_id)
    lock.acquire()
    # 使用cancel方法后仍会出现报错,对其进行错误抛出
    try:
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        loop = asyncio.get_event_loop()
        task = loop.create_task(GetStream(stream_id))
        loop.run_until_complete(task)
        responsebody = task.result()
        data = V2stream(responsebody, data)
        loop.close()
     except:
         data['error_content'] = {'code': 101, 'content': '连接被阻止'}
     finally:
         lock.release()

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
"LettuceConnectionFactory was destroyed and cannot be used anymore"这个错误是由于LettuceConnectionFactory连接工厂被销毁了,无法再使用。 解决这个问题的方法有两种: 1. 开启获取连接的校验:可以通过配置LettuceConnectionValidConfig类来设置,这样每次获取连接都会进行校验。 2. 定时校验:可以通过配置LettuceConnectionValidTask类来实现定时校验。这样可以解决长期空闲lettuce连接关闭但是netty不能及时监控到的问题。<span class="em">1</span><span class="em">2</span><span class="em">3</span> #### 引用[.reference_title] - *1* [关于黑马点评完整版代码报错LettuceConnectionFactory was destroyed and cannot be used anymore问题](https://blog.csdn.net/weixin_50786236/article/details/132130181)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_2"}}] [.reference_item style="max-width: 50%"] - *2* *3* [java.lang.IllegalStateException: LettuceConnectionFactory was destroyed and cannot be used anymore](https://blog.csdn.net/weixin_52383177/article/details/126182003)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_2"}}] [.reference_item style="max-width: 50%"] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值