‘asyncio‘ with OpenAI API Call Hangs After Extended Run Time

题意：“使用 OpenAI API 调用时，asyncio 在长时间运行后挂起”

问题背景：

I'm using asyncio alongside the OpenAI API to translate a set of texts concurrently. Initially, everything works as expected, and I see the answers from OpenAI printed in the console. However, after running for a while, the code seems to hang. Subsequent answers from OpenAI are not printed in the console, and I don't see any "retrying" messages either. Here's my code:

“我在使用 asyncio 结合 OpenAI API 并发翻译一组文本。最初，一切都按预期工作，我在控制台中看到来自 OpenAI 的回答。然而，运行一段时间后，代码似乎挂起了。来自 OpenAI 的后续回答未显示在控制台中，我也没有看到任何‘重试’消息。以下是我的代码：”

import asyncio
from aiohttp import ClientSession
import openai
import os
openai.api_key = os.getenv("OPENAI_API_KEY")

async def _atranslate(sem, messages, **model_kwargs):
    max_retry = 2
    async with sem:
        while max_retry > 0:
            try:
                response = await openai.ChatCompletion.acreate(
                    messages=messages,
                    **model_kwargs
                )
                answer = response.choices[0]['message']['content']
                print(answer)
                return answer
            except Exception as e:
                print(e)
                await asyncio.sleep(5)
                max_retry -= 1
                print('retrying...')
        raise ConnectionError('cannot reach openai!')

async def atranslate(text_list: list, source=None, target='English', max_workers=3, **model_kwargs):
    aio_session = ClientSession()
    openai.aiosession.set(aio_session)
    model_kwargs.setdefault('model', 'gpt-3.5-turbo')
    model_kwargs.setdefault('temperature', 1)
    model_kwargs.setdefault('timeout', 10)
    template = 'Translate the following {source} text into {target}:{text}'
    semaphore = asyncio.Semaphore(max_workers)
    tasks = []
    for text in text_list:
        messages = [{
            'role': 'user',
            'content': template.format(
                source=source,
                target=target,
                text=text
            )}
        ]
        tasks.append(asyncio.create_task(_atranslate(semaphore, messages, **model_kwargs)))
    results = await asyncio.gather(*tasks)
    await aio_session.close()
    return results

if __name__ == '__main__':
    textList = '... (some texts are omitted for brevity)'
    translations = asyncio.run(atranslate(textList*20, 'Korean', 'English',30))

When I run the above code, it starts off well but after some time, it simply hangs. What could be causing this? Are there any solutions or suggestions to address this issue?

“当我运行上述代码时，起初一切正常，但一段时间后，它就挂起了。可能是什么原因导致的？是否有解决方案或建议来解决这个问题？”

I have tried to change the timeout parameter or simply not raise ConnectionError, but it doesn't work.I think it might be the problem of api usage limits, but I don't know why it doesn't throw any error.

“我尝试更改超时参数或干脆不抛出 ConnectionError，但没有效果。我认为这可能是 API 使用限制的问题，但我不明白为什么它没有抛出任何错误。”

问题解决：

Two things, first I would wrap the use of ClientSession in try...finally, or use it as a contextmanager as shown here.

“两点建议：首先，我会将 ClientSession 的使用包装在 try...finally 中，或者像这里显示的那样将其用作 contextmanager。”

Also, if you are pretty sure that there are some exceptions getting swallowed somewhere, try to catch BaseException instead of Exception. Yes, there is a BaseException class which except Exception: won't catch, and the asyncio library throws exceptions derived from BaseException sometimes.

“此外，如果你确定某些异常在某处被吞掉了，可以尝试捕获 BaseException 而不是 Exception。是的，确实有一个 BaseException 类，而 except Exception: 并不会捕获它。asyncio 库有时会抛出从 BaseException 派生的异常。”