题意:“使用 OpenAI API 调用时,asyncio
在长时间运行后挂起”
问题背景:
I'm using asyncio alongside the OpenAI API to translate a set of texts concurrently. Initially, everything works as expected, and I see the answers from OpenAI printed in the console. However, after running for a while, the code seems to hang. Subsequent answers from OpenAI are not printed in the console, and I don't see any "retrying" messages either. Here's my code:
“我在使用 asyncio
结合 OpenAI API 并发翻译一组文本。最初,一切都按预期工作,我在控制台中看到来自 OpenAI 的回答。然而,运行一段时间后,代码似乎挂起了。来自 OpenAI 的后续回答未显示在控制台中,我也没有看到任何‘重试’消息。以下是我的代码:”
import asyncio
from aiohttp import ClientSession
import openai
import os
openai.api_key = os.getenv("OPENAI_API_KEY")
async def _atranslate(sem, messages, **model_kwargs):
max_retry = 2
async with sem:
while max_retry > 0:
try:
response = await openai.ChatCompletion.acreate(
messages=messages,
**model_kwargs
)
answer = response.choices[0]['message']['content']
print(answer)
return answer
except Exception as e:
print(e)
await asyncio.sleep(5)
max_retry -= 1
print('retrying...')
raise ConnectionError('cannot reach openai!')
async def atranslate(text_list: list, source=None, target='English', max_workers=3, **model_kwargs):
aio_session = ClientSession()
openai.aiosession.set(aio_session)
model_kwargs.setdefault('model', 'gpt-3.5-turbo')
model_kwargs.setdefault('temperature', 1)
model_kwargs.setdefault('timeout', 10)
template = 'Translate the following {source} text into {target}:{text}'
semaphore = asyncio.Semaphore(max_workers)
tasks = []
for text in text_list:
messages = [{
'role': 'user',
'content': template.format(
source=source,
target=target,
text=text
)}
]
tasks.append(asyncio.create_task(_atranslate(semaphore, messages, **model_kwargs)))
results = await asyncio.gather(*tasks)
await aio_session.close()
return results
if __name__ == '__main__':
textList = '... (some texts are omitted for brevity)'
translations = asyncio.run(atranslate(textList*20, 'Korean', 'English',30))
When I run the above code, it starts off well but after some time, it simply hangs. What could be causing this? Are there any solutions or suggestions to address this issue?
“当我运行上述代码时,起初一切正常,但一段时间后,它就挂起了。可能是什么原因导致的?是否有解决方案或建议来解决这个问题?”
I have tried to change the timeout parameter or simply not raise ConnectionError, but it doesn't work.I think it might be the problem of api usage limits, but I don't know why it doesn't throw any error.
“我尝试更改超时参数或干脆不抛出 ConnectionError
,但没有效果。我认为这可能是 API 使用限制的问题,但我不明白为什么它没有抛出任何错误。”
问题解决:
Two things, first I would wrap the use of ClientSession
in try...finally
, or use it as a contextmanager
as shown here.
“两点建议:首先,我会将 ClientSession
的使用包装在 try...finally
中,或者像这里显示的那样将其用作 contextmanager
。”
Also, if you are pretty sure that there are some exceptions getting swallowed somewhere, try to catch BaseException
instead of Exception
. Yes, there is a BaseException
class which except Exception:
won't catch, and the asyncio
library throws exceptions derived from BaseException
sometimes.
“此外,如果你确定某些异常在某处被吞掉了,可以尝试捕获 BaseException
而不是 Exception
。是的,确实有一个 BaseException
类,而 except Exception:
并不会捕获它。asyncio
库有时会抛出从 BaseException
派生的异常。”