【Bugs】openai.error.ServiceUnavailableError The server is overloaded or not ready yet

【Bugs】openai.error.ServiceUnavailableError: The server is overloaded or not ready yet.

@author: SUFEHeisenberg

@date: 2023/07/21

在调用ChatGPT API时报错：

openai.error.ServiceUnavailableError: The server is overloaded or not ready yet.

Solution

利用指数退避的思想：

import openai
from tenacity import (
    retry,
    stop_after_attempt,
    wait_random_exponential,
    retry_if_exception_type
)  # for exponential backoff

@retry(
    retry=retry_if_exception_type((openai.error.APIError, openai.error.APIConnectionError, openai.error.RateLimitError, openai.error.ServiceUnavailableError, openai.error.Timeout)), 
    wait=wait_random_exponential(multiplier=1, max=60), 
    stop=stop_after_attempt(10)
)

def chat_completion_with_backoff(**kwargs):
    return openai.ChatCompletion.create(**kwargs)

response = chat_completion_with_backoff(
                model=model,
                messages=[
                    {"role": "system", "content": system_msg},
                    {"role": "user", "content": longtext},
                ],
                max_tokens=max_tokens,
            )

Reference

https://community.openai.com/t/openai-error-serviceunavailableerror-the-server-is-overloaded-or-not-ready-yet/32670/20