Rate limits and temporary failures
A429 or a 503 with error code model_capacity_unavailable means the
request was refused before inference started. Wait before trying again.
Other server errors (5xx) may occur after Sail accepts the request; follow
Uncertain submissions before retrying.
If the response includes Retry-After, wait at least that many seconds.
Increase the delay after each unsuccessful attempt, for example from one to
two to four seconds, and add a small random delay so requests do not all retry
at once. Set a maximum number of attempts and a total time limit.
For automatic submission retries, send an idempotency key
with the first attempt and reuse it on every retry. Otherwise, disable
automatic retries, for example with client.with_options(max_retries=0) in
the OpenAI SDK. For an invalid request or API key, fix the cause before
resubmitting.
Uncertain submissions
A timeout, lost connection, or other server error does not tell you whether Sail started the request.- If you have a response ID, retrieve its status. For background requests, keep polling that ID. A polling timeout does not cancel the request.
- If you sent an idempotency key, reuse it with the same body and API key. See Idempotency for supported headers and replay behavior.
- Without a response ID or idempotency key, retrying may repeat the work. It can start another generation and incur another charge.
Streaming errors
A stream can return HTTP200 and still fail before or during generation.
Handle error events in the stream as well as errors when opening it. If you
retry after receiving partial output, make sure your application does not
repeat actions it already took from that output.