Service tiers

Service tiers allow you to balance processing speed, latency, and cost for inference requests. Sail serves low-latency inference by default for core models, at lower cost than traditional inference providers. You can opt in to the Balanced or Flex service tiers to cut token costs drastically for latency-tolerant work.

Service tier comparison

Tier	Queueing time	Target TPS	Price vs. competitor model APIs
Default (ASAP)	None	50-100 TPS	30-70% less
Balanced	Up to 5 min for first turn, much faster for subsequent turns (seconds)	30-50 TPS	45-80% less
Flex	Up to 1 hour for first turn, much faster for subsequent turns (seconds)	10-20 TPS	55-85% less

Check Pricing for current per-model availability.

Service tier details

Default (ASAP)

Default, aka ASAP, is the standard low-latency tier for Sail’s core models. It is characterized by:

No queueing delay - TTFT is dependent on prompt size and model.
High TPS - targeting 50-100 TPS. Your experienced TPS will vary by workload and model.
Foreground-only - this tier is not compatible with background mode or batch requests.

Developers use ASAP for: Jump to:

Default (ASAP) notes and troubleshooting

Flex

Flex is characterized by:

A flexible start - the first turn in a run is guaranteed to start within 1 hour.
Much faster subsequent turns - follow-up turns of an in-flight run see minimal (seconds) queueing delay.
Low TPS - targeting 10-20 TPS. Your experienced TPS will vary by workload and model.
Support for extreme scale by design - Flex offers the most generous capacity and highest rate limits.
Background-only - Flex is not compatible with streaming or foreground APIs.

Sail uses longest-prefix matching to automatically recognize subsequent turns.

Developers use Flex for: Jump to:

How to use Flex
Flex notes and troubleshooting

Balanced

Balanced is the middle ground between Flex and Default, characterized by:

A flexible start with a tighter bound than Flex - the first turn in a run should start within 5 minutes.
Much faster - like Flex, follow-up turns of an in-flight run see minimal (seconds) queueing delay.
Medium TPS - targeting 30-50 TPS. Your experienced TPS will vary by workload and model.
Streaming, foreground, and background-compatible.

Developers use Balanced for: Jump to:

How to use Balanced
Balanced notes and troubleshooting

Multi-turn agents and rollouts with Balanced and Flex

Balanced and Flex are designed to support multi-turn agents and rollouts. For the first turn in a run, Balanced and Flex have a “warm-up” queueing delay of up to 5 minutes (for Balanced) and 1 hour (for Flex), respectively. Afterwards, Sail automatically uses longest-prefix matching to recognize subsequent turns of an in-flight run. These follow-up turns are prioritized and incur minimal to no queueing time.

How to use service tiers

Via request body `metadata`

Set metadata.completion_window to asap, balanced, or flex:

curl https://api.sailresearch.com/v1/responses \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Explain the key ideas behind transformers.",
    "metadata": {
      "completion_window": "balanced"
    }
  }'

curl https://api.sailresearch.com/v1/chat/completions \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "messages": [
      {"role": "user", "content": "Explain the key ideas behind transformers."}
    ],
    "metadata": {
      "completion_window": "balanced"
    }
  }'

curl https://api.sailresearch.com/v1/messages \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Explain the key ideas behind transformers."}
    ],
    "metadata": {
      "completion_window": "balanced"
    }
  }'

To use Flex, you must use the Responses API with background=True or submit requests through the Batch API.

Flex is not compatible with Chat Completions, Messages, or foreground (background=False) Responses API requests.

curl https://api.sailresearch.com/v1/responses \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Explain the key ideas behind transformers.",
    "background": true,
    "metadata": {
      "completion_window": "flex"
    }
  }'

# Batch always uses flex, so no service tier is specified.
curl https://api.sailresearch.com/v1/batches \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "/v1/responses",
    "requests": [
      {
        "custom_id": "request-1",
        "params": {
          "model": "zai-org/GLM-5.3",
          "input": "Explain the key ideas behind transformers."
        }
      }
    ]
  }'

Via request header

Alternatively, if your client cannot add metadata to a request body, you can set the service tier using the X-Sail-Completion-Window request header.

curl https://api.sailresearch.com/v1/responses \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Sail-Completion-Window: balanced" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "input": "Explain the key ideas behind transformers."
  }'

curl https://api.sailresearch.com/v1/chat/completions \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Sail-Completion-Window: balanced" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "messages": [
      {"role": "user", "content": "Explain the key ideas behind transformers."}
    ]
  }'

curl https://api.sailresearch.com/v1/messages \
  -H "Authorization: Bearer $SAIL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Sail-Completion-Window: balanced" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Explain the key ideas behind transformers."}
    ]
  }'

Default behavior

For inference requests for core models, the default behavior (when no completion_window is specified) is low-latency inference (asap). The exceptions to this default behavior are:

Batch inference requests, which default to Flex and do not support other service tiers.
LoRA requests, which are Flex-only.
Requests that use custom tools, which default to Balanced for core models because ASAP does not support them.
Responses API requests with background=True use Balanced or Flex by default, since they do not support ASAP.

For Flex-only models, the default (and only) service tier is Flex.

Troubleshooting

ASAP requests

ASAP does not support custom tools (tools with type="custom") on the Responses or Chat Completions APIs. Use Balanced, or omit the service tier to default to Balanced for core models.
ASAP does not support Responses requests with background=True or the Batch API. Use Balanced or Flex for Responses requests with background=True.
Setting metadata.completion_window="asap" with custom tools or background=True returns HTTP 400 with error code unsupported_asap_request.

Balanced requests

You can send Balanced requests synchronously or as Responses requests with background=True.
A synchronous streaming Balanced request opens its stream right away and keeps the connection alive while it waits. Output starts when the model begins generating.

Flex requests

Chat Completions and Messages reject Flex with HTTP 400, with or without streaming. Use ASAP or Balanced on those endpoints.
Responses requests using Flex reject an omitted background value, background=None, or background=False with HTTP 400. Set background=True explicitly.

Request settings

If you set both X-Sail-Completion-Window and metadata.completion_window, the value in the request body takes precedence.
The model must support the tier you choose. Check Pricing for availability.

Batch and Background

Responses requests with background=True cannot use stream=True. Retrieve results by polling or using webhooks.
A polling timeout does not cancel a Responses request with background=True. Keep polling the same response ID. See Retrying requests if you lose the connection before receiving an ID.
The Batch API does not use X-Sail-Completion-Window.
Batch always uses Flex and runs in the background. Setting a subrequest’s service tier to asap or balanced returns HTTP 400.
Batch rejects models that do not support Flex.

Capacity errors

For ASAP requests, Sail may wait briefly for capacity. If none becomes available, it returns 503 with code model_capacity_unavailable without queueing the request.
If the service is overloaded, Sail can return 503 with code service_overloaded before accepting a request in any service tier.
If the response includes Retry-After, wait at least that many seconds. See Retrying requests before resubmitting after a timeout or another server error, since the request may already have started.

Getting started

Inference

Sailboxes

Guides

Security & compliance

Service tier comparison

Service tier details

Default (ASAP)

Flex

Balanced

Multi-turn agents and rollouts with Balanced and Flex

How to use service tiers

Via request body `metadata`

Via request header

Default behavior

Troubleshooting

ASAP requests

Balanced requests

Flex requests

Request settings

Batch and Background

Capacity errors

​Service tier comparison

​Service tier details

​Default (ASAP)

​Flex

​Balanced

Multi-turn agents and rollouts with Balanced and Flex

​How to use service tiers

​Via request body metadata

​Via request header

​Default behavior

​Troubleshooting

​ASAP requests

​Balanced requests

​Flex requests

​Request settings

​Batch and Background

​Capacity errors

Service tier comparison

Service tier details

Default (ASAP)

Flex

Balanced

How to use service tiers

Via request body `metadata`

Via request header

Default behavior

Troubleshooting

ASAP requests

Balanced requests

Flex requests

Request settings

Batch and Background

Capacity errors