Skip to main content
Reserved TPM is in Beta and available to select pilot customers. To join the waitlist, send us a note from the support form in your Sail dashboard (recommended), or contact support.
With Reserved TPM, you can reserve dedicated serverless inference capacity ahead of time in the form of uncached input and output tokens per minute (TPM), on your selected model and service tier. You can think of Reserved TPM as prepaying for tokens (at your normal per-token rates) in exchange for guaranteed TPM. Reserved TPM also removes the queueing time associated with Balanced and Flex altogether. Reserved TPM is fully self-serve and available on Pro and Enterprise plans.

How it works

  • You can book Reserved TPM for a start time at least one hour in advance. You will specify:
    • The model and service tier
      • Optional: The LoRA name or ID, if reserving throughput for your LoRA fine-tuned model (note - you must upload your LoRA to Sail first).
    • The duration of the reservation in hours, from 1 hour to 3 days long
    • The reservation start time, from 1 hour to 7 days in advance
    • The uncached input TPM amount to reserve
    • The output TPM amount to reserve
  • There is no surcharge for Reserved TPM. The cost of a reservation is simply the cost of pre-paying for the reserved uncached input and output token volume ahead of time: your normal per mTok rates (see Pricing), applied to the TPM and duration you reserve.
  • Eligible inference usage during your reservation is guaranteed to be able to meet or exceed the throughput limits you reserved.
  • The cost of eligible inference usage during the reservation period first draws down from the prepayment from when you booked the Reserved TPM, then is billed as regular usage once the total prepayment is depleted.
If your Sail account uses prepaid credits (Pro plan) instead of monthly invoicing (Enterprise plan), be sure to have enough credits in your balance to cover eligible usage beyond the reservation prepayment. This is important because the prepayment doesn’t include the cost of any cached input token volume. (See: What if my org’s credit balance runs out during a reservation?)

Book and use Reserved TPM

Send POST /v1/reserved-tpm with an Idempotency-Key header:
Retrying with the same Idempotency-Key will return the same reservation. Reusing a key with different fields will return HTTP 409.

Errors

Success

If the Reserved TPM you’ve requested is available, the reservation will be booked immediately and the cost will be deducted from your balance. The response will contain the reservation.

Reservation details

A reservation looks like:
usage shows how much of the reservation’s prepayment balance is used (consumed_cents) and left (remaining_cents), as of as_of. Note that updates to usage may continue for up to several hours past the end, because requests admitted near the end of the reservation can still use the value when they finish.

Use a reservation

Your inference requests need no changes. The requests you send during your Reserved TPM window with the corresponding model and service tier will be routed to your reserved capacity automatically.
Using Reserved TPM has the added benefit of removing the request queueing latency that would otherwise be incurred on the Balanced and Flex service tiers.
Don’t use the Batch API for Reserved TPM. Batch requests aren’t eligible for Reserved TPM SLAs.

List reservations

List returns your organization’s reservations, newest first, with has_more. limit is 1 to 100 (default 20). To get the next page, pass the last ID as after_id.

Get a reservation

Cancel a reservation

You can cancel a booked reservation to release the capacity, any time until it ends. Free cancel grace period: you can cancel Reserved TPM within 10 minutes of booking for a full refund of the price in Sail credits. Your organization gets one free cancellation in a rolling 24-hour window. Otherwise, reserved TPM is non-refundable.
free_cancel_until shows when a free cancel stops being available, or null when the reservation is not refundable. The response’s refund contains amount_cents and a status of pending until the credit is added, then refunded. If nothing is refunded, refund is null.

FAQ

At least 1 hour in advance. and at most 7 days in advance.
From 1 hour to 3 days (72 hours), in whole hours.
No, reservations can’t be changed after booking. You must cancel and book another reservation instead.
Reserved TPM is currently in waitlisted Beta. Soon, this feature will be open to all users on Pro and Enterprise plans.
Reserved TPM is in Beta, by request. Send us a note from the support form in your Sail dashboard, or contact support.
Up to 7 days. If you need to book further out, contact support.
Yes. Book the LoRA’s base model on Flex, with lora set to the LoRA’s name or ID. That reservation covers requests that use that LoRA. A reservation without lora covers only requests that don’t use a LoRA.
Yes, you can fetch the reservation and see the remaining value in the usage field.
No. It covers only the reservation’s designated model and tier. Requests on other tiers run as normal traffic.
You pre-pay your normal per-token prices for the entire reservation’s token uncached input and output amount when you book. If your org is on the Pro plan, which uses prepaid credits for billing, you must have enough credits to make this upfront payment.
Sail releases the capacity. Cancelling within 10 minutes of booking also refunds the full price as Sail credits, if you haven’t already done such a cancellation in the past 24 hours. Any other cancellation is not eligible for a refund.
Unused value after the reservation ends is not refunded.
Each model and service tier combination has a minimum reservation size, in dollars per hour.
Yes, monetary value rolls over minute to minute.To illustrate, if your reservation’s value is $100, any inference request with the correct model and service tier during the reservation window will draw down from that $100 until it’s depleted, regardless of when it’s sent.However, the TPM guarantees of your reservation remain constant, so bursts over the guaranteed amounts may not be serviceable if there is not sufficient capacity.
Those requests run and are billed as normal traffic, without the Reserved TPM SLA.
Eligible usage under the Reserved TPM booking specifications will continue as normal, since it is already prepaid, so long as the reservation’s total value is not depleted. Other requests will return 402 until you add credits.
A US-only reservation makes the requests it covers US-only. They run in the US and are billed as US-only, matching the reservation’s price, even without routing.allowed_countries. Still we recommend setting this on the individual requests as well in case usage exceeds your reservation.