How it works
- You can book Reserved TPM for a start time at least one hour in advance.
You will specify:
- The model and service tier
- Optional: The LoRA name or ID, if reserving throughput for your LoRA fine-tuned model (note - you must upload your LoRA to Sail first).
- The duration of the reservation in hours, from 1 hour to 3 days long
- The reservation start time, from 1 hour to 7 days in advance
- The uncached input TPM amount to reserve
- The output TPM amount to reserve
- The model and service tier
- There is no surcharge for Reserved TPM. The cost of a reservation is simply the cost of pre-paying for the reserved uncached input and output token volume ahead of time: your normal per mTok rates (see Pricing), applied to the TPM and duration you reserve.
- Eligible inference usage during your reservation is guaranteed to be able to meet or exceed the throughput limits you reserved.
- The cost of eligible inference usage during the reservation period first draws down from the prepayment from when you booked the Reserved TPM, then is billed as regular usage once the total prepayment is depleted.
Book and use Reserved TPM
SendPOST /v1/reserved-tpm with an Idempotency-Key header:
- Base model
- LoRA fine-tune
Idempotency-Key will return the same reservation.
Reusing a key with different fields will return HTTP 409.
Errors
Success
If the Reserved TPM you’ve requested is available, the reservation will be booked immediately and the cost will be deducted from your balance. The response will contain the reservation.Reservation details
A reservation looks like:usage shows how much of the reservation’s prepayment balance is used
(consumed_cents) and left (remaining_cents), as of as_of. Note that
updates to usage may continue for up to several hours past the end, because
requests admitted near the end of the reservation can still use the value when
they finish.
Use a reservation
Your inference requests need no changes. The requests you send during your Reserved TPM window with the corresponding model and service tier will be routed to your reserved capacity automatically.List reservations
List returns your organization’s reservations, newest first, withhas_more.
limit is 1 to 100 (default 20). To get the next page, pass the last ID as
after_id.
Get a reservation
Cancel a reservation
You can cancel abooked reservation to release the capacity, any time until
it ends.
Free cancel grace period: you can cancel Reserved TPM within 10 minutes
of booking for a full refund of the price in Sail credits. Your organization
gets one free cancellation in a rolling 24-hour window.
Otherwise, reserved TPM is non-refundable.
free_cancel_until shows when a free cancel stops being available, or null
when the reservation is not refundable. The response’s refund contains
amount_cents and a status of pending until the credit is added, then
refunded. If nothing is refunded, refund is null.
FAQ
How far ahead can I book Reserved TPM?
How far ahead can I book Reserved TPM?
How long can a reservation be?
How long can a reservation be?
Can I change a reservation?
Can I change a reservation?
Who can use Reserved TPM?
Who can use Reserved TPM?
How do I get access?
How do I get access?
How far in advance can I book?
How far in advance can I book?
Can I reserve TPM for a LoRA fine-tune?
Can I reserve TPM for a LoRA fine-tune?
lora set to the LoRA’s name
or ID. That reservation covers requests that use that LoRA. A
reservation without lora covers only requests that don’t use a LoRA.Can I see how much of a reservation I've used?
Can I see how much of a reservation I've used?
usage field.Does a reservation cover other service tiers?
Does a reservation cover other service tiers?
How does billing work?
How does billing work?
What happens if I cancel?
What happens if I cancel?
What happens to value I don't use?
What happens to value I don't use?
What's the smallest reservation?
What's the smallest reservation?
Does the reservation's value carry across minutes?
Does the reservation's value carry across minutes?
What happens to usage above the reserved value during the reservation period?
What happens to usage above the reserved value during the reservation period?
What if my org's credit balance runs out during a reservation?
What if my org's credit balance runs out during a reservation?
402 until you add
credits.How does a US-only reservation work?
How does a US-only reservation work?
routing.allowed_countries. Still we recommend setting this on the
individual requests as well in case usage exceeds your reservation.