Rate limits and tiers

Rate limits protect capacity and keep traffic predictable. AnyPath uses two common limits:

• RPM: requests per minute
• TPM: tokens per minute

Your account tier is the product policy source for account-level RPM/TPM. API keys can set narrower limits for workload isolation. Routes and upstream provider keys can also have their own limits. The effective runtime limit is the strictest applicable limit across account policy, API key policy, the selected model, its eligible credential routes, and upstream key availability.

Treat /v1/models and Console limits as the source of truth for the current effective values.

Tier and effective limits

AnyPath tiers are transparent account policies:

Layer	What it controls	Can it raise the effective limit?
Account tier / user policy	Account-level RPM/TPM and model access	Yes, up to platform and route capacity
API key settings	Per-key RPM/TPM, model allowlist, route policy, and budget	No; key settings only narrow the account policy
Route / upstream key	Supply capacity for a selected shared route	No; unavailable or lower upstream limits can reduce the effective limit

For model discovery, call GET /v1/models with the same API key. When available, each model entry shows the effective RPM/TPM after route policy and supply capacity are applied.

Default account tiers are:

Tier	Account RPM	Account TPM	How an account gets there
Starter	120	2,000,000	Every new account
Builder	600	10,000,000	10 USDT paid in total; automatic
Scale	1,500	30,000,000	200 USDT paid in total; automatic
Enterprise	3,000	60,000,000	Granted by an operator; higher limits are set per account

• Tiers are evaluated every hour. The only condition is what the account has paid in total, which only grows, so an account keeps a tier it has reached.
• An API key without its own limit shares the account’s limit. A limit set on one key applies to that key only.
• TPM counts an estimate made when the request arrives: the whole prompt (about one token per four characters) plus the request’s max_tokens (512 when unset). An inline image or other media part counts as 1,600 tokens, not by the length of its base64 text. It is not the final usage; billing uses actual usage.
• There is also a Trial tier (30 RPM, 500,000 TPM). It is only assigned by an operator to slow an account down for a while; when the assignment ends the account returns to the tier it has earned.

These are account-policy defaults, not guaranteed capacity. Console and /v1/models remain the runtime source of truth because API key settings, selected routes, provider availability, and upstream limits can lower the effective values.

Throttled responses

When a request is throttled, the API returns 429 Too Many Requests with Retry-After. The error code says whose limit was reached: rate_limit_exceeded is your own account or key limit; supplier_rate_limited means you are within your limits and the suppliers of that model have no capacity left right now. Back off the same way in both cases; for the second, another model may be served immediately.

Rate-limit headers

Responses report your own account and API key window, including 429 responses. The window is one minute.

Header	Meaning
x-ratelimit-limit-requests	Effective requests-per-minute limit for this key
x-ratelimit-remaining-requests	Requests left in the current window
x-ratelimit-reset-requests	Seconds until the window resets
x-ratelimit-limit-tokens	Effective tokens-per-minute limit for this key
x-ratelimit-remaining-tokens	Tokens left in the current window
x-ratelimit-reset-tokens	Seconds until the window resets

The effective limit is the tighter of the account tier and the API key setting. A metric with no limit has no header. Token headers appear on requests that carry a token estimate, and count that estimate rather than final usage.

These headers never describe the supplier account that served the request. Supply capacity can still reject a request while your own window has room.

Recommended retry behavior

1. Respect Retry-After when present.
2. Use exponential backoff with jitter when it is absent.
3. Keep a per-key queue for bursty workloads.
4. Do not retry authentication, permission, validation, or payment errors as rate-limit errors.

import random
import time

def wait_seconds(attempt: int, retryAfterHeader: str | None = None) -> float:
    if retryAfterHeader:
        return float(retryAfterHeader)
    return min(60, (2 ** attempt) + random.random())

What to monitor

• 429 rates by API key and provider
• account tier and user-policy changes in Console
• route health and provider availability in Console
• input and output token volume against TPM
• queue wait time before dispatch

If shared capacity is repeatedly unavailable, choose another available model.

Tier upgrades

Tier upgrades are based on what the account has paid in total. Accounts are evaluated automatically every hour, and a qualifying account is promoted without any request from you. An operator can set a higher or lower tier for a specific account, for a while or for good; those changes do not change API request semantics and are audited separately.

Automatic downgrade, when enabled, must use a grace period. A short drop in traffic should not immediately reduce production limits.