Skip to content

rate-limits.mdx

Latest commit

 

History

History
194 lines (142 loc) · 6.58 KB

rate-limits.mdx

File metadata and controls

194 lines (142 loc) · 6.58 KB
Raw
title Rate Limits
description Understanding ModelsLab API rate limits, the RateLimit response headers, request queuing, and concurrency limits across subscription plans.
**Rate limits help ensure fair usage and optimal performance for all users. Each subscription plan has specific limits for queued API requests.**

Request Queue Limits

Our API uses request queue limits to manage server load and ensure optimal performance. The limits vary based on your subscription plan:

**5 queued API requests** 
Perfect for individual developers and small projects getting started with our APIs.
**10 queued API requests** 
Ideal for growing businesses and applications with moderate usage requirements.
**15 queued API requests** 
Designed for enterprise applications and high-volume usage scenarios.

Rate Limit Response Headers

Every ModelsLab API response carries its own rate-limit state, so you can throttle against the real remaining budget instead of discovering a limit by exceeding it.

Header Meaning
RateLimit-Limit Requests permitted in the current window.
RateLimit-Remaining Requests still available. Throttle against this.
RateLimit-Reset Seconds until the window resets.
RateLimit-Policy Structured field: "<policy>";q=<quota>;w=<window seconds>.
RateLimit Structured field: "<policy>";r=<remaining>;t=<reset seconds>.
Retry-After Seconds to wait. Sent when the limit is exhausted.

The legacy X-RateLimit-Limit and X-RateLimit-Remaining headers are still sent alongside these and are not going away.

curl -sD - -o /dev/null https://modelslab.com/api/agents/v1/changelog | grep -i ratelimit
RateLimit-Limit: 120
RateLimit-Remaining: 119
RateLimit-Reset: 60
RateLimit-Policy: "default";q=120;w=60
RateLimit: "default";r=119;t=60
These headers are present on unauthenticated responses too, including `401`, so an agent can read its budget before it holds a credential.

Control-plane policies

Surface Limit
Authenticated control plane 120 requests/minute
Billing and wallet mutations 15 requests/minute
Auth endpoints 20 requests/minute per IP and email
Signup 3/minute, 10/hour, 20/day per client IP

Buckets are keyed per credential, so two tokens on one account do not consume each other's budget.

Generation endpoints (`/api/v6`, `/api/v7`, `/api/v8`) signal a rate-limit refusal as **HTTP 200** with `"status": "error"` and `"code": "rate_limited"` in the body — not as a `429`. Branch on the body, not the HTTP status. See [Error Codes](/error-codes).

How Request Queuing Works

Request queuing ensures that API calls are processed sequentially in a controlled manner. Here's what you need to know:

Understanding the Limits

  • Sequential Processing: Requests are processed one after another in queue order
  • Queue Management: New requests are added to the queue and processed when previous ones complete
  • Per Account: Limits apply to your entire account, not per API endpoint
  • Real-time: The limit is enforced in real-time as requests come in

Example Scenarios

If you have 5 requests in the queue and try to make a 6th request, it will be rejected until one of the queued requests completes and is processed. You can have up to 10 requests in the queue. Any additional requests will be rejected until queue space becomes available. Up to 15 requests can be queued, providing the highest capacity for enterprise applications.

Request Queuing

When you reach your queue limit:

  1. Queue Full: Additional requests are rejected with a rate limit error
  2. Sequential Processing: Requests are processed one after another in queue order
  3. FIFO Order: Requests are processed in First-In-First-Out order
  4. Automatic Processing: Queued requests are automatically processed as previous ones complete

Best Practices

Optimizing for Rate Limits

Keep track of your queued requests to avoid hitting limits unexpectedly. Add exponential backoff retry logic for better handling of rate limit scenarios. When possible, combine multiple operations into single requests to reduce queue usage. Consider upgrading your plan if you consistently hit queue limits.

Error Handling

When you hit rate limits, you'll receive an HTTP 429 status code with details about the limit:

{
  "status": "error",
  "message": "Rate limit exceeded. Maximum 5 queued requests allowed.",
  "retry_after": 30
}

Upgrading Your Plan

If you need higher queue limits:

  1. Log in to your ModelsLab account
  2. Navigate to the billing section
  3. Select a higher tier plan
  4. New limits take effect immediately
**Need more than 15 queued requests?** Contact our enterprise team for custom solutions tailored to your specific needs.

Frequently Asked Questions

Additional requests are rejected with a rate limit error. You'll need to wait for queued requests to complete before making new ones. No, queue limits are enforced in real-time, not on a daily basis. They're based on current queue size, not total requests per day. No, the queue limit applies to your entire account across all API endpoints.

Support

Need help with rate limits or want to discuss custom solutions?