Claude Fable 5.1 & GPT-6 Astra packages are live

Rate Limiting

Free

Rate limiting that protects the service without punishing legitimate clients — algorithm choice, key selection, headers, and distributed enforcement.

212 lines8.4 KB Qwen API
targetModels
Qwen3.8-MaxQwen3.8-Flash-NextQwen3.8-27BQwen3.8 FamilyFuture Qwen Models
name
rate-limiting
category
API
description
Rate limiting that protects the service without punishing legitimate clients — algorithm choice, key selection, headers, and distributed enforcement.
license
MIT
author
Agent.md maintainers
last-verified
reviewed-by
unreviewed
<!-- Generated from models/_canonical by scripts/build-model-variants.js. Edit the canonical source, not this file. Behavioural profile for Qwen: scripts/model-profiles.json -->

#Task boundary

  1. Implement only what the task names; no extra abstractions or files.
  2. English-only comments and identifiers.
  3. Stop when the checklist passes.

#Purpose

Rules for limiting request rates. Two distinct goals, often conflated:

  1. Protection — keep one caller from exhausting capacity for everyone.
  2. Fairness / monetisation — enforce plan quotas.

They need different keys, different windows, and different responses. Decide which one each limiter serves before configuring it.


#Algorithms

AlgorithmBurstMemory per keyBoundary problem
Fixed window2× at the boundary1 counterYes — 2× the limit across a boundary
Sliding window logNoneO(n) timestampsNo, but memory grows with traffic
Sliding window counterSlight2 countersNegligible
Token bucketConfigurable, intended2 valuesNo
Leaky bucketNone; smooths output1 queueNo

Token bucket is the default. It expresses the real requirement — a sustained rate plus an allowance for bursts — in two numbers, and it costs two values per key.

lua
-- Atomic token bucket in Redis. Read-then-write in application code races.
local bucket = redis.call("HMGET", KEYS[1], "tokens", "ts")
local tokens = tonumber(bucket[1]) or tonumber(ARGV[3])
local ts     = tonumber(bucket[2]) or tonumber(ARGV[4])
local delta  = math.max(0, tonumber(ARGV[4]) - ts)
tokens = math.min(tonumber(ARGV[3]), tokens + delta * tonumber(ARGV[1]))
if tokens < 1 then return {0, tokens} end
redis.call("HMSET", KEYS[1], "tokens", tokens - 1, "ts", ARGV[4])
redis.call("EXPIRE", KEYS[1], tonumber(ARGV[2]))
return {1, tokens - 1}

Fixed windows are tempting because they are trivial, but a client can send the full limit at 59.9s and again at 60.1s — twice the intended rate. Do not use them for anything protective. → Database/redis


#Choose the key deliberately

KeyLimitsWeakness
API key / accountFairness, quotasAbsent for unauthenticated traffic
User idPer-user fairnessSame
IP addressUnauthenticated abuseNAT and mobile carriers share one IP; IPv6 is cheap to rotate
IP + routeLogin brute forceDistributed attacks bypass it
Account + routeCredential stuffing on one account
GlobalBackstop against overloadBlunt

Limit on more than one key at once. Per-IP alone does not stop a distributed credential-stuffing run against one account; per-account alone lets one host spray many accounts. → Security/authentication

Determine the client IP correctly. Behind a proxy, X-Forwarded-For is client-controlled unless you take the value at a known hop count from a trusted proxy set. Trusting the leftmost value lets any caller forge an unlimited number of identities and bypass the limiter entirely.

Weight expensive routes rather than counting all requests as one. A report endpoint costing 30 database seconds should consume more tokens than a health check.


#Respond correctly

yaml
HTTP/1.1 429 Too Many Requests
Retry-After: 30
RateLimit-Limit: 1000
RateLimit-Remaining: 0
RateLimit-Reset: 30
  1. 429, never 403 — clients and SDKs retry on 429 and give up on 403.
  2. Retry-After is mandatory. Without it, well-behaved clients retry immediately and make the overload worse.
  3. Send RateLimit-* headers on successful responses too, so clients can slow down before they are blocked.
  4. Use 503 with Retry-After for whole-service overload, distinct from a per-caller 429.

Publish the limits in your documentation. An undocumented limit is discovered during an integration's launch.


#Distributed enforcement

Per-instance counters mean the effective limit is limit × instances, and it changes when you autoscale.

  1. Use a shared store (Redis) with an atomic check-and-decrement — the Lua script above, not GET then SET.
  2. Decide the failure mode explicitly: if the limiter's store is unavailable, do you fail open (serve traffic, unprotected) or fail closed (reject everything)? Fail open is usually right for an API, fail closed for a login endpoint. Write the decision down.
  3. Prefer enforcement at the edge (CDN/WAF/API gateway) for volumetric abuse — a request rejected at the edge costs you nothing.
  4. Local in-process limiting is a reasonable second layer, never the only one.
nginx
# Edge layer: reject volumetric abuse before it reaches an application process
limit_req_zone $binary_remote_addr zone=api:20m rate=20r/s;
limit_req      zone=api burst=40 nodelay;
limit_req_status 429;
ts
// Application layer: quota per API key, atomic, shared across instances
const { allowed, remaining } = await bucket.take(`rl:key:${apiKey}:${route}`, {
  refillPerSecond: 10,
  capacity: 100,
  cost: ROUTE_COST[route] ?? 1,
});
if (!allowed) return res.status(429)
  .set({ "Retry-After": "30", "RateLimit-Remaining": "0" }).end();
LayerEnforcesStore
CDN / WAF (Cloudflare, Fastly)Volumetric floods, bot trafficEdge-local
API gateway (Kong, Envoy, APISIX)Per-consumer plan quotasRedis
Application middlewareRoute-weighted business limitsRedis
Database / connection poolFinal backstopstatement_timeout

#Do not punish legitimate clients

  1. Warn before enforcing. Ship a new limit in log-only mode, measure who would have been blocked, then enforce.
  2. Give a higher burst allowance than the sustained rate; real clients are bursty.
  3. Exempt health checks, and internal service-to-service traffic that has its own backpressure.
  4. Never permanently lock an account on rate-limit breach — that is a denial-of-service primitive against your own users. Use temporary backoff.
  5. Provide a documented path to a raised limit.

#Anti-patterns

Anti-patternWhy it failsFix
Fixed window for protection2× the limit across the boundaryToken or sliding window
Read-then-write counterRaces under concurrencyAtomic Lua / INCR
Per-instance countersEffective limit scales with instance countShared store
Trusting leftmost X-Forwarded-ForForgeable; limiter fully bypassedTrusted-proxy hop count
IP-only limitingNAT punishes many; attackers rotateMultiple keys together
Account-only limitingOne host sprays many accountsAdd per-IP
403 for rate limitsClients do not retry429
429 without Retry-AfterImmediate retries worsen overloadAlways include it
Headers only on rejectionClients cannot self-regulateSend on success too
All routes weighted equallyExpensive routes dominate costWeighted token cost
Undocumented limitsDiscovered during a customer launchPublish them
Permanent lockoutSelf-inflicted DoSTemporary backoff
Enforcing a new limit without measuringBreaks existing integrationsLog-only rollout first
Undefined store-failure behaviourUnpredictable under partial outageExplicit fail-open/closed

#Checklist

  • Each limiter's purpose — protection or quota — is stated
  • The algorithm is token bucket or sliding window, not a fixed window
  • Counter updates are atomic
  • Limits are enforced from a shared store, not per instance
  • Limiting keys on both identity and network address
  • Client IP is derived from a trusted-proxy hop count
  • Expensive routes consume proportionally more budget
  • 429 is returned with Retry-After
  • RateLimit-* headers are sent on successful responses
  • Service-wide overload returns 503, distinct from per-caller 429
  • Store-unavailable behaviour is an explicit, documented decision
  • Volumetric abuse is rejected at the edge
  • New limits ship in log-only mode first
  • Limits are documented, with a path to request an increase