How API Rate Limiting Actually Works

Most teams describe rate limiting with one line of product copy. It usually sounds like this: six hundred requests per minute, or one hundred requests per IP, or five login attempts before lockout. That line is useful for documentation. It is a poor description of the mechanism.

A real rate limiter is a control system for shared capacity. It decides who is consuming budget, what kind of work spends that budget, where the counter lives, how bursts are shaped, which layer enforces the decision, and what the client is told when the budget is gone. Those choices change whether the limiter feels fair, whether it protects the origin, whether it accidentally punishes users behind carrier NAT, and whether an attacker can turn the edge into a very expensive denial-of-service amplifier.

This article goes through the mechanics carefully. We will look at identity keys, fixed windows, sliding windows, token buckets, leaky-bucket style smoothing, distributed counter storage, 429 responses, shared-store races, layered enforcement, and the operational mistakes that make a limiter either porous or hostile.

Rate Limiting Is A Capacity Control Problem, Not A Decoration On Error Handling

The first mistake is to think of rate limiting as a special HTTP error path.

That is the visible symptom. The real job is upstream of that.

A service has finite capacity in several different dimensions:

• CPU time
• database concurrency
• cache bandwidth
• outbound calls to dependencies
• money spent on third-party APIs
• state growth in queues or logs
• human tolerance for abuse, such as password guessing or SMS spam

A limiter exists to keep one actor from consuming a disproportionate share of one of those resources.

That means the service owner must decide what they are actually protecting.

Consider three endpoints:

1. GET /profile reads a cached document in 4 ms.
2. POST /password/reset sends an SMS that costs money.
3. POST /search fans out to Elasticsearch and a reranker model.

If you apply one flat request counter to all three, you have hidden the real cost model.

The problem gets worse in modern systems because the expensive part is often not the frontend itself. The expensive part is a dependency behind it:

• an LLM call that costs euros per thousand tokens
• a fraud service billed per transaction
• a card-network authorisation attempt with contractual limits
• a slow relational query that pins a hot lock

A good limiter therefore answers two questions before it counts anything:

1. who is the consumer?
2. what budget is being consumed?

Without those two decisions, the rest of the algorithm talk is theatre.

The Identity Key Matters More Than The Counter Algorithm Most People Start With

The phrase per user or per IP sounds simple until you deploy it.

Identity in APIs is messy.

Per IP

Per-IP rate limiting is useful at the edge because it is cheap and often the only identity available before authentication. It is also blunt.

Many legitimate users can share one public IP:

• an office in Brussels behind one egress NAT
• a university in Prague
• a mobile carrier gateway in rural Greece
• a hotel WiFi in Lisbon

One attacker can also spread across many IPs by using cloud instances, botnets, or residential proxies.

Per-IP limits therefore catch some crude abuse and misclassify some normal traffic.

Per account or per API key

Per-account limits are better once the caller is authenticated. They map more cleanly to real tenants and commercial plans. They still have traps:

• one account may serve many human users
• one tenant may legitimately drive batch traffic at night
• a compromised key now exposes the whole account budget

Composite keys

Real systems often need more structure.

A composite key can include several dimensions:

tenant_id + api_key_id + route_group + method

or for a login flow:

source_ip + username

or for a search cluster:

tenant_id + endpoint_cost_class

This lets you express policy more accurately.

For example:

• limit unauthenticated /login by IP and by username
• limit /search more tightly than /profile
• limit one tenant's expensive write endpoints separately from cheap reads
• limit one webhook source independently of the rest of the account

Weighted cost units

Sometimes one request should count as more than one unit.

Suppose a SaaS platform in Frankfurt exposes these endpoints:

GET /catalog/item        cost = 1
POST /search            cost = 3
POST /render-pdf        cost = 8
POST /llm/summarise     cost = 25

If every call counts as one request, the budget does not match reality. A user can burn through GPU time or third-party spend while technically staying under the request cap.

A cost-aware limiter tracks units rather than raw request count:

remaining_budget -= route_cost

That is often a more honest system than the conventional one-number marketing limit.

The difficult part is social, not mathematical. Once you introduce weighted costs, your product, billing, documentation, and observability all need to agree on what a request budget means.

Fixed Windows Are Cheap And Popular Because They Collapse To One Counter And One Expiry

The simplest serious limiter is the fixed window.

Pick a window length, often 60 seconds. Store one counter per identity key for that window. Reject once the counter reaches the limit.

Pseudo-code:

window = floor(now / 60)
key = tenant_id + ':' + route + ':' + window
count = increment(key)
if count == 1:
    expire(key, 60)
if count > limit:
    reject
else:
    allow

That is popular because it is cheap:

• one key per active identity per window
• one atomic increment operation
• natural TTL cleanup
• easy mental model for dashboards

For many internal services, this is good enough.

The problem is the boundary.

Suppose the limit is 6 requests per minute.

A caller sends 6 requests at 12:00:59.700 and then 6 more at 12:01:00.050.

The label says 6 requests per minute. The system just allowed 12 requests in about 350 ms of real time.

That is the fixed-window cliff.

The general short-burst bound is close to twice the nominal rate. The attacker spends the tail of one window and the head of the next.

Timeline:

12:00:59.700  request 1  allowed
12:00:59.760  request 2  allowed
12:00:59.820  request 3  allowed
12:00:59.880  request 4  allowed
12:00:59.940  request 5  allowed
12:01:00.000  window resets
12:01:00.010  request 6  allowed
12:01:00.040  request 7  allowed
12:01:00.070  request 8  allowed
12:01:00.100  request 9  allowed
12:01:00.130  request 10 allowed
12:01:00.160  request 11 allowed

The limiter is not buggy. It is doing exactly what the algorithm implies. The algorithm simply does not produce a smooth rate shape.

That makes fixed windows acceptable for some problems and poor for others.

Good fixed-window use cases

• cheap edge abuse filters
• low-stakes admin APIs
• internal tooling where rough fairness is acceptable
• coarse plan limits where sub-second burst shape does not matter

Bad fixed-window use cases

• login throttling
• payment attempt control
• expensive search or LLM endpoints
• anything where a short spike can overload a dependency before the minute looks large on a graph

The fixed window is the right place to start because it teaches the problem cleanly. It is not the right place to stop for many internet-facing systems.

Sliding Windows Try To Make The Budget Match Real Elapsed Time

A sliding limiter exists because time does not naturally reset on a neat boundary.

There are several ways to approximate or implement a sliding window.

Sliding log

The most literal version stores timestamps for recent accepted requests.

For each new request:

1. delete timestamps older than now - window
2. count the remaining timestamps
3. allow only if the count is below the limit
4. append the new timestamp if allowed

Pseudo-code:

trim all entries older than now - 60s
if len(entries) >= limit:
    reject
else:
    append(now)
    allow

This is conceptually clean because it represents the actual last 60 seconds. It is also expensive at scale because you are storing and trimming many timestamps.

A Redis sorted-set implementation is common for moderate traffic:

ZREMRANGEBYSCORE key -inf (now - window)
current = ZCARD key
if current < limit:
    ZADD key now request_id
    EXPIRE key window
    allow
else:
    reject

It works. The memory and write amplification can become painful on hot keys.

Weighted sliding window

A cheaper approximation keeps two counters:

• the current window
• the previous window

Then it weights the previous window according to how much of it still overlaps the moving interval.

Example with a 60-second window, 15 seconds into the current minute:

effective_count = current_count + previous_count * 0.75

because 45 of the previous 60 seconds are still inside the last-minute view.

This is not perfect, but it is much smoother than a hard reset and much cheaper than a full request log.

GCRA and arrival-time style models

Telecom and networking literature often express rate control in terms of a theoretical arrival time rather than explicit counters. Generic Cell Rate Algorithm, or GCRA, is one popular formulation.

Instead of counting requests in buckets, it tracks when the next request would be acceptable under a configured rate and burst tolerance. Each new request either advances the schedule or gets rejected for arriving too early.

Conceptually:

allowed if now >= theoretical_arrival_time - burst_tolerance
then update theoretical_arrival_time

This produces very smooth behaviour and maps well to the idea of a continuous rate rather than a resettable minute.

The implementation details vary, but the important point is that sliding and GCRA-style models are attempts to make the effective limiter match real passage of time rather than calendar edges.

Why this matters operationally

If the protected dependency is sensitive to short spikes, a sliding or arrival-time model is often the difference between a stable graph and an incident.

A queue does not care that it is a new minute. A PostgreSQL connection pool does not care that a counter key expired. Your limiter should care about what the dependency experiences, not what looks tidy in a dashboard bucket.

Token Bucket Is Often The Most Practical Compromise For Public APIs

Many public APIs want two properties at the same time:

1. allow a legitimate short burst
2. prevent sustained overuse

Token bucket is a good fit for that shape.

The model is:

• the bucket has a maximum capacity
• tokens refill at a constant rate
• each request spends one or more tokens
• if not enough tokens remain, the request is rejected or delayed

The core update is:

tokens = min(capacity, tokens + refill_rate * delta_seconds)
if tokens >= cost:
    tokens -= cost
    allow
else:
    reject

Suppose the bucket capacity is 10 and the refill rate is 2 tokens per second.

• after 5 idle seconds, an empty bucket refills to 10
• a caller may then burst 10 unit-cost requests immediately
• after that, only 2 unit-cost requests per second are sustainable

That is usually close to what product people mean when they say a client should be able to burst a little but not sustain abuse.

Why token bucket feels fairer

A fixed window might allow a burst because of an accidental reset boundary. Token bucket allows a burst because the client genuinely accumulated unused budget.

That is a much more intuitive story.

Weighted requests fit naturally

Token bucket also works well with weighted costs:

GET /profile       cost = 1
POST /search       cost = 3
POST /llm          cost = 15

Now the same mechanism expresses both fairness and cost control.

Token bucket versus leaky bucket

These names are often used loosely, so the useful distinction is behavioural.

• Token bucket allows bursts up to stored capacity, then enforces the refill rate.
• Leaky bucket is often used to mean a queue or drain process that emits work at a steady pace.

In API discussions, people sometimes say leaky bucket when they really mean any smoothed rate controller. The real question is whether the algorithm allows saved-up burst credit or enforces a nearly constant output cadence.

If the service can tolerate short bursts but not sustained excess, token bucket is usually the more natural public-API model.

If the goal is to shape outbound work to a dependency at a strict drain pace, a queue with leaky-bucket-like behaviour may be a better description.

Distributed Rate Limiting Is Mostly A Shared-State Problem Disguised As A Math Problem

Once a service runs on more than one worker or more than one edge location, the limiter stops being local.

Imagine three application instances in Dublin behind a load balancer. The limit is 10 requests per minute per API key. One hot client lands requests on all three instances.

If each instance keeps its own in-process counter, the client can get almost 30 requests through before any one instance notices.

That is not rate limiting. That is three unrelated guesses.

Shared store

The usual answer is a shared store such as Redis, a strongly consistent database row, or a platform-specific coordination service.

The shared store must support an atomic decision pattern:

1. read current state
2. compute whether the request is allowed
3. update state if allowed
4. return the decision

Those steps have to behave like one operation from the caller's perspective.

If they are split into separate client round trips without isolation, races appear:

worker A reads count = 9
worker B reads count = 9
worker A decides allow and writes 10
worker B decides allow and writes 10

Two requests passed when only one budget slot remained.

That is why Redis Lua scripts and similar server-side evaluation patterns are common.

A representative fixed-window script shape is:

local current = redis.call('INCR', KEYS[1])
if current == 1 then
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end
if current > tonumber(ARGV[2]) then
  return {0, current}
end
return {1, current}

For token bucket or sliding-window logic, the script becomes more complex because it has to store timestamps or token state. The principle stays the same: the shared system makes the decision atomically.

Multi-region complications

Things get more interesting when traffic lands in Madrid, Frankfurt, and Warsaw at the same time.

Now you have to choose between:

• one globally consistent limiter with cross-region latency
• region-local limiters with eventual convergence and some leakage
• a hybrid design with coarse global budget and fine local budget

There is no free answer.

A fully centralised store gives better correctness and worse latency. A purely regional store gives better latency and weaker global accuracy.

Some systems split the problem by layer:

• coarse edge limits per IP in-region
• stronger per-account limits in a shared core store
• very expensive endpoint quotas in a single authoritative ledger

That is not elegant on a whiteboard. It often matches production reality better than pretending one algorithm solves every dimension.

Fail-open or fail-closed

The shared store itself can fail.

If Redis is unavailable, what should the API do?

• Fail open: allow traffic so the service stays available, accepting abuse risk.
• Fail closed: reject traffic because the limiter is part of safety, accepting availability loss.

The answer depends on the endpoint.

For login throttling, failing open during an outage may invite brute-force attacks exactly when your observability is already degraded.

For a low-risk read-only catalogue API, failing closed may create a self-inflicted outage that is worse than the abuse you were preventing.

Serious systems classify endpoints by failure policy rather than assuming one universal answer.

A Good Limiter Has Clear Wire Semantics, Not Just Internal State

The client needs to know what happened.

The core rejection status is HTTP 429, standardised in RFC 6585 as Too Many Requests.

That status alone is not enough for well-behaved clients. A decent response usually also carries timing or budget hints.

Retry-After

The most widely understood field is Retry-After.

Example:

HTTP/1.1 429 Too Many Requests
Retry-After: 12
Content-Type: application/json
 
{
  'error': 'rate_limited',
  'message': 'Search budget exhausted for this API key'
}

That tells the client to wait 12 seconds before retrying.

It is useful when the limiter really knows a reasonable next-attempt time. It is less useful when the system is probabilistic or multi-layered and cannot produce one honest number.

Rate-limit headers

By mid-2026, API ecosystems still contain several header families:

• older X-RateLimit-* conventions
• vendor-specific RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset
• the newer IETF draft model based on RateLimit and RateLimit-Policy

A representative modern shape looks like this:

RateLimit-Policy: 'default';q=100;w=60
RateLimit: 'default';r=12;t=8

Read that as:

• policy default
• quota 100 per 60 seconds
• remaining 12
• reset or retry horizon 8 seconds

Whatever header family you choose, consistency matters more than fashion. Clients need to know whether the numbers refer to:

• the current window
• the most restrictive active policy
• a token bucket's remaining credit
• the next retry time
• a daily plan quota rather than a short-burst limiter

If documentation and wire behaviour drift apart, client retries become guesswork.

429 is not the only possible pressure signal

Some systems also use:

• 403 for policy-level refusal after abuse escalation
• 503 when the backend is unavailable rather than budget-limited
• delayed responses instead of immediate rejections for anti-bruteforce logic
• CAPTCHA or proof-of-work challenges at the edge

That is fine as long as the semantics stay clear.

A login endpoint, for example, may intentionally slow responses after repeated failures instead of returning clean 429s, because slowing the attack is safer than giving a crisp timing oracle. A public data API usually wants the opposite: fast honest rejections with accurate reset metadata.

Layered Rate Limiting Works Better Than Pretending One Counter Protects Everything

Large systems often need several limiters at different layers because they are protecting different things.

Edge limiter

This is the first coarse filter.

Common use cases:

• per-IP abuse control
• bot suppression
• cheap volumetric filtering before the origin

The edge limiter should be fast and blunt.

API gateway or app limiter

This layer usually has richer identity:

• authenticated API key
• tenant plan
• route cost class
• method

This is where business policy often lives.

Dependency-specific limiter

Downstream systems may need their own protection:

• database concurrency caps
• search-cluster query budgets
• per-rail payment attempt budgets
• LLM spend caps

This layer is not redundant. A general app budget may still allow one tenant to saturate one dependency shape.

Queue and worker limiter

If accepted requests enqueue background jobs, the limiter may need to exist on the enqueue path and on the worker execution path. Otherwise the API politely accepts traffic and explodes later in a queue you forgot to guard.

A mature design therefore often looks like this:

CDN / WAF           -> per-IP volumetric control
API edge            -> per-key request budget
Route-specific app  -> weighted per-endpoint budget
Worker / dependency -> concurrency or spend budget

That is not duplication. It is scoped protection.

One counter at one layer cannot express all of those constraints honestly.

The Hard Failures Come From Operational Mismatches, Not From Forgetting The Formula

Most production limiter bugs are not about mathematics. They are about environment.

Shared NAT false positives

Per-IP limits can punish classrooms, offices, hotels, mobile carriers, and corporate egress gateways. The limiter works exactly as designed and still produces a bad user experience.

Key cardinality explosions

If you key by too many dimensions, your state store explodes. If you key by too few, your fairness collapses.

For example:

user_id + route + method + region + user_agent + locale + build_number

might be technically precise and operationally absurd.

Clock assumptions

Sliding-window approximations, TTLs, and reset metadata all depend on time. Region skew, stale clocks in edge nodes, or inconsistent monotonic versus wall-clock use can create strange rejection patterns.

Thundering herd on reset

If clients know the reset moment exactly and all wake up together, the limiter itself can create synchronous burst traffic. Token-bucket style designs reduce this problem. So does jitter in client backoff.

Asymmetric retries

A client that retries aggressively after 429 can amplify the problem. A client SDK should understand the limiter semantics. If it treats 429 like a transient network blip and retries with the same backoff as 503, you built a traffic multiplier.

Hidden write amplification

A sliding-log limiter on a hot public endpoint can turn into a storage write problem. You protected the database and overloaded Redis instead.

Blind plan limits

A daily quota and a short-burst limiter are different tools. Many APIs document only the daily quota and then surprise clients with per-second or per-minute enforcement they never explained.

Cost leakage through side paths

Teams sometimes protect the synchronous API call but forget that the accepted request triggers:

• a webhook fan-out
• a PDF render job
• an email send
• a vector embedding task
• a fraud-screening call

The apparent limit holds. The real cost leaks elsewhere.

Observability that shows only blocks

If you only graph rejected requests, you are missing half the story. You also need:

• keys nearing exhaustion
• top spenders by weighted budget
• shared-store latency
• fail-open versus fail-closed counts
• mismatch between edge and app-layer decisions
• region skew in acceptance rates

A limiter is control infrastructure. It needs instrumentation like any other control infrastructure.

The Right Algorithm Depends On The Endpoint And The Failure You Care About

There is no one best limiter. There is a best fit for a specific traffic shape and risk.

Login and password reset

Goals:

• slow brute force
• avoid user-enumeration leaks
• handle hostile retries

Typical shape:

• per-IP and per-username composite keys
• sliding window or delayed-response control
• optional challenge step after threshold
• fail-closed bias for serious outages

Public read API

Goals:

• fairness
• origin protection
• predictable client semantics

Typical shape:

• per-API-key budget
• token bucket or sliding window
• clear 429 and retry metadata
• generous burst capacity for legitimate short spikes

Search or analytics endpoint

Goals:

• protect expensive dependencies
• reflect cost differences across query shapes

Typical shape:

• weighted budgets
• per-tenant cost units
• concurrency caps in addition to request budgets
• strong observability on high-cost keys

Paid third-party dependency

Goals:

• stop cost explosions
• preserve margin

Typical shape:

• low-latency frontend limiter plus hard daily spend quota
• route cost weights
• emergency global kill switch
• alerting well before actual exhaustion

Webhook ingestion

Goals:

• absorb legitimate retry bursts
• reject floods without breaking valid senders

Typical shape:

• per-provider or per-endpoint key
• token bucket with moderate burst allowance
• idempotency and queue protection behind the limiter

These are different problems. If you use the same per-minute fixed window for all of them, the simplicity is probably fake.

Concurrency Limits And Rate Limits Solve Different Bottlenecks

Rate and concurrency are related. They are not the same control.

A rate limit says how much work may begin over time:

100 requests per second

A concurrency limit says how much work may be in flight at once:

no more than 25 renders running now

Why the distinction matters becomes obvious when request duration changes.

Suppose a PDF rendering endpoint receives 10 requests per second.

• If each render takes 50 ms, the steady in-flight load is small.
• If each render takes 4 seconds, the same arrival rate creates a large queue and worker pile-up.

Little's law gives the intuition:

concurrency ≈ arrival_rate × service_time

So:

10 req/s × 0.05 s  = 0.5 in flight on average
10 req/s × 4 s     = 40 in flight on average

The rate did not change. The concurrency did.

That is why a pure rate limiter can still leave a system overloaded. It controls arrivals, not how long work occupies scarce resources.

Typical places where concurrency matters more than rate

• a database connection pool with 60 available connections
• a GPU inference worker fleet with 8 model slots
• a card-network client with a small number of parallel sockets
• a PDF renderer or image pipeline that holds large memory buffers
• a webhook delivery worker that can be blocked by slow receivers

In those environments, a concurrency limiter is often the control that protects the actual bottleneck.

Pseudo-code:

if in_flight >= concurrency_limit:
    reject_or_queue
else:
    in_flight += 1
    try:
        handle_request()
    finally:
        in_flight -= 1

That is a different shape from token bucket or sliding window. One is about occupancy. The other is about arrival rate.

Queueing is not a free substitute

Some teams say they do not need concurrency limits because they already have a queue. A queue can help, but it changes the failure mode rather than removing it.

If the queue is unbounded, overload becomes memory growth and long-tail latency instead of immediate rejection. If the queue is bounded, the queue itself becomes another limiter and you still need a policy for overflow.

Reasonable designs often combine the controls:

• rate limit at the edge to stop obvious floods
• concurrency limit at the expensive worker pool
• bounded queue in between

That gives the system three chances to fail gracefully instead of one giant blob of pressure.

Concurrency can also be partitioned

If one tenant is noisy and one tenant is premium, the worker limit may need to be partitioned or weighted the same way rate budgets are partitioned.

Example:

global render slots = 40
tenant A cap = 12
tenant B cap = 4
free-tier shared pool = 8
premium shared pool = 16

This is not purely a fairness issue. It stops one customer or one traffic class from monopolising the entire in-flight pool during a slow dependency incident.

When people say rate limiting saved their system, they often mean a bundle of controls saved it. The rate limiter prevented infinite arrival growth. The concurrency limit prevented slow work from occupying every slot. The queue absorbed the short mismatch between supply and demand.

Treat them as separate tools or the wrong bottleneck will stay unprotected.

Client Backoff, Jitter, And Idempotency Are Part Of The Limiter Design

A limiter is not finished when the server emits 429. The client behaviour after 429 determines whether the limiter calms the system or simply turns bursts into synchronised retry storms.

Backoff should follow the limiter semantics

If the server says:

Retry-After: 10

the client should not retry after 200 ms because its generic network stack retries all 4xx or 5xx responses on a fixed schedule. That sounds obvious. It fails in real SDKs all the time.

A polite client retry loop looks more like:

if status == 429:
    wait = retry_after or computed_backoff()
    sleep(wait + jitter)
    retry

The jitter matters. If ten thousand clients all wake exactly when the limit resets, the limiter becomes a metronome for herd behaviour.

Common jitter strategies include:

• full jitter: random value between zero and the backoff ceiling
• equal jitter: half the backoff plus a random half
• decorrelated jitter: next delay depends partly on the previous delay

The exact formula matters less than the principle: do not let every client align on the same second boundary.

Idempotency changes what is safe to retry

For safe reads, retrying after 429 is usually straightforward.

For writes, you need to ask whether the first attempt might already have been accepted upstream or partially executed before the limiter or gateway reported failure back to the caller.

That is why serious write APIs pair retry guidance with idempotency keys.

Example:

POST /payments
Idempotency-Key: 6f4dcb8b-9d73-4f73-8eb6-5b8c96ad4c6e

Now the client can retry after a timeout or 429-related ambiguity without duplicating the financial action. The limiter and the write semantics reinforce each other.

Slowdown versus hard rejection

For some abuse cases, slowing a caller is better than returning a crisp block. Login throttling is the classic case. If a password attacker can learn exactly when the limit resets, the limiter becomes predictable. Introducing response delay, challenge steps, or progressive cooldown can be more useful than a clean per-minute wall.

That also means the client experience has to be designed consciously. A human user who mistypes a password twice should not feel as if the system is broken, while an automated spray should feel increasingly expensive.

SDKs are part of the contract

If you publish an API and also publish official SDKs, the SDK is where good limiter citizenship becomes real:

• honour Retry-After
• surface remaining budget when available
• stop retrying non-idempotent writes blindly
• add jitter automatically
• expose hooks so callers can log limiter state

Without that, every customer reinvents retry logic badly and your server pays for their mistakes.

Circuit breakers and client-side shedding

Advanced clients sometimes add their own local budgeting on top of server limits. If a desktop sync client already knows it is sending too fast, it can shed work locally before the server has to reject it. That saves bandwidth and smooths user experience.

In other words, the cleanest rate-limited system is often cooperative on both sides:

• the server publishes honest limits and timing
• the client respects them and spreads retries out
• idempotency makes safe retries possible
• jitter stops the whole fleet from stampeding on reset

The limiter is therefore not only an algorithm at the edge. It is part of an end-to-end behaviour contract between caller and service.

Rate Limiting Is Honest Only When The Budget, Identity, And Enforcement Layer Agree

The simplest possible rate limiter is a counter with an expiry. That can still be useful.

The honest production limiter is more opinionated. It decides whose traffic is being measured, what kind of work the budget represents, how bursts are shaped, where shared state lives, how failures behave, and what clients are told on rejection.

That is why the one-line documentation limit is only the outer label.

Inside the system, the real questions are sharper:

• is the identity key fair?
• is the budget tied to actual cost?
• does the burst shape protect the dependency you care about?
• do all workers see the same remaining budget?
• can the client back off intelligently?
• will the limiter fail in the direction you intended during an outage?

Once you ask those questions, the architecture usually becomes clearer.

• Fixed windows are cheap and coarse.
• Sliding models reduce reset cliffs.
• Token buckets allow intentional bursts while preserving an average.
• Shared atomic state is mandatory once workers multiply.
• Layered enforcement protects different bottlenecks honestly.

A good rate limiter is not the one with the fanciest algorithm name. It is the one whose internal mechanics match the actual pressure points of the system it is supposed to protect.