Which rate limiting algorithm would you choose for a public API, and why?

A token bucket is a common choice: it allows short bursts up to the bucket size and a steady average rate after that, and it needs little memory. A sliding window is more accurate if you must never exceed a hard limit. A fixed window is simplest but allows up to twice the limit across a boundary. The trade-off is accuracy against memory and complexity.

You run twenty API servers. How do you enforce one limit per user across all of them?

Keep the counters in a shared store such as Redis, keyed by user and window, and update them atomically with an increment or a script. Local counters would give each user twenty times the limit. The trade-off is a network call per request, which you can reduce with local batching at the cost of some accuracy. At large scale, shard the counters by user key across several Redis nodes, watch for one very busy key, and remember that replication is asynchronous, so a failover can lose recent counts.

Your Redis store goes down. Should the rate limiter reject requests or allow them?

Decide per endpoint. For general traffic, fail open so the service stays up, and use a small local in-memory limit as a fallback. For sensitive endpoints such as login, fail closed. Set a short timeout on the store so a slow store does not slow every request. The trade-off is availability against protection.

What identifies the client you are limiting, and what are the problems with each choice?

An API key or user ID is best because it identifies the caller. An IP address is a fallback but shared networks block many users together, and attackers rotate addresses. Combine them: a key where you have one, an address limit for anonymous traffic. The trade-off is precision against being able to limit callers who are not logged in.

What should the API return when a client is over its limit?

HTTP 429 Too Many Requests, with a Retry-After header, and headers showing the limit, the remaining count and the reset time. This lets clients back off instead of retrying at once. The trade-off is exposing limit information, which is usually worth it for friendly clients.

Rate Limiter | TheLearningLoop

Why you need a rate limiter, and when to use one

Every public API has a limit on what it can serve. One buggy client in a loop, a scraper or an attacker can use all of it and leave real users with errors. A rate limiter caps how many requests a client may make in a period and rejects the rest, usually with HTTP status 429 Too Many Requests.

It gives you two things interviewers always ask about:

Availability: one noisy client cannot take the service down for everyone.
Fairness and cost control: each user gets a share, and expensive calls (login attempts, paid third-party APIs) stay bounded.

Use one when:

The API is public or used by many tenants.
Calls are expensive or have a per-user quota (paid plans).
You need to slow brute-force logins or scraping.
A downstream service has a hard limit you must respect.

Skip it, or use something else, when:

Traffic is internal and trusted, and capacity is simply too low: add capacity or a queue instead.
The problem is a sustained overload from everyone: use load shedding or autoscaling, since a per-client limit does not help.
You can accept slower responses: a queue absorbs the burst without rejecting.

Try it

Send ten requests under a limit of five per ten ticks, then change the algorithm and the traffic.

Rate Limiter

Step 1 of 12 · Press Space to play, arrow keys to step

AlgorithmTraffic

Initial state. The rule is at most 5 requests per 10 ticks per client. Requests arrive at ticks 8, 8, 9, 9, 9, 10, 10, 11, 11, 11.

tick:: 0
allowed:: 0
rejected:: 0
most allowed in any 10 ticks:: 0

1 tick = one step of the simulation.

Speed

Things to try:

Run Fixed window counter with the burst at the window boundary. The counter resets at tick 10, so ten requests get through in four ticks, double the limit.
Switch to Sliding window log or Token bucket with the same traffic and compare the most requests allowed in any ten ticks. Expect 5 for the sliding log and 6 for the token bucket: the bucket starts full with five tokens and refills a little before the burst ends. A token bucket removes the reset at the clock edge, but it still allows a burst up to the bucket size plus whatever refills.
Watch the counters. The counter named tick counts simulation steps, while the rule and the request times are in ticks of time, so on steady traffic one step is not one tick.
Choose Limiter store is down (fail open) and note that nothing is limited. Then decide whether your service should fail open or closed.

Rate limiting algorithms

Algorithm	How it works	Good when	Weakness
Fixed window counter	Count requests per clock window, reset at the boundary	You want the simplest, cheapest option	A burst at a boundary can send up to twice the limit
Sliding window log	Keep the time of every request and count those in the last window	Accuracy matters and traffic per client is small	Memory grows with the number of requests
Sliding window counter	Weight the current and previous window counts	An accurate-enough, cheap compromise	An approximation, not exact
Token bucket	A bucket refills at a steady rate and each request takes a token	Bursts are allowed up to a cap	Two settings to tune: rate and size. A burst up to the bucket size still gets through
Leaky bucket	Requests enter a queue that drains at a fixed rate (the queue form; the meter form acts like a token bucket, which is how NGINX `limit_req` works)	You need a smooth output rate	Bursts wait in the queue, adding delay

Where to put the limiter

At the edge (API gateway or load balancer): blocks abuse before it reaches your code, but often sees only the key or address, not the user's plan.
In the application: can limit by user, plan or endpoint, at the cost of every server needing the same counters.
In a shared store (for example Redis): all servers update the same counters, so the limit is global across the fleet.

Limit by whatever identifies the caller: API key, user ID, or IP address as a fallback. IP addresses are shared by many users behind one network, so they are a weak key.

How the shared counter works

With many servers, each one cannot keep its own count or a client would get the limit multiplied by the number of servers. The common approach is:

Build a key such as rl:user42:<window>.
Increment it in Redis and set an expiry on first use. Two commands do this (INCR, then EXPIRE with NX, which needs Redis 7.0 or later) and they heal themselves if the second is lost. A script or MULTI makes it strictly atomic, meaning the steps run as one with no other request in between. The expiry is a TTL (time to live): how long Redis keeps the key before deleting it.
If the count is above the limit, reject with 429 and a Retry-After header.
Return headers that show the remaining quota so clients can slow down.

Use a script or an atomic command, because a separate read and write can race when two servers act at once.

Sizing and scaling a rate limiter

Every check is a round trip to the shared store, so the limiter adds a network hop to each request. The numbers below are rough and illustrative. Measure your own.

Cost per request: about one store call, often around a millisecond inside one data centre. This is the main cost of a strict global limit.
Memory: a counter is small. If a key costs around a hundred bytes, a million active clients need roughly a hundred megabytes.
Throughput: one Redis node can handle very many simple commands a second, but a busy API can outgrow it. Spread the counters across several nodes by hashing the client key.
Hot keys: one very busy client lands on one node. A small local limit in front of the store absorbs most of that traffic.
Failover: Redis replication is asynchronous, so after a failover the newest counts may be lost and clients briefly get a fresh allowance. Most services accept this.
Window resets: if every client retries the moment a window ends, the load arrives in a spike. Add a little random delay to Retry-After.
Several regions: a single global counter costs a cross-region call. Many systems give each region a share of the limit instead and accept small errors.
Responses: return 429 when the caller is over its limit and 503 when the server itself is overloaded. Header names vary; many APIs use X-RateLimit-Limit, X-RateLimit-Remaining and Retry-After.

Rate limiter vs the alternatives

Concern	Rate limiter	Queue with a worker pool	Autoscaling
Stops one client using everything	Yes	No	No
Protects against sudden spikes	Yes, from one client (not from everyone at once)	Yes, by absorbing them	Slowly
Keeps every request	No	Yes	Yes
Cost of extra capacity	None, apart from the store call	Some	High
Adds delay to accepted requests	None	Yes	None
Best for	Abuse and fairness	Work that can wait	Real growth

Use a rate limiter for fairness and abuse, a queue when work can wait, and autoscaling when genuine demand grows.

Avoiding a new single point of failure

The limiter's store is now on the path of every request. Decide what happens when it fails:

Fail open: let requests through. The service stays up but is unprotected.
Fail closed: reject requests. The service is protected but unavailable.

A common choice is to fail open for general traffic and fail closed for sensitive endpoints such as login. Also keep a small in-memory local limit as a fallback, replicate the store, and set a short timeout so a slow store does not slow every request.

Common mistakes

Per-server counters. A client gets the limit times the number of servers.
Separate read then write. Two requests can both pass. Use an atomic operation.
Limiting only by IP. Shared networks get blocked together, and attackers rotate addresses.
No Retry-After header. Clients retry at once and make it worse.
Same limit for every endpoint. Login and search need different limits.
Forgetting the store failure case. Decide fail open or closed on purpose.

How to talk about a rate limiter in an interview

Ask who is limited (user, key, IP), where the limiter lives and how strict it must be.
Pick an algorithm and say why: token bucket to allow bursts, sliding window for accuracy.
Say that counters live in a shared store with atomic updates, and return 429 with Retry-After.
Name the trade-off: strict global limits cost a network call per request, so many systems accept small errors in exchange for speed.

Further learning

Caching: the same shared-store pattern, with eviction and invalidation.
Load balancer: another component on the path of every request, with its own failure modes.

Where you meet rate limiters in real systems

Almost every public API enforces limits, often with the response headers that show your remaining quota. Here are three drawn out, the first with configuration you can run, then more examples.

Example 1: Limiting requests per client

Two common setups: NGINX limits by client address at the edge, and a shared Redis counter limits by user across every server. Open the example below to see the picture and the configuration.

Config: an edge limit and a shared counterSetup 1 config and 2 Redis scriptsCode no code changes

The limiter looks up the caller's counter. If it is under the limit the request goes on. If not, the limiter answers 429 and the API never sees it.

events {}
http {
  # One shared 10 MB zone keyed by client IP, allowing 5 requests per second on average.
  limit_req_zone $binary_remote_addr zone=perip:10m rate=5r/s;
  server {
    listen 8080;
    location /api/ {
      # burst=10 queues short spikes; nodelay serves them at once; the rest get 429.
      limit_req zone=perip burst=10 nodelay;
      limit_req_status 429;
      # Placeholder upstream. Replace with your application.
      proxy_pass http://127.0.0.1:9000;
    }
  }
}

The NGINX file passed nginx -t on the official image. The Redis commands and the token bucket script were run against a real Redis 7 server: five calls were allowed, the sixth was rejected, and one more was allowed after two seconds of refill. Names and numbers are examples. The script trusts the time the caller passes in, so servers with different clocks can skew the refill. Production code often reads the time from Redis itself (TIME) and never lets the elapsed time go below zero.

Example 2: Limits by plan

In this illustrative example, a free plan allows 60 requests a minute and a paid plan allows 600. The limiter looks up the caller's plan and uses a different bucket size.

The gateway identifies the caller by API key, reads their plan limit, and checks the shared counter before forwarding the request.

Example 3: The boundary burst

A fixed window resets on the clock. A client that sends five requests at the end of one window and five at the start of the next gets ten through in a moment.

With a limit of five per window, the first five land at the end of window one and the next five at the start of window two. All ten are allowed in about four ticks.

More real-world scenarios

Scenarios 4 to 10 are illustrative. The numbers are examples, not any provider's real limits.

4. Login protection

A limit of a few attempts per minute per account slows password guessing. It fails closed, because an open limiter here would be a security hole.

5. Public APIs with quotas

Developer platforms give each key a quota and return headers with the remaining count and reset time, so well-behaved clients can slow down.

6. Web scraping defence

Sites limit by address and by behaviour, and challenge clients that read pages faster than a person could.

7. Third-party API calls

A service that calls a paid provider limits its own calls to stay inside the provider's quota and avoid extra charges.

8. Multi-tenant SaaS

Each customer gets its own limit, so one tenant's batch job cannot slow the others. This is the noisy neighbour problem.

9. Message and email sending

A sender limits messages per user per hour, which also protects the product's sending reputation.

10. Internal services

Services limit callers they do not trust to stay well-behaved, together with timeouts and retries with back-off.

Tools and managed services

Kind	Name	Notes
Edge proxy	NGINX `limit_req`, Envoy	Limit by address or header at the edge
Shared store	Redis	Atomic counters and scripts for global limits
Managed	AWS WAF rate-based rules, API Gateway throttling	Limits applied before traffic reaches your code
Managed	Cloud API gateways on Google Cloud and Azure	Offer quotas and throttling policies
Library	Token bucket libraries for most languages	In-process limits for one server

The takeaway

Whenever one caller could use up capacity meant for everyone, put a rate limiter in front, keep its counters in a shared store, and decide on purpose whether it fails open or closed.

Rate Limiter

Key takeaways

Why you need a rate limiter, and when to use one

Try it

Rate Limiter

Rate limiting algorithms

Where to put the limiter

How the shared counter works

Sizing and scaling a rate limiter

Rate limiter vs the alternatives

Avoiding a new single point of failure

Common mistakes

How to talk about a rate limiter in an interview

Further learning

Where you meet rate limiters in real systems

Example 1: Limiting requests per client

Example 2: Limits by plan

Example 3: The boundary burst

More real-world scenarios

4. Login protection

5. Public APIs with quotas

6. Web scraping defence

7. Third-party API calls

8. Multi-tenant SaaS

9. Message and email sending

10. Internal services

Tools and managed services

The takeaway

Interview questions