Blog Writer

title: "API Rate Limiting — 5 Patterns That Actually Scale" slug: api-rate-limiting-patterns-that-scale meta_description: "Five production-proven API rate limiting patterns with code examples. From token buckets to sliding windows — pick the right approach for your traffic profile." target_keyword: "API rate limiting" secondary_keywords:

• "rate limiting patterns"
• "token bucket algorithm"
• "API throttling best practices" word_count: 2847 estimated_read_time: "12 min" category: "Engineering" tags:
• api-design
• rate-limiting
• scalability cta_type: newsletter

API Rate Limiting — 5 Patterns That Actually Scale

Your API handles 10,000 requests per second until it doesn't. One aggressive scraper, one retry storm from a misbehaving client, and your database connection pool is exhausted before your on-call engineer finishes reading the PagerDuty alert. Rate limiting isn't optional infrastructure — it's the difference between a graceful 429 response and a cascading outage.

This post walks through the five patterns we have actually run in production at Northbeam Analytics — when each one is the right choice, when it stops being one, and the small implementation details that turn a clean algorithm into a 2 AM page.

TL;DR — Pattern Selection

1. Token Bucket — The Default Choice for Most APIs

The token bucket algorithm is the most widely deployed rate-limiting pattern for a reason: it handles bursts gracefully while enforcing a sustained rate ceiling. Tokens accumulate at a fixed rate (say, 100/second), and each request consumes one token. When the bucket is empty, requests receive a 429 until tokens replenish.

The implementation is short, the failure modes are well understood, and Redis MULTI/EXEC makes it safe under concurrency. The one trap: setting the bucket too large to "be generous" turns a rate-limit into an availability problem, because a single bad actor can drain the bucket and starve well-behaved clients before the refill catches up.

2. Sliding-Window Log — Exact, Expensive, Worth It Sometimes

The sliding-window log keeps a sorted set of recent request timestamps and counts how many fall inside the window for each new request. It is the most accurate of the five patterns and the most expensive. We run it on the public API where customers are billed per call; we do not run it on the internal RPC mesh where token bucket is fine.

3. Fixed-Window Counter — Don't

Fixed-window counters allow 2× the configured rate at the window boundary. We learned this the painful way during a retry storm: a misbehaving client got through 200 requests in two seconds against a "100 requests per minute" limit because it caught the second half of one window and the first half of the next. Use a sliding-window counter instead — same memory profile, no boundary attack.

4. Sliding-Window Counter — The Pragmatic Compromise

The sliding-window counter approximates the sliding-window log with two integer counters and a weighted average. Memory and per-request work look like fixed-window counter; the boundary behavior looks like sliding-window log. This is the pattern we recommend when token bucket isn't a fit and the log is too expensive.

5. Leaky Bucket — Concurrency, Not Throughput

Leaky bucket is often described as a token-bucket variant; it isn't. It enforces an outflow rate, not an arrival rate, and is the right primitive when the constraint is "this downstream can only handle N concurrent calls" rather than "this client gets N requests per second." We use it in front of a payment processor that throttles us at 50 in-flight transactions.

Pitfalls We've Hit

Internal Linking Suggestions

Social Distribution Copy

Twitter/X (under 280 chars): Most rate-limiting tutorials stop at token buckets. Five patterns we've actually run in production at 10K+ RPS — including the one that turned a retry storm into a quiet alert instead of a page. Thread.

LinkedIn (under 700 chars): Rate limiting seems simple until you're debugging a cascading failure at 2 AM. After implementing all five major patterns across different production systems, here's the unromantic ranking — including the one pattern that should not exist outside of tutorials, and the one most teams reach for too early.

This sample illustrates the skill's output format. Northbeam Analytics is a fictional company used recurringly across these sample outputs. Real client data is never included in sample outputs.