Rate Limiting

Now that Snip runs as many replicas behind nginx, we have a new problem that only exists because we scaled out. We want to cap each client at, say, 100 requests per minute — to fend off abuse and accidental hammering. The naive way is to count requests in a variable inside the process. With three replicas, that quietly becomes a 300/minute limit. Scaling out broke our limiter. This chapter fixes it the right way.

WHY before HOW

A rate limit is a shared promise: “this client gets 100 requests per minute, total.” A promise the whole fleet must keep can’t live in any one process’s memory — it has to live somewhere all replicas can see. For Snip, that’s Redis.

Why the counter must live in Redis

Picture three stateless app replicas (from the last chapter). If each keeps its own in-memory counter for client X:

in-memory per replica  →  app-1: 100   app-2: 100   app-3: 100   = 300 allowed  ✗
shared in Redis        →  rl:X = 300 across all three            = 100 allowed  ✓

A per-process counter multiplies the limit by the replica count — and the multiplier changes every time you scale. The only way the limit means “100 across the whole system” is if every replica increments one shared counter. Redis is perfect for this: it’s already a dependency, it’s fast (sub-millisecond), and INCR is atomic — concurrent increments from different replicas can’t lose a count or race. This is the same reason the cache and click queue live in Redis: shared state belongs in shared storage, not in the stateless app.

The fixed-window algorithm

The simplest correct limiter is fixed window: chop time into fixed buckets (here, 60-second windows), count requests per client per bucket, reject once the count passes the limit. The bucket resets when it expires.

The mechanics in Redis are three operations:

1. INCR rl:<client> — atomically bump this client’s counter and read the new value.
2. On the first hit (count just became 1), EXPIRE rl:<client> 60 — start the 60-second clock so the window auto-resets.
3. If the count is over the limit, return 429 Too Many Requests.

That’s the whole algorithm. Now the code.

The middleware

Here’s the rate_limit middleware from src/main.rs, verbatim:

/// Fixed-window rate limit per client (keyed by `X-Forwarded-For`), backed by
/// Redis so the limit is shared across all app instances.
async fn rate_limit(
    State(state): State<AppState>,
    headers: HeaderMap,
    req: Request,
    next: Next,
) -> Result<Response, StatusCode> {
    const LIMIT: i64 = 100; // requests per window
    const WINDOW: i64 = 60; // seconds

    let client = headers
        .get("x-forwarded-for")
        .and_then(|v| v.to_str().ok())
        .map(|s| s.split(',').next().unwrap_or(s).trim().to_string())
        .unwrap_or_else(|| "local".to_string());

    let key = format!("rl:{client}");
    let mut conn = state.cache.clone();
    let count: i64 = conn.incr(&key, 1).await.unwrap_or(0);
    if count == 1 {
        let _: Result<(), _> = conn.expire(&key, WINDOW).await;
    }
    if count > LIMIT {
        return Err(StatusCode::TOO_MANY_REQUESTS);
    }
    Ok(next.run(req).await)
}

Let’s read it top to bottom.

What an axum middleware is

A middleware in axum is just an async fn that sits between the network and your handlers. The two arguments that make it middleware are at the end:

• req: Request — the incoming request, not yet handed to a handler.
• next: Next — the rest of the pipeline. Calling next.run(req).await runs the actual handler and gives you back the Response.

So the function can do work before the handler (count the request), then either short-circuit with an error (return Err(...)) or pass the request through (Ok(next.run(req).await)). It can also pull from shared state with State(state): State<AppState> and read request headers with headers: HeaderMap — both are axum extractors, filled in for you from the request.

It’s wired in once, in main:

        .layer(middleware::from_fn_with_state(state.clone(), rate_limit))

from_fn_with_state turns a plain function into a layer and hands it a clone of AppState (so the middleware can reach Redis). .layer(...) applies it to every route — /shorten, /{code}, all of them go through the limiter first.

Identifying the client

    let client = headers
        .get("x-forwarded-for")
        .and_then(|v| v.to_str().ok())
        .map(|s| s.split(',').next().unwrap_or(s).trim().to_string())
        .unwrap_or_else(|| "local".to_string());

We key the limit on X-Forwarded-For — the header nginx stamped with the real client IP in the previous chapter. Without nginx, every request would arrive from nginx’s own IP and all clients would share one bucket; with it, each client gets their own. The .split(',').next() matters because X-Forwarded-For can be a comma-separated chain (client, proxy1, proxy2) — the first entry is the original client. If the header is missing (you’re hitting the app directly in dev), it falls back to "local". Notice the Rust Option chain: get returns Option, and_then/map transform it only if present, and unwrap_or_else supplies the default — no nulls, no panics.

The window logic

    let key = format!("rl:{client}");
    let mut conn = state.cache.clone();
    let count: i64 = conn.incr(&key, 1).await.unwrap_or(0);
    if count == 1 {
        let _: Result<(), _> = conn.expire(&key, WINDOW).await;
    }
    if count > LIMIT {
        return Err(StatusCode::TOO_MANY_REQUESTS);
    }
    Ok(next.run(req).await)

conn.incr(&key, 1) is the atomic Redis INCR: it creates the key at 1 if it didn’t exist, otherwise bumps it, and returns the new value. Only when count == 1 — the first request of a new window — do we set EXPIRE, starting the 60-second clock. Once the key expires, the next request recreates it at 1 and the window resets. Past LIMIT, we return 429; otherwise next.run(req).await lets the request through to its real handler. Because every replica runs this exact code against the same Redis key, the count is genuinely fleet-wide.

The catch: edge bursts

Fixed window is simple and cheap, but it has a known weakness. Because the window resets abruptly at a fixed instant, a client can send 100 requests at :59 and another 100 at :01 of the next window — 200 requests in ~2 seconds, even though no single window exceeded 100.

Good enough, not perfect

For Snip’s goal — blunt abuse protection — fixed window is the right call: one INCR, one conditional EXPIRE. When you need to smooth those edge bursts, you reach for a sliding window or token bucket. The full comparison is in Rate Limiting, and the design exercise is Design a Rate Limiter.

Try it

Bring up the stack (replicas optional — the point is the shared limit):

docker compose up --build --scale app=3

Fire more than 100 requests inside one minute and watch the status codes flip to 429:

for i in $(seq 1 130); do
  curl -s -o /dev/null -w "%{http_code}\n" localhost:8080/health
done | sort | uniq -c

You should see roughly 100 responses of 200 and the rest 429 — even though the requests were spread across three replicas, because they all incremented the same rl: key in Redis. Watch the key live:

docker compose exec redis redis-cli --scan --pattern 'rl:*'
docker compose exec redis redis-cli ttl '<key from the scan>'   # e.g. rl:172.18.0.1

(The key holds your IP as nginx saw it — on Docker’s internal network that’s a gateway address like 172.18.0.1, not 127.0.0.1 — so use whatever the scan printed.)

Wait 60 seconds for the window to expire and the next request succeeds again.

Check your understanding

1. You run 4 replicas, each with an in-memory counter capped at 100/min. What’s the actual limit a single client experiences, and why?
2. Why is EXPIRE set only when count == 1 instead of on every request?
3. What does keying on X-Forwarded-For accomplish, and what would the limiter do wrong if nginx didn’t set that header?
4. Describe the edge-burst weakness of fixed window with a concrete timeline. Which algorithms fix it?
5. In the middleware signature, what are the roles of req: Request and next: Next? What happens if the function returns Err(StatusCode::TOO_MANY_REQUESTS) instead of calling next.run?

Show answers

1. The actual limit is 400/min, not 100. Each replica keeps its own in-memory counter capped at 100, so four replicas allow 100 each — and the multiplier changes every time you scale. The limit only means “100 across the whole system” if every replica increments one shared counter, which is why it must live in Redis.
2. INCR creates the key at 1 on the first request of a new window, so count == 1 is exactly “this window just started.” Setting EXPIRE only then starts the 60-second clock once per window; setting it on every request would keep pushing the expiry forward (a sliding reset) and the window would never actually expire, so it could never reset.
3. Keying on X-Forwarded-For gives each real client its own bucket, using the original client IP that nginx stamped into the header. Without nginx setting it, every request would arrive from nginx’s own IP, so all clients would share one bucket — one busy client could exhaust the limit for everyone, and the limiter would be useless.
4. Because the window resets abruptly at a fixed instant, a client can send 100 requests at :59 and another 100 at :01 of the next window — 200 requests in ~2 seconds, though no single window exceeded 100. A sliding window or token bucket smooths those edge bursts (see Rate Limiting).
5. req: Request is the incoming request not yet handed to a handler; next: Next is the rest of the pipeline, so calling next.run(req).await runs the real handler and returns its Response. If the function returns Err(StatusCode::TOO_MANY_REQUESTS) instead, it short-circuits — next.run is never called, the handler never executes, and the client gets a 429. What the limiter buys: fleet-wide abuse protection. What it costs: shared state and one more Redis dependency.