Where to Go Next

You built a real service. Not a toy in a single file, but Snip — a URL shortener that persists, caches, scales across replicas, rate-limits, offloads work to a queue, reports on itself, degrades gracefully, and boots from one docker compose up. Every step was a decision, and every decision was judged by the same question this book keeps asking: what does this buy us, and what does it cost?

This last chapter does two things. First it replays the journey as a ledger — not “we added X” but “X bought us this, and charged us that.” Then it points at the road ahead: the moves you’d make when Snip’s traffic outgrows what we built, each linked to the concept page that explains the real, production-grade version.

Part 1 — What each step bought us

Read this top to bottom and you can watch the trade-offs accumulate. Nothing here is free; the skill is knowing which bill you’re willing to pay.

STEP                        WHAT IT BOUGHT                         WHAT IT COST
-------------------------------------------------------------------------------------------
MVP (axum, in-memory)       a working service, fast feedback       data vanishes on restart
Postgres                    durability, real persistence           a network hop on every read
Redis cache (cache-aside)   fast redirects, less DB load           staleness; cache invalidation
Load testing                a baseline you can beat                discipline — measure, don't claim
Replicas + nginx            throughput, no single point            you must be stateless
Rate limiting (Redis)       protection from abuse/runaway          shared state; one more dependency
Queue + worker              redirects stay fast under write load   eventual click counts, not exact
Observability               you can see inside the running fleet   instrumentation everywhere
Resilience                  it bends instead of breaking           timeouts/retries/fallback code
Docker Compose              one command runs the whole stack       config to maintain; not yet prod

:::note The thread, made literal That right-hand column is the book. Every concept in Parts 0–10 — caching, replication, queues, load balancing — is a way of moving cost from one place to another: from latency to staleness, from simplicity to throughput, from exactness to availability. You don’t escape the cost. You choose where to put it, on purpose. :::

Part 2 — Where to go next

Snip today comfortably handles a lot. But traffic has a way of finding the next bottleneck, and each one has a well-trodden answer. Here’s the map — the symptom you’ll see, and the concept page that explains the grown-up fix.

The database becomes the bottleneck

You scaled the app to many replicas, but they all talk to one Postgres. Reads pile up first (Snip is read-heavy), then writes. Two classic moves, in order:

• Read replicas — copy the database, send reads to the copies, writes to the primary. This is the read/write split, and it pairs naturally with separating your read and write paths. See Read Replicas & CQRS.
• Sharding — when even the write primary is saturated, split the data itself across many databases by key. See Partitioning & Sharding, and to add or remove shards without reshuffling everything, Consistent Hashing.

A few short codes go viral

One link in a million gets a thousand times the traffic of the rest. That single key becomes a hot partition — sharding doesn’t save you, because all the heat lands on one shard. The fix is a different shape of problem (replicate-the-hot-key, per-key caching, request coalescing). This is the celebrity problem, and it deserves its own treatment: Hot Partitions & the Celebrity Problem.

Global users, far from your server

A redirect that’s 5 ms in your datacenter is 250 ms for someone on the other side of the planet — physics, not your code. Two moves:

• CDN / edge — serve redirects from points of presence close to the user. A URL shortener is an almost-perfect CDN workload: tiny, cacheable, read-heavy. See Content Delivery Networks.
• Multi-region & disaster recovery — run Snip in more than one region for latency and survival. Once you do, you must answer “how much data can we lose, and how fast must we recover?” — RPO and RTO. See Disaster Recovery (RPO/RTO).

Better rate limiting

Our limiter was a deliberately simple fixed counter. Production limiters use sliding windows or token buckets so a burst at a window boundary can’t sneak through double the limit, and so you can allow short bursts while capping the sustained rate. See Rate Limiting.

Safer rollouts

docker compose up redeploys by stopping and starting — fine for one machine, terrifying for production. Real deploys ship to a fraction of traffic first and watch the metrics before going wide: blue/green, canary, rolling. The observability you built is exactly what makes these safe, because you can see a bad release before everyone does. See Deployment Strategies.

Compare your build to the textbook design

You built Snip from the ground up, decision by decision. Now read the canonical interview-style write-up of the same system and diff it against what you made: where did you agree, what did you skip, what would you defend? See Design a URL Shortener.

You’ve done both halves

Here’s what’s quietly remarkable about where you are. Most people who study system design have read about caches and queues and replicas. Most people who can run a service have never been forced to name the trade-off behind each piece. You’ve now done both:

• Parts 0–10 taught you the concepts and the trade-offs — the vocabulary of building systems.
• Part 11 made you build and scale a real one — and measure every win, so no improvement was a claim, it was a number.

So the book’s recurring question isn’t abstract for you anymore. What does this buy us, and what does it cost? You can answer it with your own running system in front of you: you’ve watched a cache cut tail latency and seen the staleness it introduced; you’ve added replicas and felt the constraint that they be stateless; you’ve moved click-counting off the hot path and accepted that the count is now eventual. That’s not knowledge from a slide. That’s earned.

The next system you design — at work, in an interview, for fun — you’ll reach for these moves not because a guide told you to, but because you’ve felt the bottleneck they fix and you know the bill they charge. Go build it.

:::tip One last reframe Scaling is never “make it bigger.” It’s “find the current bottleneck, choose where to move the cost, measure, repeat.” Snip will never be done — and that’s the point. There’s always a next bottleneck, and now you know how to meet it. :::

Try it

Pick one extension from Part 2 and actually implement it on your Snip. Suggested starting points, easiest first:

• Read replica (simulated). Add a second Postgres to docker-compose.yml, point a read-only pool at it, and route redirect’s DB lookups there while shorten writes to the primary. (Skip real streaming replication; just prove the read/write split in code.) Re-run the load test — did read throughput climb?
• Sliding-window rate limiter. Replace the fixed counter with a sliding-window or token-bucket algorithm in Redis. Write a test that fires a burst across a window boundary and confirm the old limiter would have let double through and yours doesn’t.
• Edge-cache headers. Add Cache-Control headers to redirects so a CDN (or even a browser) could cache them, and reason about how cache invalidation would work when a link is deleted.

Whatever you choose: state the trade-off before you build it, then measure whether the cost landed where you predicted.

Check your understanding

1. For each step in the Part 1 ledger, cover the “what it cost” column and try to reproduce it from memory. Which costs are latency, which are correctness/staleness, and which are operational complexity?
2. Read replicas help with read load but not write load; sharding helps with both. At what point would you reach for sharding rather than just more read replicas — and how would you choose a shard key for Snip specifically?
3. Why doesn’t sharding solve the hot-key (celebrity) problem, and what would? Sketch one approach.
4. A URL shortener is described as an “almost-perfect CDN workload.” Why? What single property of a deleted link makes edge caching tricky, and how would you handle it?
5. Open-ended: you’re given Snip and told “we’re going global next quarter.” Which two moves from Part 2 would you do first, in what order, and what would you measure to prove each one paid off?

Show answers

1. The costs sort into three buckets. Latency: Postgres adds a network hop on every read. Correctness/staleness: the cache introduces staleness and cache-invalidation work, and the queue makes click counts eventual rather than exact. Operational complexity: replicas force you to be stateless, the Redis rate limiter and cache add shared state and dependencies, observability means instrumentation everywhere, resilience means timeout/retry/fallback code, and Compose adds config to maintain. The ledger’s whole point: every step moves cost from one place to another — you choose where, on purpose.
2. You reach for sharding once even the write primary is saturated — read replicas only offload reads, so they don’t help when writes are the bottleneck. For Snip, the natural shard key is the code (the primary key you look up by): it’s high-cardinality and evenly distributed, so hashing on it spreads both reads and writes across shards. See Partitioning & Sharding.
3. Sharding distributes different keys across shards, but a viral link is one key — all its heat lands on a single shard, so splitting the data doesn’t help. What does: replicate the hot key across nodes, cache it per-key (it’s tiny and immutable, so a CDN or in-memory cache absorbs the load), or coalesce concurrent requests for it. See Hot Partitions & the Celebrity Problem.
4. Because a redirect is tiny, cacheable, and read-heavy — exactly what a CDN serves best, letting points of presence near the user answer without crossing the planet. The tricky property is that a deleted link is still cached at the edge: invalidation. You handle it by setting short/explicit Cache-Control TTLs and issuing a purge/invalidation to the CDN when a link is deleted, so the edge stops serving the dead code.
5. Open-ended — e.g. do CDN / edge caching first, then multi-region. CDN first because it’s the cheapest, highest-leverage win for a tiny read-heavy redirect and needs no new write topology; prove it by measuring p95/p99 redirect latency from far-away regions (it should drop sharply) and the origin’s request rate (it should fall as the edge absorbs reads). Then multi-region for survival and write-latency, proven by failover drills against your RPO/RTO targets and write latency measured per region. State each trade-off before building and check the cost landed where you predicted.