Edge & Serverless

For most of this book “the server” was a long-lived process you ran in a datacenter, sitting close to its database. Two newer execution models invert pieces of that. Serverless functions ditch the long-lived process: code runs only when invoked, scales to zero when idle, and you’re billed per invocation. Edge compute ditches the single datacenter: code runs at hundreds of CDN points of presence, a few milliseconds from the user, instead of one region far away. Both are extreme cases of ideas you already know — statelessness and CDNs — pushed to their logical end. This page asks the recurring question of each: what does running at the edge or on-demand buy us, and what does it cost? It’s a snapshot as of 2025.

Two shifts, often conflated

SERVERLESS  no server to manage; runs on invocation; scales to zero;
            billed per request + per ms of compute  (e.g. function-as-a-service)

EDGE        runs at many POPs near the user instead of one central region;
            single-digit-ms network latency to the client

They’re orthogonal — you can have central serverless, or a long-running edge process — but in practice they’re sold together, because the same constraints (no persistent local state, fast startup) make code portable to both. The unifying theme: the runtime is ephemeral and you don’t pick where it lives.

The stateless constraint (now mandatory, not advisable)

Earlier, statelessness was good practice — it let you scale horizontally behind a load balancer. At the edge and in serverless it’s physics. A function instance may be created for one request and destroyed after; the next request may land at a different POP on a different continent. There is no reliable local disk, no in-process memory that survives, no sticky session. Every bit of state must live somewhere external — a database, a cache, an object store, a durable-state primitive — and the function must fetch it each time.

The constraint is the feature

Mandatory statelessness is what lets the platform spin up a thousand copies of your function in seconds and scale to zero when traffic stops. You give up local state and get effortless elasticity in return. That’s the trade in one sentence: you can’t keep anything between requests, so the platform is free to run you anywhere, any number of times, including zero.

Cold starts: the tax on scaling to zero

If a function scales to zero, the first request after idle has to create an execution environment before your code runs — the cold start. Subsequent requests hit a warm instance and skip it. The size of that tax depends entirely on what has to be created, which is the central architectural divide in this space.

Under the hood — isolates vs microVMs vs containers

                 startup       isolation         memory/instance   typical use
V8 isolate       ~sub-ms–ms    shared process,   tiny (~MBs)       edge workers
                               per-isolate heap                    (JS/Wasm only)
microVM          ~100+ ms      hardware-level VM  ~tens of MBs      serverless funcs
(e.g. Firecracker)             boundary                            (any language)
container        ~hundreds ms  namespace/cgroup   ~hundreds MBs    general workloads
                 to seconds    isolation

• V8 isolates (the model behind CDN edge workers) don’t boot a VM at all — they spin up a new JavaScript/Wasm sandbox inside an already-running process, so cold starts are near-zero. The cost: you’re confined to languages that compile to that sandbox, with tight memory and CPU limits and no arbitrary native code.
• microVMs (such as Firecracker, used under major serverless platforms) boot a stripped-down lightweight virtual machine in ~a hundred-plus milliseconds. The cost is a heavier start than an isolate; the buy is a real machine boundary that can run any language and arbitrary code safely.
• Containers are heavier still and slower to start, but run almost anything with familiar tooling.

This is a clean instance of what does it buy, what does it cost. The lighter the sandbox, the faster it starts and the less it can do; the heavier the sandbox, the slower the cold start and the more general the workload. There is no free lunch — only a dial between startup speed and capability.

By the numbers — when cold starts actually bite

Cold-start frequency matters more than cold-start cost. Suppose a function takes 300 ms to cold start and 20 ms warm, and you serve 1,000 requests in a burst on a freshly-scaled fleet:

if 50 instances cold-start, then serve ~20 reqs each warm:
   cold-start cost paid:  50 × 300 ms = 15 s of added latency, spread across 50 unlucky requests
   the other 950 requests:  20 ms each (warm)
   → ~5% of requests see the tax; p50 is fine, p99 spikes

So cold starts are a tail-latency problem, not an average one — they punish the unlucky first request after a scale-up or idle period. Mitigations (provisioned/“always-warm” instances, keeping functions small, choosing a lighter sandbox) all trade cost or capability for fewer cold starts. Keeping a fleet warm defeats scale-to-zero’s savings; that is the trade-off.

When the edge wins — and when data-locality bites

The edge’s whole pitch is proximity to the user: terminate TLS, serve cached content, run auth/redirects/personalization, and rewrite requests a few milliseconds from the browser instead of a round-trip across an ocean. For work that needs little or no central data, that’s a huge latency win.

But your database usually lives in one region. And here the edge inverts on you: a function running in Sydney that has to query a database in Virginia pays the trans-Pacific round-trip on every data access — and if a single request makes several sequential queries, those round-trips stack. You moved the compute next to the user and away from the data it needs. The edge made the user-to-compute hop tiny and the compute-to-data hop enormous.

  central function:   user ──(150 ms)──► region ──(1 ms)──► DB     one slow hop
  naïve edge function: user ──(2 ms)──► edge POP ──(150 ms)──► DB   the hop just moved
                                              × N sequential queries = N × 150 ms  ← worse!

The edge is for the data-light hop

Edge compute shines when the request can be answered at the edge: cache hits, signed-URL checks, header rewrites, geolocation routing, A/B assignment, light personalization, and reads served from edge-replicated or globally-distributed data. It hurts when a request needs chatty access to a single-region primary database — you’ve traded one moderate hop for many long ones. The rule of thumb in 2025: put data-light, latency-sensitive logic at the edge; keep data-heavy, transactional logic next to the data. Some teams bridge the gap with globally-replicated databases or edge caches, which buy locality back — at the cost of replication complexity and weaker consistency.

Operational shape: the good and the sharp edges

Serverless and edge change your operational posture. You get elasticity for free (scale-to-zero, no capacity planning, pay only for what you use — great for spiky or low-baseline traffic) and you shed a class of ops work. In return you inherit new sharp edges: cold-start tail latency; per-invocation billing that can surprise you under a traffic spike or a retry storm; hard limits on execution time, memory, and payload size; the difficulty of holding a connection pool open from ephemeral instances to a connection-limited database; and harder local debugging of code that only really runs on the platform. None of these are dealbreakers — they’re the cost column you weigh against the elasticity in the buy column.

Key trade-offs

• Scale-to-zero vs always-warm: pay-nothing-when-idle and effortless burst scaling vs cold-start tail latency on the first request after idle.
• Light sandbox (isolate) vs heavy sandbox (microVM/container): near-instant starts and tight limits vs slower starts and the freedom to run any language and native code.
• Edge proximity vs data locality: milliseconds to the user vs potentially many long hops to a single-region database — great for data-light work, painful for data-heavy work.
• Per-request billing vs provisioned capacity: cheap for spiky/low-baseline load vs predictable cost (and no cold starts) for steady high load.

It actually happened — when the edge has one config, its blast radius is the internet

On 8 June 2021, a single customer configuration change triggered a latent software bug at the edge-platform/CDN provider Fastly, and a large fraction of the web went dark simultaneously — major news sites, government portals, and large retailers returned errors within minutes of each other. The outage was largely resolved within about an hour, but the lesson is structural: pushing compute and config to a shared global edge means a shared global blast radius. Centralizing logic at the edge buys you proximity and a single control point — and that same single control point is exactly what can take everyone down at once. It’s the trade-off of this entire page written across the front pages: the edge’s reach is also the edge’s risk, which is why teams pair it with multi-provider strategies, staged config rollouts, and graceful degradation.

The architect’s lens

Edge and serverless push statelessness and CDNs to their logical end — adopt them through the five questions:

• Why do they exist? Because the long-lived, single-region process wastes money when idle and sits far from the user. Serverless ditches the persistent process (scale to zero, bill per invocation); edge ditches the single datacenter (run at hundreds of POPs, single-digit-ms from the browser).
• What problem do they solve? Effortless elasticity and proximity. Mandatory statelessness is what lets the platform spin up a thousand copies in seconds and scale to zero; the edge collapses a trans-ocean round trip to a few milliseconds for data-light work (cache hits, auth, header rewrites, geo-routing).
• What are the trade-offs? The lighter the sandbox the faster it starts and the less it can do — V8 isolates start in sub-ms but run only JS/Wasm; microVMs (Firecracker) take ~100+ ms but run any language. Scale-to-zero buys idle savings but pays a cold-start tail-latency tax on the first request after idle (a p99 problem, not p50).
• When should I avoid it? When a request needs chatty access to a single-region primary database — the edge inverts on you, trading one moderate user-to-region hop for N long compute-to-data hops. Keep data-heavy, transactional logic next to the data; and always-warm provisioning defeats scale-to-zero’s whole point.
• What breaks if I remove it (or over-centralize)? You lose elasticity and proximity — back to capacity planning and a far-away region. But the inverse risk is real too: pushing logic to a shared global edge means a shared global blast radius — Fastly 2021, where one config change took a slice of the web down at once.

Check your understanding

1. Why is statelessness mandatory at the edge and in serverless, where elsewhere in the book it was merely good practice? What do you give up and get in return?
2. What is a cold start, and why is it better described as a tail-latency problem than an average-latency problem?
3. Contrast V8 isolates, microVMs, and containers on startup time and capability. State the buy/cost for choosing a lighter sandbox.
4. Explain how moving a function to the edge can make latency worse. When does the edge win, and when does data-locality bite?
5. Using the Fastly 2021 outage, explain the blast-radius trade-off of centralizing logic at a shared global edge.

Show answers

1. Because instances are ephemeral and placed anywhere: one created per request, possibly destroyed after, and the next request may land at a different POP — there’s no reliable local disk, surviving memory, or sticky session. So all state must live externally and be fetched each time. You give up local state; in return the platform can spin up a thousand copies in seconds and scale to zero, giving effortless elasticity. The constraint is the feature.
2. A cold start is the time to create a fresh execution environment for the first request after idle (or after a scale-up), before your code runs; warm instances skip it. It’s a tail-latency problem because only the unlucky first requests pay it — p50 stays fine while p99 spikes — so it punishes the first request after a scale event rather than raising the average.
3. V8 isolates start in roughly sub-millisecond-to-millisecond time (new sandbox inside a running process) but only run JS/Wasm with tight limits; microVMs (e.g. Firecracker) start in ~100+ ms with a real VM boundary that runs any language; containers start in hundreds of ms to seconds and run almost anything. Choosing a lighter sandbox buys near-instant cold starts and density and costs capability (restricted languages, tighter memory/CPU, no arbitrary native code).
4. The database typically lives in one region. An edge function near the user but far from that database pays a long round-trip on every data access, and several sequential queries stack those long hops — so you replaced one moderate user-to-region hop with many long compute-to-data hops. The edge wins for data-light, latency-sensitive work (cache hits, auth/redirects, header rewrites, geo-routing, light personalization) and bites when a request needs chatty access to a single-region primary.
5. On 8 June 2021 a single config change hit a latent bug at Fastly and took down a large slice of the web at once. Centralizing logic at a shared global edge buys proximity and one control point, but that same single point is a shared blast radius: one bad config or bug can fail everyone simultaneously — the reach of the edge is also its risk, which is why teams add multi-provider strategies, staged rollouts, and graceful degradation.