Statelessness & Sessions

Horizontal scaling rests on a single, almost boring-sounding property: any node can serve any request. If that’s true, a load balancer can send the next request to whichever server is free, and adding capacity is as simple as adding a box. The moment it isn’t true — the moment a request can only be handled by the one server that remembers something about this user — your fleet stops being interchangeable and your scaling story falls apart. This page is about that property, why it matters, and where the inconvenient state actually goes.

Why local state breaks horizontal scaling

Imagine a server that, when you log in, stores your session in its own RAM. Now there are three servers behind a load balancer:

   request 1 (login)  ──► web #1   [stores session in #1's memory]
   request 2 (profile)──► web #2   [no session here — who are you?]  ✗

The user is logged in on #1 but lands on #2, which has never heard of them. The fix can’t be “always send this user back to #1,” because then #1 is a bottleneck and a SPOF — if it dies, the session dies with it, and you can’t rebalance load freely. Local, in-memory state pins requests to machines, and pinning is the enemy of elastic scaling.

A stateless service holds no per-client memory between requests. Everything it needs to handle a request either comes in the request or is fetched from a shared store. Two stateless servers are truly identical, so any of them can handle any request — and you can add, remove, or replace them freely. That interchangeability is the entire point.

Stateless doesn’t mean “no state anywhere”

The state still exists — it just doesn’t live inside the app process. It moves to a shared tier (a cache, a database) that all nodes can reach. “Stateless service” means the service instances are disposable, not that the system has amnesia.

Where does session state go?

Once you evict session state from the app server’s memory, it has to live somewhere. There are three common answers, and they sit on a spectrum from “least change” to “most pure.”

Option 1: Sticky sessions (session affinity)

Tell the load balancer: “once a user lands on a server, keep sending them there.” State stays in the app server’s memory; the LB just routes consistently (via a cookie or source IP).

What does it buy us, and what does it cost? It buys almost no code change — your existing in-memory sessions just work. It costs you most of the benefit of being stateless:

• If that server dies, every session on it is lost (users logged out).
• Load can’t rebalance freely — a “hot” server stays hot until its users leave.
• Deploys are disruptive: draining a node logs its users out or forces re-routing.

Sticky sessions are a pragmatic stopgap, not a destination. They quietly reintroduce the pinning you were trying to escape.

Option 2: Central session store (e.g. Redis)

Move sessions out to a shared, fast data store — typically Redis — that every app server can read and write. Servers become stateless; the session lives in one place all of them share.

                 ┌─ web #1 ─┐
   clients ─►LB ─┼─ web #2 ─┤ ──► Redis (sessions)
                 └─ web #3 ─┘

What does it buy us, and what does it cost? It buys true statelessness with server-side control — you can revoke a session instantly by deleting the key, store rich data, and survive any app node dying. It costs you:

• A network hop on every request (fast, but not free), and a dependency you must keep available.
• The session store itself becomes a thing to scale and make highly available — you’ve concentrated the state, so now it matters.

This is the most common production answer for stateful login systems precisely because instant revocation and server-side control are worth the hop.

Option 3: Stateless tokens (JWTs)

Put the session data in the token the client holds — a signed JWT. The server stores nothing; it just verifies the signature on each request and trusts the claims inside. The state lives on the client.

What does it buy us, and what does it cost? It buys the purest statelessness — no session store at all, no extra hop, infinitely horizontally scalable verification. It costs you the thing the central store gave you: revocation. A signed token is valid until it expires, so logging someone out “now” is hard — you’re stuck with short expiries plus refresh tokens, or a denylist that quietly reintroduces server-side state. Tokens also can’t grow without bloating every request.

The revocation trade is the whole story

The central-store-vs-JWT decision is fundamentally revocation vs simplicity. Server-side sessions can be killed instantly but cost a lookup; JWTs need no lookup but can’t be un-issued. Many real systems use both: short-lived JWTs for fast verification plus a server-side refresh-token store for control. See OAuth & JWT for the mechanics.

A quick comparison

Approach	Server stateless?	Revoke instantly?	Extra hop?	Failure of one node
Sticky sessions	No	Yes (kill server)	No	Sessions lost
Central store	Yes	Yes	Yes	Sessions survive
JWT	Yes	Hard	No	Sessions survive

The principle underneath

Notice the pattern: making the service stateless doesn’t delete the state — it relocates it to a shared tier, and then that tier becomes the thing you scale and protect. This is the recurring move in all of scaling: you don’t remove a constraint, you move it somewhere you’ve decided you can manage. Stateless app servers are the foundation that lets the web tier scale out (Vertical vs Horizontal); the price is that session state, cache state, and data all get pushed into shared infrastructure you now have to run well.

By the numbers — the JWT you re-send on every request

The token’s purity has a weight. A signed JWT carries its claims as base64url (which inflates the bytes by ~33%) plus a signature — a minimal one runs ~300 bytes, and an RS256 token with a few roles and scopes easily passes ~800 bytes to 1 KB. The client sends it on every request.

JWT path:    ~0.8 KB token × 10,000 req/s ≈ 8 MB/s of token re-sent upstream, continuously
             verify = a local signature check, no network hop
cookie path: ~40-byte session id + one Redis GET (~sub-millisecond on a LAN) per request

That’s the revocation-vs-simplicity trade as raw bytes: the JWT skips the lookup but re-ships its whole payload on every call and grows with every claim you add, while the opaque session id stays tiny and pays a small, fast lookup instead. Neither is free — you’re choosing which cost (bandwidth-and-bloat, or a hop-and-a-store) you’d rather pay.

The architect’s lens

Statelessness is the discipline horizontal scaling is built on — run it through the five questions:

• Why does it exist? Because horizontal scaling rests on one property — any node can serve any request. The moment a request can only be handled by the one server that remembers a user, the fleet stops being interchangeable. Statelessness removes per-client memory from the app process.
• What problem does it solve? It makes service instances disposable and identical, so a load balancer can route to any free server and you can add, remove, or replace boxes freely without losing anyone’s session.
• What are the trade-offs? Stateless doesn’t delete state — it relocates it to a shared tier, and now that tier is the thing you scale and protect. The store-vs-JWT choice is revocation vs simplicity: a central session store revokes instantly but costs a network hop per request; a signed JWT needs no hop or store but can’t be un-issued and re-ships its ~0.8 KB payload on every call.
• When should I avoid an option? Treat sticky sessions as a stopgap, not a destination — they quietly reintroduce the pinning you were escaping. Avoid pure JWTs where instant logout matters; avoid a central store where the extra hop and dependency are unacceptable.
• What breaks if I keep state local? In-memory sessions pin each user to one machine: that server becomes a SPOF, its death logs everyone on it out, load can’t rebalance, and a deploy that drains a node disrupts users — the whole horizontal-scaling story collapses.

Check your understanding

1. Precisely why does in-memory session state prevent a load balancer from treating servers as interchangeable?
2. Sticky sessions require almost no code change. What specific benefit of statelessness do they give up in exchange?
3. What is the single biggest trade-off between a central session store and stateless JWTs?
4. “Stateless doesn’t mean no state anywhere.” Explain where the state actually goes and why the service is still called stateless.
5. Why does relocating session state make the session store a new thing you must scale and protect?

Show answers

1. Because in-memory state means only the one server that remembers a user can handle their next request, so the load balancer can no longer send any request to whichever server is free. Local state pins requests to machines, and pinning destroys the interchangeability horizontal scaling depends on.
2. Sticky sessions keep state in the app server’s memory and just route the user back to the same box — so they give up the disposability/interchangeability of stateless nodes: if that server dies its sessions are lost, load can’t rebalance freely, and deploys that drain a node log users out. They quietly reintroduce the pinning you were escaping.
3. Revocation vs simplicity. A central session store can be killed instantly (delete the key) but costs a network lookup on every request; a stateless JWT needs no lookup and no store but can’t be un-issued — a signed token stays valid until it expires, so logging someone out “now” is hard.
4. The state still exists — it just moves out of the app process to a shared tier (a cache like Redis, or a database) that all nodes can reach. The service is still “stateless” because the service instances are disposable and interchangeable, not because the system has amnesia.
5. Because you’ve concentrated the state into one shared tier that every app server now depends on — so that store becomes a thing you must keep available, make highly available, and scale. The recurring move in scaling is that you don’t remove a constraint, you relocate it somewhere you’ve decided you can manage.