Reverse Proxies & API Gateways

Every system needs a front door — one place the outside world talks to, behind which your real servers hide. That front door does jobs that every request needs but that you don’t want scattered across every service: terminating encryption, checking who’s calling, deciding which backend handles the request. The blocks that play this role are the reverse proxy and, its souped-up cousin, the API gateway. They’re the last block in this part because they tie the others together: a gateway is a place where routing, caching, load balancing, and security all converge.

Forward vs reverse proxy: who is hidden

Both are intermediaries, but they hide opposite ends of the conversation:

   FORWARD PROXY  — sits in front of CLIENTS, hides the client from the server
        [ clients ] ──► [forward proxy] ──► internet ──► server
        (corporate web filter, VPN egress; the server sees the proxy, not you)

   REVERSE PROXY  — sits in front of SERVERS, hides the servers from the client
        client ──► internet ──► [reverse proxy] ──► [ your servers ]
        (the client sees one address; it never learns which backend served it)

A forward proxy acts on behalf of clients — it’s what a company puts between its employees and the internet (filtering, caching, anonymizing). A reverse proxy acts on behalf of servers — the client thinks it’s talking to one machine, but the proxy is quietly distributing requests across a fleet behind it. The rest of this page is about the reverse direction, because that’s where system design lives: it’s the public face of your backend.

What a reverse proxy does

At its plainest, a reverse proxy receives client requests and forwards them to backend servers, returning their responses. Even this simple role earns its keep:

• TLS termination — handle HTTPS encryption/decryption once at the edge, so your backends speak plain HTTP internally and don’t each carry the crypto burden or the certificates.
• Load balancing — spread requests across backends (this is exactly the L7 load balancer from earlier; the roles overlap heavily).
• Caching — serve cached responses without bothering the backend at all.
• Compression & buffering — gzip responses, absorb slow clients so backends aren’t tied up.
• Hiding topology — clients see one address; you can rewire, add, or remove backends freely.

The API gateway: a reverse proxy with opinions

An API gateway is a reverse proxy that takes on application-level responsibilities — the cross-cutting concerns every API needs. Instead of each service implementing auth, rate limiting, and logging separately (and inconsistently), you implement them once, at the gateway, in front of everything. This is especially central in microservices, where the gateway is the single, coherent entry point to a sprawl of internal services.

   client ──► [ API GATEWAY ] ──► service A (orders)
                  │  • TLS termination          ──► service B (users)
                  │  • authentication & authz    ──► service C (catalog)
                  │  • rate limiting / quotas
                  │  • routing (path/host → service)
                  │  • request/response aggregation
                  │  • logging, metrics, tracing
                  └────────────────────────────────►

What it centralizes:

• Authentication & authorization — verify the token / API key once, at the door, so backends can trust that whoever reached them is already authenticated.
• Rate limiting & quotas — protect everything behind it from abuse and overload in one place. This pairs directly with Rate Limiting, which the gateway is the natural home for.
• Routing — map a public path or host to the right internal service (/orders/* → orders service), so the messy internal map never leaks to clients.
• Aggregation — combine several backend calls into one client response, so a mobile app makes one request instead of five (sometimes split out as a “backend-for-frontend”).
• Observability — uniform logging, metrics, and request tracing for every call, in one spot.

Centralizing cross-cutting concerns is the real win

The point of a gateway isn’t any single feature — it’s consistency. Auth, rate limits, and logging implemented ten times across ten services will be subtly wrong in ten different ways. Implemented once at the gateway, they’re uniform and your services get smaller, focusing only on their actual business logic. That’s the trade you’re buying.

The cost: a single point of failure (and a fat one)

Everything funnels through the gateway, which means the gateway is your availability and, often, your latency:

• Single point of failure. If it’s down, everything is down — no service is reachable. So a gateway must be run redundantly (multiple instances behind a load balancer / anycast IP), which reintroduces the very infrastructure it sits in front of.
• Performance bottleneck. Every request pays the gateway’s processing cost; a slow gateway slows the whole system.
• Operational coupling. It becomes a place where many teams’ concerns collide. Pile too much logic into it and it turns into a fragile, hard-to-change chokepoint — a distributed monolith’s front desk.

What does a gateway buy us, and what does it cost? It buys one consistent, secure, observable front door and dramatically simpler backend services. It costs you a critical component that must be made highly available, a potential bottleneck, and the discipline to keep it thin — routing and cross-cutting concerns, not business logic.

Don’t let the gateway swallow your business logic

It’s tempting to keep adding “just one more thing” to the gateway until it holds the rules that belong inside services. When the front door knows everything, every change touches it, every team is blocked on it, and its failure is total. Keep it to cross-cutting concerns (auth, limits, routing, TLS); push domain logic into the services behind it.

How it ties the part together

Notice how much of this part converges here: the gateway terminates TLS, does L7 load balancing, caches responses, enforces rate limits, and routes to the right service — which might then drop work onto a message queue and read from the right database. DNS got the user to the gateway’s address; the gateway gets the request to the right place. The toolkit is complete — and the gateway is where the pieces meet.

By the numbers — a single gateway caps your whole system

“Single point of failure” is not rhetoric; it’s the series-availability rule from the nines applied to the front door. Because every request passes through the gateway, its availability multiplies onto everything behind it:

   one gateway at 99.9%, perfect backends:
       system availability ≤ 99.9%   →  ~8.8 hours/year down, no matter how good the services are

   two gateways in parallel, each 99.9% (one is enough):
       1 − (0.001 × 0.001) = 0.999999   →  six nines

A lone gateway hands its own uptime to the entire system as a ceiling. Running two redundantly turns that ceiling into a floor — which is precisely why a gateway must never ship as a single box. What does centralizing the front door buy us? Consistency and simpler backends — at the cost of a component you are now obligated to make redundant.

The architect’s lens

A gateway is the front door for everything behind it — which is exactly why you weigh it with the five questions before centralizing on it:

• Why does it exist? Because every system needs a single front door, so the cross-cutting jobs every request needs — TLS termination, auth, routing — aren’t scattered (and subtly wrong) across every service.
• What problem does it solve? It centralizes those concerns once — auth, rate limiting, routing, aggregation, observability, TLS — so backends shrink to pure business logic and behave consistently instead of as ten subtly-different implementations.
• What are the trade-offs? You buy consistency and simpler services at the cost of a fat single point of failure: because every request funnels through it, its availability becomes the system’s ceiling (a lone 99.9% gateway caps everything at ~8.8 hours/year down, no matter how good the backends). It’s also a latency bottleneck and a place where many teams’ concerns collide.
• When should I avoid it? Avoid letting it swallow domain logic — keep it to cross-cutting concerns; a gateway that knows everything becomes a fragile chokepoint where every change touches it and every team is blocked on it. A tiny single-service app may not need the extra hop at all.
• What breaks if I remove it? Each service must re-implement auth, rate limiting, TLS, and routing itself (inconsistently), and clients must learn your internal service map. You also lose the single place to enforce rate limiting and uniform observability across every call.

Check your understanding

1. Forward vs reverse proxy: in each, which party is hidden, and who is the proxy acting on behalf of?
2. List three jobs a plain reverse proxy does, and explain why TLS termination at the edge is valuable.
3. What does an API gateway add beyond a reverse proxy? Why is implementing auth once at the gateway better than in each service?
4. Why is a gateway a single point of failure, and what must you do to keep that from being fatal?
5. Explain the failure mode of putting business logic into the gateway, and the rule that prevents it.

Show answers

1. A forward proxy sits in front of clients and hides the client from the server (it acts on behalf of clients — corporate filter, VPN egress); a reverse proxy sits in front of servers and hides the servers from the client (it acts on behalf of servers — the client sees one address and never learns which backend served it).
2. Any three of: TLS termination, load balancing, caching, compression/buffering, hiding topology. TLS termination at the edge is valuable because HTTPS encryption/decryption is handled once there, so backends speak plain HTTP internally and don’t each carry the crypto burden or manage certificates.
3. A gateway adds application-level cross-cutting concerns — auth/authz, rate limiting, routing, request/response aggregation, and observability. Doing auth once at the gateway beats per-service auth because ten separate implementations will be subtly wrong in ten different ways; centralized, it’s uniform, and backends shrink to just business logic.
4. Everything funnels through it, so if it’s down, everything is down — the gateway is your availability. To keep that from being fatal you run it redundantly (multiple instances behind a load balancer / anycast IP), which reintroduces the very infrastructure it sits in front of.
5. Piling domain logic into the gateway makes it a fragile chokepoint where every change touches it, every team is blocked on it, and its failure is total. The rule: keep the gateway to cross-cutting concerns (auth, limits, routing, TLS) and push domain logic into the services behind it — what does it buy us, and what does it cost? answered by keeping it thin.