Revision · Data — Storage & Retrieval

Part 2 treated data as the rock beneath liquid code: storage decisions calcify into every query, migration, and consumer, so getting the trade-off right early is the cheapest it will ever be. Its throughline: distribution buys scale and survival and charges you in consistency, complexity, and operational pain — every page a different invoice for that one purchase.

What this part covered

• Data outlives code — volume, coupling, and correctness make storage sticky, so the reversibility test (how expensive is the undo?) tells you where to spend your deliberation budget.
• SQL vs NoSQL — the first fork isn’t “SQL good, NoSQL bad”; relational buys schema, joins, and ACID while NoSQL buys flexibility and scale, and the real axis is your access patterns and consistency needs.
• Indexing (B-tree vs LSM) — an index makes reads fast by making writes slower; B-trees update in place (read-optimized), LSM-trees append and compact (write-optimized), and you pay in read, write, and space amplification.
• Replication — copying data buys availability and read throughput and sends a consistency bill; single-leader, multi-leader, and leaderless topologies trade write simplicity against write availability, and async replication introduces lag anomalies.
• Partitioning & sharding — splitting one dataset across machines once it no longer fits, where the shard key is the most important decision; range vs hash trade range-scan ability against even spread, and hotspots ruin the party when one shard runs hot.
• Transactions & ACID — the guarantees that make a pile of writes one correct unit; isolation is a spectrum of anomalies traded for concurrency, and two-phase commit makes those guarantees expensive across machines.
• Data modeling — the craft that ties it together: normalization stores each fact once, denormalization pre-assembles the answer, and you model for how you’ll read the data, not just how it’s shaped.

The takeaway

Walk the six questions in dependency order — shape records, index them, survive a dead node, outgrow one machine, keep it correct, then model for reads — and each page is the same trade-off applied one layer deeper. When distribution forces you to confront what “the truth” even means across machines, the consistency and consensus material in the distributed-systems part waits; and when a single shard gets hammered, the hot-partition problem is at the end of the road.