The Blog
Notes on databases, distributed systems, and the engineering habits that keep software dependable in production.
## Where We Are In Part 1 we built a Write-Ahead Log. Every mutation — write or delete — is recorded to disk and fsynced before we acknowledge it to the caller. We have durabilit...
## Where We Are Post 1 built the WAL — every write is fsynced to disk before being acknowledged. Post 2 built the memtable — an in-memory skip list that keeps keys sorted and ser...
## Implementing a Write-Ahead Log in Go A Write-Ahead Log is the mechanism behind the D in ACID. Before any mutation reaches your data files, a record describing that mutation is...
## Introduction You have three backend servers. Each one can handle 1,000 requests per second. Together they can handle 3,000 — but only if requests are spread evenly across all ...
## Where We Left Off In Part 1 we built a reverse proxy that distributes requests across three backends using round robin. It works — but it is blind. Kill one of the backends an...
## Introduction Your API receives a request. You need to send a confirmation email, resize an image, call a third-party webhook, and update an analytics counter. You could do al...
## Introduction Your application writes a row. PostgreSQL says "committed." The server loses power one millisecond later. When it comes back, is your row there? The answer is y...
## Introduction Your application has one user. Concurrency is not a problem. That user reads data, writes data, nothing conflicts with anything. Then you have ten thousand users...
## Introduction Your single database server handled your first 100,000 users just fine. Then you hit a million. Queries slowed down. Disk filled up. You threw more RAM at it. It ...
## Why B+ Trees Exist Your database has 50 million rows. You query by user ID. Without an index, the database reads every row until it finds yours. That's a full table scan. It g...
## Why Single-Database Transactions Don't Scale A transaction in a single database is simple. You start a transaction, make changes, commit. Either everything commits or nothing ...
## Building a Production Distributed Messaging System: The Architecture, The Problems, and The Plan You open Discord. You type a message and hit enter. Half a second later, every...
## Why Replication Exists Your database will crash. Not maybe. It will. Hard drives fail, memory corrupts, someone runs the wrong command, power goes out. When that happens, you ...
## Introduction Most developers understand SQL. Fewer understand what happens after the query planner finishes — how rows are physically laid out on disk, why certain writes cost...
## Introduction At some point you ship something that works perfectly in staging. The code is clean. The tests pass. You deploy it and feel good about it. Six months later you'r...
## Why This Matters in Production Every backend engineer has a story about a production incident that made ACID properties real. Maybe it was a race condition in a payment system...
## The Real Cost - What happens when you don't have idempotency Every backend engineer has experienced this: a user clicks a button, the request times out, they panic and click i...
## Introduction Your database has 50 million users. Someone searches by email. Without an index, the database reads every single row until it finds a match. That's a full table s...