System design fundamentals

It distributes requests across instances, which is what makes horizontal scaling and rolling deploys possible. The cost is that your services must become stateless, because a given user's next request may land anywhere. That single requirement pushes session state into a shared store or a signed token, and it is the most common thing candidates forget to say. Sticky sessions avoid it and reintroduce the problem they were avoiding, since a instance's failure now loses whoever was pinned to it.

A layer 4 balancer routes on IP and port without looking at the payload, so it is fast and protocol-agnostic and cannot make decisions based on the request. A layer 7 balancer parses HTTP, so it can route by path or header, terminate TLS, retry idempotent requests and apply per-route rate limits — at the cost of more work per request. Most application traffic wants layer 7; layer 4 suits raw throughput and non-HTTP protocols.

Read capacity and a degree of isolation, so heavy analytical queries do not compete with transactional ones. What it does not give you is a consistent view: replication is asynchronous in most configurations, so a replica lags the primary by anything from milliseconds to minutes under load. That lag is the source of the classic bug where a user posts a comment and does not see it, which is why the follow-up question is always what the user who just wrote sees.

Fluency here means knowing what each addition costs, not that it exists — and the interview asks for the second column.

block            buys you                     costs you
---------------  ---------------------------  ------------------------------------
load balancer    capacity, rolling deploys    statelessness becomes mandatory
read replica     read throughput              replication lag, stale reads
cache            latency, origin protection   invalidation, staleness, stampedes
queue            decoupling, load absorption  duplicates, reordering, silent backlog
append-only log  replay, many consumers       retention cost, partition key locked in
object store     cheap durable bytes          no queries, eventual listing
search index     relevance and text query     a second copy that drifts
shard            growth past one machine      cross-shard queries, rebalancing, hot keys

Read the right-hand column first, because that is where the follow-up questions come from. Every one of them is a question an interviewer can ask, and each has a specific answer: where does session state live now, what does the user who just wrote see, how does a cached item become wrong, how do you detect the consumer falling behind.

The habit worth building is to say the cost as you add the block. "I would put a cache in front of this, which means I now have to decide how it gets invalidated and what happens on a cold start" is a different answer from "I would add a cache", and it is the difference the round is measuring. Adding components without naming their cost reads as pattern matching rather than design, and every unnecessary component is one more thing you will be asked to defend.

Because it stores opaque bytes addressed by key, with no query capability, no transactions across objects and, historically, eventually consistent listing. It is the right home for images, backups, exports and logs — anything large, immutable and retrieved by a key you already have. The design consequence is that you need a database alongside it holding the metadata, because "find all files uploaded last Tuesday by this user" is a question the object store cannot answer.

Caching your responses at points of presence physically nearer the user, so a request for a static asset never reaches your origin and travels a much shorter distance. The latency win is mostly about distance and the origin protection is mostly about volume. The design questions it raises are cache keys — whether the response varies by header or cookie — and invalidation, because a purge across a global network is neither instant nor free.

When the caller needs the outcome to proceed. Queuing a payment authorisation does not remove the wait; it moves it somewhere the user cannot see, and now you own a correlation problem as well. A queue is right when the work merely must happen — sending a receipt, updating a search index, recalculating a recommendation — and wrong when the answer determines what the caller does next.

A queue delivers each message to one consumer and forgets it once acknowledged, so it models work to be done. A log is an ordered, retained sequence that many consumers read independently at their own positions, so it models facts that happened. The log's retention is what allows replay — adding a new consumer that processes history — and its partition key is chosen early and effectively permanently, because changing it reorders everything.