System Design

What this area actually covers

System design is the work of turning a vague requirement into a structure of components with defined responsibilities, and then justifying the structure against the properties it was supposed to deliver — latency, availability, consistency, cost, and the ability of a team to keep operating it at three in the morning. It covers how you split a workload across machines, where state lives and how many copies of it exist, how components talk (synchronously over an API, asynchronously over a queue or log), what happens when one of them is unreachable, and how you find out that something is wrong before your users tell you.

The heart of it is that these properties trade against each other and cannot all be maximised. Replicating data across regions buys you availability and costs you consistency and write latency. Adding a cache buys read throughput and costs you a new class of bug where the cache and the store disagree. Splitting a monolith into services buys independent deployment and costs you distributed failure modes, a network hop in every call path, and the loss of a database transaction as your correctness mechanism. There is no design that avoids paying; the discipline is choosing which currency to pay in and being able to say why.

Two adjacent things get wrongly folded in. Object-oriented and code-level design — class boundaries, patterns, dependency direction — is a real skill, usually assessed in a separate "low-level design" round, and it answers a different question. Infrastructure operation — writing Terraform, tuning a Kubernetes cluster, configuring a mesh — is valuable but is execution of a design rather than the design itself; plenty of strong operators design poorly and some good designers have never written a Helm chart.

A third confusion is worth pre-empting because it wastes the most preparation time. System design is not the study of large systems. It is the study of constrained systems, and the constraint is what generates the design. A service handling a hundred requests a second with a hard requirement that no payment is ever double-counted is a richer design problem than one handling a hundred thousand requests a second of idempotent reads. Candidates who prepare by memorising the architectures of famous products absorb the answers to problems whose constraints they never learned, which is why they come apart the moment the constraints are changed.

The twelve areas underneath

This section is split into twelve subsections. The first eight are the building blocks and the reasoning around them, the next three are areas that behave differently enough to be examined on their own terms, and the last is the place where everything is assembled into whole designs. If you are preparing for a specific loop, work the fundamentals subsection first and the case studies last, because a case study you attempt before you can estimate is just reading.

Design Fundamentals covers requirements gathering, capacity estimation, and how to structure the first ten minutes of a design round. It exists as a separate subsection because it is the part of the round with the highest marginal return and the least practice behind it: most candidates have thought about databases and almost none have rehearsed extracting requirements aloud. You will find question checklists, the arithmetic conventions for estimation, and how to make scoping visible rather than accidental.

Scalability & Load Balancing covers horizontal against vertical scaling, load-balancer strategies, and what it takes for a service to be genuinely stateless. It is separate because scaling is where a design either survives or reveals a shared mutable assumption, and the mechanics — sticky sessions, health checking, connection draining, the difference between layer 4 and layer 7 balancing — have to be concrete before the rest is useful.

Caching covers placement, eviction, invalidation, and the two failure modes that define the topic: a stampede when a hot key expires, and coherence when two copies disagree. It earns its own subsection because caching is the single most commonly proposed and least commonly reasoned-about component in design interviews. Inside, each pattern is paired with what goes stale and for how long.

Databases & Storage covers choosing a store, sharding and partitioning, replication topologies, and storage-engine trade-offs such as B-tree against log-structured merge tree. It is here rather than in the databases section because the question is different: not how to write a good query but what a store's shape does to your architecture, and what a shard key commits you to for the life of the system.

Consistency & Availability covers CAP and PACELC as practical statements rather than slogans, the consistency models between strong and eventual, consensus, and exactly what you surrender under partition. It is separated because it is the most misquoted material in the field, and because the useful version of it is a vocabulary for saying precisely which anomaly your design permits.

Messaging & Streaming covers queues against logs, delivery guarantees, ordering, backpressure, and the recurring "exactly once" claim. It exists apart from the API subsection because asynchronous communication changes the failure model rather than just the syntax: you gain a buffer and inherit duplicates, reordering and a backlog that can grow without producing a single error.

API & Contract Design covers resource modelling, pagination, idempotency, versioning and backward compatibility. It sits in system design rather than only in backend engineering because at design scale the contract is what allows two teams to change independently, and a design that requires synchronised releases across services has quietly failed at the thing it was drawn to achieve.

Rate Limiting & Resilience covers throttling algorithms such as token bucket and sliding window, circuit breakers, retries with jitter, bulkheads, and how a system degrades gracefully. Its own subsection because these are the mechanisms that decide whether load in excess of capacity produces a slow system or a dead one, and because the algorithms have properties worth knowing precisely rather than by name.

Search & Ranking Systems covers inverted indexes, relevance scoring, and the pipeline from ingestion to query. It is broken out because search is a common design prompt with a body of specific mechanics that general architecture knowledge does not supply, and because a search index is the clearest everyday example of a derived, deliberately stale copy of the truth.

Observability at Scale covers metrics, logs, traces, cardinality control, and designing systems that can be debugged live. It exists because "how would you know this was failing" is the highest-frequency follow-up in the entire round, and because cardinality is a real cost constraint that catches out candidates who propose tagging every metric with a user identifier.

Security & Multi-tenancy covers tenant isolation, secrets management, defence in depth, and threat modelling a distributed system. Separate because multi-tenancy is a structural decision made early — shared schema, schema per tenant, database per tenant — with consequences for cost, blast radius and compliance that cannot be retrofitted cheaply.

Full Case Studies works complete designs end to end: a URL shortener, a news feed, chat, ride-hailing and others. It is last on purpose. A case study is where the other eleven subsections get exercised together under a stated set of constraints, and its value comes from watching the constraints drive the choices rather than from the finished picture.

Where it sits in a real system

Every non-trivial system is assembled from a small set of building blocks, and fluency means knowing what each one costs, not that it exists. A load balancer distributes requests and thereby forces you to decide whether your services hold state. A relational database gives you transactions and a query planner within one node's reach, and forces a decision about sharding once it exceeds that. A replica gives you read capacity and introduces replication lag, which is why a user who just posted a comment sometimes cannot see it. A cache absorbs read load and introduces invalidation, staleness, and the stampede that happens when a hot key expires and ten thousand requests hit the origin at once. A message queue decouples producer from consumer and hands you retries, duplicate delivery, ordering questions and a backlog that grows silently. An object store gives you cheap durable bytes and no query capability. A search index gives you relevance and is by nature a stale derived copy of the truth.

That list is worth holding as a ledger rather than a menu, because in the interview you will be asked what each addition cost you.

Composition is where it becomes design rather than a parts list. Consider a URL shortener, which sounds trivial and is a good demonstration of how the constraints, not the feature, dictate the shape:

Notice that the read/write ratio — overwhelmingly reads — is what makes every row defensible, and that the last row exists because one feature refuses to obey the pattern the rest of the design uses. Finding that row is the skill. Reciting the parts list is not.

The first ten minutes

More of the score is decided before you draw a box than candidates expect, and the opening has a structure you can rehearse. It is not a script to recite; it is an order of operations that keeps you from designing a system nobody asked for.

flowchart TD
    A[Prompt arrives, deliberately vague] --> B[Ask about users, scale and read write mix]
    B --> C[Name the two or three hard requirements]
    C --> D[State what is out of scope and why]
    D --> E[Estimate volume, storage and bandwidth aloud]
    E --> F[Sketch the smallest design that meets it]
    F --> G[Pick the riskiest component and go deep]

The step people skip is the fourth. Naming what you are excluding turns an omission into a decision, and it is the cheapest way to look senior in the first five minutes.

Each step has a purpose you should be able to state. The questions in step two exist to convert an unbounded problem into a bounded one, and the read/write ratio in particular does more to shape a design than any other single number, because it decides whether caching and replication are the answer or an irrelevance. Step three matters because most prompts contain one requirement that is genuinely hard and several that are routine, and identifying which is which is the judgement being assessed. Step five keeps you honest about whether the design you are about to draw is even necessary — a great many prompts, sized properly, fit on one large machine with a replica, and saying so is a stronger answer than a distributed design nobody needed.

Step six is where candidates most often overreach. The smallest design that meets the stated requirements is the correct starting point, and you should say explicitly that you are starting simple and will add machinery where a specific requirement forces it. That framing gives every subsequent addition a reason, which is exactly what you want when you are asked to defend it. Beginning with a full microservice topology leaves you with nothing to justify and no room to demonstrate reasoning.

Estimation you do out loud

Estimation gets treated as a ritual and it is actually a filter. Its purpose is not precision; it is to establish whether a requirement implies one machine or fifty, because that single fact determines whether the rest of your design is proportionate. The convention is to round aggressively, state every assumption, and carry the units.

Work an example. Suppose a service records an event every time a user opens a document, and the product has ten million monthly active users who each open twenty documents a day.

Events per day     = 10,000,000 users x 20 opens          = 200,000,000
Average per second = 200,000,000 / 86,400                 ~ 2,300 per second
Peak, assume 5x    = 2,300 x 5                            ~ 12,000 per second
Bytes per event    = ids, timestamp, small payload        ~ 200 bytes
Raw per day        = 200,000,000 x 200 bytes              = 40 GB per day
Raw per year       = 40 GB x 365                          ~ 14.6 TB per year
Replicated 3x      = 14.6 TB x 3                          ~ 44 TB per year

Now read what those numbers actually say, because that is the part that scores. Twelve thousand writes per second at peak is beyond a single unsharded relational primary for this workload, which means the write path needs partitioning or an append -only log rather than a table with an index on every column. Forty terabytes a year means retention is a design decision, not an afterthought, so you should ask how long the events must be queryable and propose rolling older data into aggregates or cold storage. And the 5x peak assumption is the one to say aloud and invite correction on, because if the real answer is 50x — a product with a hard daily spike — the design changes shape entirely.

Notice what was not done. No attempt was made to be accurate to two significant figures, and every input was a stated assumption rather than a fact asserted from nowhere. An interviewer who disagrees with your 5x will tell you, and the conversation that follows is more valuable than the arithmetic.

Who does this work

Formally, senior and staff engineers, technical leads, and architects. In practice the split worth understanding is between the people who make these decisions and the people who live with them. A staff engineer choosing a data store spends the week writing a document, arguing with three teams about a migration path, and estimating cost at projected volume. A site reliability engineer inherits that choice and discovers under load that the shard key was wrong, which is why SREs are frequently the sharpest design interviewers — they have watched theories fail.

Solution architects on the specify side design systems they will not build, which sharpens their trade-off articulation and can dull their instinct for what is actually painful to operate. And a large amount of real design is done by mid-level engineers without the title, in a pull request that adds a cache or a queue and quietly commits the system to a new failure mode.

A day rarely looks like a whiteboard. It looks like reading dashboards from the last incident, sketching two options, writing the document that compares them, and then defending the choice to people whose priorities differ from yours. The interview is a compressed simulation of that last part.

The document is worth dwelling on, because it is the real deliverable and it has a recognisable shape: the problem, the constraints with numbers attached, two or three options, what each buys and costs, the recommendation, and the migration path from what exists today to what is proposed. Candidates who have written one of those recognise the design round immediately, because the round is that document delivered verbally in forty-five minutes.

Demand, adoption and how that is changing

Demand is very high and structurally so, for a reason that has nothing to do with fashion: the default architecture of a serious product is now distributed. Cloud platforms made replication, queues and managed stores available to any team with a credit card, which means the questions this discipline answers — what happens when a region goes away, which writes must be linearisable, what does this cost at ten times the volume — are now live questions for ordinary companies rather than for a handful of large ones. Cost pressure sharpened it further: the last few years have made "what does this design cost per month" a first-class design constraint rather than a finance concern, and candidates who can reason about it stand out.

This is also the round that most often decides senior and staff outcomes, and it is worth being direct about why. At entry and mid level, a hiring decision rests mostly on whether someone can code competently and will be pleasant to work with, both of which other rounds establish. Above that, the question changes to whether this person can be handed an ambiguous problem and trusted to produce a defensible plan that other engineers will follow — and the design round is the only part of a loop that observes that directly. Committees also treat it as the down-levelling signal: a candidate who codes well but designs shakily is very commonly offered a level lower rather than rejected, and the feedback that produces that outcome nearly always cites the design round.

What has changed in the format is the weight on interrogation. Models will produce a plausible high-level architecture for any well-known system on request, so a fluent-sounding first sketch now proves much less than it did, and interviewers have compensated by pushing harder on the follow-ups: what breaks if this node dies mid-write, why not the other store, what does this cost, how would you know it was failing. The premium moved from producing the diagram to surviving questions about it.

A second shift is the rise of the design-review format, in which you are handed an existing architecture and asked what is wrong with it. It is a better test than the blank prompt in one specific way: you cannot steer it towards material you rehearsed. Preparing for it means practising critique — reading a design and asking where the single points of failure are, which component has no owner during an incident, and which assumption the whole thing rests on.

What makes it hard

The first difficulty is that the question is deliberately underspecified, and most of the score is earned before you draw anything. "Design a ride-hailing service" has no answer until someone establishes scale, read/write mix, latency expectations, consistency requirements and what is explicitly out of scope. Candidates who start drawing immediately have skipped the part being assessed, which is whether you can impose structure on ambiguity rather than wait to be told what to build. The strong version of this is scoping that is visibly deliberate: naming what you are excluding and why, so the interviewer can see a decision rather than an omission.

The second is that trade-off defence is genuinely a distinct skill from making choices. You will be pushed on every significant decision, sometimes correctly and sometimes as a test of whether you fold. What is being read is not whether you picked the option the interviewer prefers — usually several are fine — but whether you know what your choice costs. "Postgres, because the access patterns are relational and I want transactions on the booking path; I accept that I will need to shard by region past a few tens of thousands of writes per second, and here is why that key works" is a strong answer. "Postgres, because it scales well" is a weak one even when Postgres is right. Candidates lose points by abandoning a defensible choice under mild pressure, which reads as having had no reason for it.

The third, and the reason experience is hard to substitute here, is that the judgement comes from having operated systems. Reading tells you that replication lag exists; running a service teaches you that the bug report will be "my comment disappeared" and that the fix is routing that user's reads to the primary for a few seconds. Reading tells you a cache can go stale; an incident teaches you that the real problem was a thundering herd on a cold cache after a deploy, at four in the morning, and that the mitigation is request coalescing and jittered TTLs. Reading tells you queues buffer load; being paged teaches you that an unbounded queue converts a throughput problem into a two-hour latency problem that looks fine on every dashboard you had. This is knowledge shaped by consequence, and it shows up in interviews as specificity — the candidate who says "we set the TTL to ninety seconds and still got a stampede on deploys, so we added jitter" has clearly been there, and the one who says "caching improves performance" has not.

Fourth, and underrated: estimation. Being able to say roughly how much storage a year of events consumes, or whether a requirement implies one machine or fifty, is what separates a design that is sized from a design that is decorative. It is arithmetic, but it has to be done aloud, from assumptions you state.

Fifth, and the one that catches strong engineers, is that the same design is correct at one company and negligent at another. A five-person team with a single product should not run a service mesh, and a regulated bank cannot run the pragmatic thing a five-person team should. Because the interview strips out the organisational context, candidates default to the architecture from their last job, which is defensible only if they can say what about that context made it right.

Consistency, replication and the honest version of eventual

The consistency material is where fluency and understanding diverge most sharply, because the vocabulary is easy and the implications are not. Start with the simplest observable consequence: a user writes something and immediately reads it back from a different replica.

sequenceDiagram
    participant U as User
    participant P as Primary
    participant R as Replica
    U->>P: Write new comment
    P-->>U: 201 Created
    U->>R: Read the thread
    R-->>U: Thread without the comment
    P->>R: Replicate, milliseconds later

The gap between the third and fifth lines is the whole of replication lag. Nothing has failed and no error was returned, which is why this class of bug arrives as a confused support ticket rather than an alert.

The mitigations are worth knowing by name because they are what a follow-up asks for. Read-your-writes consistency can be achieved by routing a user's reads to the primary for a short window after they write, or by having the client carry the version it last saw and having the replica wait until it has caught up to that version. Monotonic reads — never seeing time go backwards — can be achieved by pinning a user to one replica. Both are targeted fixes for specific anomalies, and offering one by name is far stronger than offering "we would use strong consistency", which is a way of saying you would like the problem to not exist.

CAP is the theorem everyone cites and few state correctly. It says that when a network partition occurs, a distributed system must choose between remaining available and remaining consistent; it does not say you must choose two of three properties as a lifestyle. The more useful framing for design work is PACELC, which adds the case that matters far more often: in the absence of a partition, you are still trading latency against consistency, because a write acknowledged by a quorum across regions is slower than a write acknowledged locally. Most of your real decisions live in that second clause, since partitions are rare and latency is every day.

What this means practically is that "eventually consistent" is not a property to accept or reject but a question to make specific. Eventually how long — milliseconds, or minutes if a consumer is backed up? Which anomalies are visible in the meantime: stale reads, reads that go backwards, two clients seeing different orders of the same two events? And what happens to the business if a user acts on the stale value? A design that answers those three questions has engaged with consistency. A design that says the word has not.

Why study it

Study it if you are targeting senior or above, because it is the round with the most leverage over both the offer and the level. Study it if you want to move from implementing decisions to making them, since the skill and the promotion criterion are the same thing. And study it because unlike coding-round practice, the preparation improves your actual work: understanding what replication lag does to a user experience makes you better on Tuesday, not just in an interview.

Be honest about who should deprioritise it. If you are interviewing for an entry or junior role, most loops weight coding far more heavily and a shaky design round will not sink you. Specialists interviewing within their specialism will usually face a domain-shaped design round instead — a data engineer gets pipeline and modelling questions, an ML engineer gets training and serving architecture — so generic web-scale case studies are a poor use of your time. And if you have never operated anything, the fastest route to a better design round is not more case studies.

That is the warning worth stating plainly. Reading design write-ups without having operated a system produces a candidate who can draw the boxes and cannot defend them: the diagram is correct, the vocabulary is fluent, and the first follow-up about failure behaviour or cost or how you would detect the problem gets a confident answer with nothing behind it. Interviewers detect this quickly and reliably, because they only have to ask "how would you know that was happening" once. If you are in this position, the highest-return move is to build something small, put load on it, break it deliberately, and watch what your monitoring does and does not tell you. One system you have genuinely operated is worth more in this round than fifty case studies you have read.

Your first hour

Take a system you have actually worked on — however small, however unglamorous — and write its design document backwards, from the running thing to the reasoning. This works far better than a case study because you can check every claim against reality.

One page. Start with the requirements as they really were, including the ones nobody wrote down. Then list every component and, for each, one sentence on what it buys and one on what it costs. Then estimate: requests per second at peak, data added per month, and how far the current setup is from falling over. Then the section that matters most, written as a table:

Failure                     Detected by        Blast radius        Current mitigation
Primary DB unreachable      Health check       All writes fail     Manual failover, ~10 min
Cache cluster cold          Latency alert      Origin overload     None — this is a real gap
Queue consumer stalls       Lag alert          Silent staleness    Alert exists, no runbook
Third-party API slow        No alert           Requests pile up    None — no timeout set

Fill the "current mitigation" column honestly, including the rows where the answer is "nothing". Those rows are the most valuable output of the exercise: each one is a specific, credible thing you can talk about in an interview, and several are probably worth fixing at work this month. You end the hour with a one-page document and a gap list — which is exactly the artefact a staff engineer produces, at a smaller scale.

If you have a second hour, spend it on the row you marked as the worst gap: work out what you would have to build to close it, and how you would know it had worked. That second hour turns a list of weaknesses into a design conversation, which is the actual shape of the interview.

What this is not

It is not memorising reference architectures. Candidates arrive able to reproduce a canonical news-feed design and come apart when the requirements are shifted slightly, because they learned an answer rather than a method. The round is designed to detect this and does.

It is not low-level or object-oriented design, which asks how to structure classes and interfaces within one service and is usually a separate round with separate criteria. It is not cloud certification either: knowing which managed service maps to which need is useful, but a certification tests recall of a vendor's catalogue, while a design round tests whether you can justify needing the thing at all.

It is not the same as being an architect by title, and in many organisations the deepest design judgement sits with senior engineers who are still writing code, precisely because they remain exposed to the consequences of their choices.

It is not a competition to name the most components, either. Adding a queue, a cache, a search index and a service mesh to a problem that did not require them reads as pattern matching rather than design, and every unnecessary component is another thing you will be asked to defend. Restraint is a scored behaviour.

And it is not a search for the right answer. Two competent engineers will design the same system differently and both be defensible. The interviewer is not comparing your diagram to a key; they are working out whether you understood the requirements, whether your choices follow from them, and whether you know what each one cost you.

Interviewers score the follow-up, not the diagram — the question that separates candidates is "how would you know this was failing", and only operating something teaches you the answer.