System design fundamentals

It distributes requests across instances, which is what makes horizontal scaling and rolling deploys possible. The cost is that your services must become stateless, because a given user's next request may land anywhere. That single requirement pushes session state into a shared store or a signed token, and it is the most common thing candidates forget to say. Sticky sessions avoid it and reintroduce the problem they were avoiding, since a instance's failure now loses whoever was pinned to it.

A layer 4 balancer routes on IP and port without looking at the payload, so it is fast and protocol-agnostic and cannot make decisions based on the request. A layer 7 balancer parses HTTP, so it can route by path or header, terminate TLS, retry idempotent requests and apply per-route rate limits — at the cost of more work per request. Most application traffic wants layer 7; layer 4 suits raw throughput and non-HTTP protocols.

Read capacity and a degree of isolation, so heavy analytical queries do not compete with transactional ones. What it does not give you is a consistent view: replication is asynchronous in most configurations, so a replica lags the primary by anything from milliseconds to minutes under load. That lag is the source of the classic bug where a user posts a comment and does not see it, which is why the follow-up question is always what the user who just wrote sees.

Fluency here means knowing what each addition costs, not that it exists — and the interview asks for the second column.

block            buys you                     costs you
---------------  ---------------------------  ------------------------------------
load balancer    capacity, rolling deploys    statelessness becomes mandatory
read replica     read throughput              replication lag, stale reads
cache            latency, origin protection   invalidation, staleness, stampedes
queue            decoupling, load absorption  duplicates, reordering, silent backlog
append-only log  replay, many consumers       retention cost, partition key locked in
object store     cheap durable bytes          no queries, eventual listing
search index     relevance and text query     a second copy that drifts
shard            growth past one machine      cross-shard queries, rebalancing, hot keys

Read the right-hand column first, because that is where the follow-up questions come from. Every one of them is a question an interviewer can ask, and each has a specific answer: where does session state live now, what does the user who just wrote see, how does a cached item become wrong, how do you detect the consumer falling behind.

The habit worth building is to say the cost as you add the block. "I would put a cache in front of this, which means I now have to decide how it gets invalidated and what happens on a cold start" is a different answer from "I would add a cache", and it is the difference the round is measuring. Adding components without naming their cost reads as pattern matching rather than design, and every unnecessary component is one more thing you will be asked to defend.

Because it stores opaque bytes addressed by key, with no query capability, no transactions across objects and, historically, eventually consistent listing. It is the right home for images, backups, exports and logs — anything large, immutable and retrieved by a key you already have. The design consequence is that you need a database alongside it holding the metadata, because "find all files uploaded last Tuesday by this user" is a question the object store cannot answer.

Caching your responses at points of presence physically nearer the user, so a request for a static asset never reaches your origin and travels a much shorter distance. The latency win is mostly about distance and the origin protection is mostly about volume. The design questions it raises are cache keys — whether the response varies by header or cookie — and invalidation, because a purge across a global network is neither instant nor free.

When the caller needs the outcome to proceed. Queuing a payment authorisation does not remove the wait; it moves it somewhere the user cannot see, and now you own a correlation problem as well. A queue is right when the work merely must happen — sending a receipt, updating a search index, recalculating a recommendation — and wrong when the answer determines what the caller does next.

A queue delivers each message to one consumer and forgets it once acknowledged, so it models work to be done. A log is an ordered, retained sequence that many consumers read independently at their own positions, so it models facts that happened. The log's retention is what allows replay — adding a new consumer that processes history — and its partition key is chosen early and effectively permanently, because changing it reorders everything.

Vertical means a bigger machine, which is simple, requires no code changes and has a ceiling plus a single point of failure. Horizontal means more machines, which has no practical ceiling and requires statelessness, a load balancer and a plan for coordinating state. The honest sequencing is to scale vertically until it hurts, because a surprising number of systems fit on one large machine with a replica — and saying so is a stronger answer than a distributed design nobody needed.

Shared mutable state. In-memory sessions, an in-process cache used as a source of truth, a scheduled job that must run exactly once, a counter incremented locally, a file written to local disk. Each one means a second instance disagrees with the first. Making a service stateless is mostly the work of finding these and moving them to a shared store, or making them idempotent so duplication is harmless.

Because something shared becomes the bottleneck — usually the database, but also a lock, a queue partition, or a third-party rate limit. Beyond that point, extra instances add contention rather than capacity, and can make things worse by increasing connection count on the constrained resource. The useful framing is that horizontal scaling moves the bottleneck rather than removing it, so the design question is always what the next bottleneck will be.

Availability multiplies down a chain, and the arithmetic is worth doing once because it changes how you feel about adding a dependency.

Each dependency independently available 99.9% of the time.

1 dependency   0.999            = 99.90%   ~43 min/month unavailable
2 dependencies 0.999^2 = 0.998   = 99.80%   ~86 min/month
3 dependencies 0.999^3 = 0.997   = 99.70%   ~2h 10m/month
5 dependencies 0.999^5 = 0.995   = 99.50%   ~3h 36m/month

Nothing in your code got worse. You took on more ways to fail, and the composite is worse than any individual part. Latency composes the same way and worse, because tail latencies add: if each call has a p99 of 100ms, the chance that a request touching five of them avoids every tail is about 95%, so your p99 is dominated by whichever dependency was slowest.

The three fixes are worth naming as a set. Remove the dependency from the critical path. Make it optional, returning a degraded response when it is unavailable. Or make it asynchronous, which converts an availability dependency into a latency one — you accept the order even while fulfilment is down, at the cost of handling the message arriving twice.

The design principle that follows is to keep the synchronous call graph shallow and put everything else behind a queue, which is why "how many services does this request touch synchronously" is a question worth asking of any architecture.

A signal from a component that cannot keep up, telling its upstream to slow down rather than silently queuing more work. Without it, an overloaded system accumulates unbounded work, latency grows until every response is useless, and memory eventually runs out. With it, the pressure propagates to the edge where it can be handled by shedding load or rejecting requests — which is worse for individual users and better for the system.

Deliberately rejecting work when the queue of pending requests is longer than you can serve within the deadline. It is correct precisely when the alternative is accepting requests you will fail anyway, because a fast 503 with a Retry-After lets the caller do something sensible while a slow timeout wastes both sides' resources. Candidates almost always suggest scaling up here, which is right eventually — shedding is what you do in the ninety seconds before capacity arrives.

The first design anyone reaches for is a counter per client per minute, and it has a specific hole worth naming before the interviewer does.

Fixed window, limit 100 per minute

  10:00:59   client sends 100 requests   -> all allowed, window 10:00 is full
  10:01:00   client sends 100 requests   -> all allowed, window 10:01 is fresh

  200 requests in about one second, at twice the rate the limiter exists
  to enforce, and at exactly the burst the downstream is least able to absorb.

The token bucket fixes it and is cheaper than it looks, because you never run a refill process — you store two numbers and compute the refill on arrival:

per client:  tokens (a number), last_refill (a timestamp)

on request:
    elapsed = now - last_refill
    tokens  = min(capacity, tokens + elapsed * refill_rate)
    last_refill = now
    if tokens < 1: reject
    tokens -= 1; allow

Three details are the actual answer. Capacity and refill rate are separate knobs, so capacity is your burst allowance and the rate is the sustained limit — real clients are bursty, and a perfectly smooth limiter rejects reasonable traffic. The refill and the consume must be one atomic step, or two concurrent requests both read the last token and both proceed. And the map of clients grows forever unless idle entries are evicted, which is safe because a bucket idle long enough to refill completely is indistinguishable from a fresh one.

The distributed version moves the counter to a shared store with an atomic script, adding a network hop per request — which is why per-instance limits with the rate divided by the instance count is a common and defensible approximation.

A rate limit constrains requests per unit of time to protect the system from bursts; a quota constrains total consumption over a longer period, usually for commercial reasons. They are enforced differently — a rate limit needs a fast counter close to the request path, a quota needs durable accounting that survives restarts. Conflating them produces either a rate limiter that has to be transactional or a quota that resets when a pod restarts.

In the client, at a CDN, at the edge of your system, in the application process, or in a shared store such as Redis. The further from the origin, the greater the latency win and the harder the invalidation, because you control fewer of the copies. An in-process cache is fastest and is per-instance, so ten instances have ten different views; a shared cache is consistent across instances and adds a network hop.

Cache-aside, where the application checks the cache, loads on a miss and populates it, which is the common default and leaves the cache and the store able to disagree. Read-through and write-through, where the cache sits in front and handles loading or writing, which centralises the logic and couples you to the cache. And write-behind, which acknowledges the write and persists later, buying latency at the risk of losing data on failure.

Both appear under load and neither shows up in testing.

Cache stampede
  A popular key expires. Ten thousand concurrent requests all miss, and all ten
  thousand hit the origin at once for the same value.
  Fixes: coalesce the requests so only one recomputes; jitter the TTL so a
  cohort of keys does not expire together; refresh ahead of expiry.

Thundering herd on a cold cache
  A deploy or a restart empties the cache. Every request is a miss until it
  refills, and the origin sees full traffic with no protection.
  Fixes: warm the cache before taking traffic; roll instances gradually;
  allow serving stale while refreshing in the background.

The pattern underneath both is that caches fail all at once rather than gradually, so the origin sees a step change rather than a ramp. That is why a system comfortably serving a 95% hit rate can collapse when the rate drops to zero for thirty seconds — the origin was never sized for the full load, which is the whole reason the cache was added.

The mitigation worth naming unprompted is serving stale data while refreshing. Most read paths tolerate a value a few seconds out of date far better than they tolerate a timeout, and making that trade explicit — stale-while-revalidate — is usually the cheapest available resilience.

Three ways, and they need different answers. The underlying data changed and nothing invalidated the entry, which is a correctness problem solved by invalidating on write or accepting a bounded TTL. Two writers raced and the cache kept the older value, solved by writing through a single path or by versioning. Or the cache was populated from a replica that was itself stale, which is the subtle one because everything looks correct at every step.

Least-recently-used is the sensible default because access patterns are usually temporally clustered, and most cache products implement it or an approximation. Least-frequently-used suits workloads with a stable hot set and adapts badly to changes. The more important decision is usually the eviction trigger — a memory bound rather than a count — and whether eviction under pressure is acceptable at all, because a cache that silently drops a rate-limit counter is a correctness problem rather than a performance one.

A cache is a copy of something you could recompute, keyed for lookup, and tolerating misses. A materialised view is a persisted derived result that is maintained rather than recomputed on demand, so it is expected always to exist. The practical difference is failure behaviour: a cache miss falls back to the source, while a stale materialised view just returns the wrong answer, which is why a view needs a refresh strategy and a way to detect that it has fallen behind.

By the access pattern and the consistency requirement, not by preference. If the data is relational, the queries are varied, and correctness matters, a relational database with transactions is the default and the burden of proof sits elsewhere. Reach for a key-value store when you always know the key, a document store when the entity is naturally one nested object, a wide-column store when write volume is very high and access patterns are known in advance, and a graph store when relationships are the data rather than an attribute of it.

Splitting data across machines so that no single one holds it all, which is what gets you past one machine's write capacity. It commits you to a shard key, because that key determines where every row lives and therefore which queries are cheap. Queries that include the key touch one shard; queries that do not must fan out to all of them and merge, which is slower than the unsharded version was. Changing the key later is a migration of everything.

The failure is not a crash. It is one shard doing most of the work while the others idle, and it follows directly from the key.

Shard by customer_id, hashed        Shard by created_date
  shard 0  ~25% of writes             shard 0  Jan-Mar   0% of writes
  shard 1  ~25%                       shard 1  Apr-Jun   0%
  shard 2  ~25%                       shard 2  Jul-Sep   0%
  shard 3  ~25%                       shard 3  Oct-Dec   100%  <- all of today

Sharding by date puts every write on one shard at a time, so you have the operational cost of four machines and the write capacity of one. It is a recognisable mistake because the reasoning that produces it — "queries are usually by date range" — is about reads.

Hashing the customer identifier spreads writes evenly and has its own failure: one enormous customer is still one shard. That is the hot key problem, and the usual mitigations are a composite key that splits a large tenant across several shards, or moving the largest tenants onto dedicated infrastructure.

The question to ask of any candidate key is what happens when one value of it becomes ten per cent of traffic. If the answer is "that shard falls over", the key is wrong for a system where any single entity can grow.

Partitioning splits data so each node holds a subset, which is about capacity. Replication copies data so several nodes hold the same subset, which is about availability and read throughput. They are orthogonal and usually combined: shard for capacity, then replicate each shard for durability. Confusing them produces designs that add replicas to solve a write-capacity problem, which does not help because every replica takes every write.

Because two-phase commit requires every participant to hold locks until the coordinator decides, so a slow or failed coordinator blocks all of them, and availability becomes the product of every participant's. The common alternative is a saga: a sequence of local transactions with compensating actions when a later step fails, which trades atomicity for availability and requires designing a business-level undo. The best answer is often to move the boundary so the transaction is not distributed at all.

Read speed is subsidised by writes: every insert, update or delete touching an indexed column must maintain the index, and each index consumes storage and cache that the table itself wanted. So the right number of indexes is the smallest set that serves the queries you actually run. In a design discussion the relevant point is that a write-heavy table with eight indexes has a write amplification problem that no amount of scaling the application layer fixes.

Take four shards and the obvious placement rule, then add a fifth machine.

Placement: shard = hash(key) mod N

  N = 4   key "user-1001" -> hash 8,472,193 mod 4 = 1   lives on shard 1
  N = 5   key "user-1001" -> hash 8,472,193 mod 5 = 3   lives on shard 3

Roughly 80% of all keys change shard. Every cache entry is a miss and every
row must move, to add 25% more capacity.

Consistent hashing places both keys and nodes on a ring and assigns each key to the next node clockwise. Adding a node then captures only the arc between it and its predecessor, so about 1/N of keys move rather than most of them, and only from one neighbour.

flowchart LR
    K[Key hashed onto the ring] --> R[Walk clockwise]
    R --> N[First node found owns the key]
    N --> V[Virtual nodes spread each machine<br/>across many ring positions]
    V --> B[Load stays even and<br/>a departure spreads over all peers]

The virtual-node detail is the part that matters in practice. With one ring position per machine the arcs are uneven by chance, so one node gets far more than its share, and when a node leaves its entire load lands on a single neighbour — often enough to topple it in turn. Giving each machine a few hundred positions makes the distribution even and spreads a departure across every remaining peer.

This is why it appears in caches and in partitioned stores alike: both need to add and remove capacity without reshuffling everything, and both are the kind of system where a mass invalidation is itself the outage.

The history. A table holds current state, so an update destroys what was there before, while a log holds the sequence of events that produced the state. That enables replay to rebuild a projection, adding a new consumer that processes the past, and auditing what happened rather than what is. The costs are retention, the need for every consumer to be idempotent, and that answering "what is the current state" now requires folding the log or maintaining a projection.

That when a network partition occurs, a distributed system must choose between remaining available and remaining consistent. It does not say you pick two of three as a lifestyle, which is the usual misstatement. Partitions are rare, so the more useful framing for design work is PACELC, which adds the everyday case: in the absence of a partition you are still trading latency against consistency, because a write acknowledged across regions is slower than one acknowledged locally.

Very little on its own, which is why it should always be turned into three specific questions. Eventually how long — milliseconds, or minutes when a consumer is backed up? Which anomalies are visible meanwhile: stale reads, reads that go backwards, two clients seeing different orders of the same events? And what does the business do if a user acts on the stale value? A design that answers those has engaged with consistency; one that says the phrase has not.

Nothing fails and no error is returned, which is why this arrives as a confused support ticket rather than an alert.

sequenceDiagram
    participant U as User
    participant P as Primary
    participant R as Replica
    U->>P: Write a new comment
    P-->>U: Created
    U->>R: Read the thread
    R-->>U: Thread without the comment
    P->>R: Replicate, milliseconds later

The gap between the third and fifth lines is the whole of replication lag. The user did exactly what the system invited them to do and the system lied to them.

Two named fixes, and offering one by name is far stronger than saying you would use strong consistency, which is a way of asking for the problem not to exist.

Read-your-writes routes a user's reads to the primary for a short window after they write, or has the client carry the version it last saw so the replica can wait until it has caught up. It fixes the case above and only that case.

Monotonic reads pins a user to one replica, so they never see time go backwards by hitting a further-behind replica on the next request. Without it, a user can see their comment, refresh, and watch it disappear — which is worse than never seeing it.

Both are targeted fixes for specific anomalies, which is the point: you name the anomaly you are preventing rather than reaching for a global guarantee.

A rule that a read or write must be acknowledged by a minimum number of replicas. If reads plus writes exceed the total replica count, then any read set overlaps any write set, so a read is guaranteed to see the latest acknowledged write. That is the whole arithmetic, and it lets you tune the trade: writing to all and reading from one favours reads, the reverse favours writes. It does not protect against every anomaly, only staleness of acknowledged writes.

An operation is idempotent when performing it several times leaves the same state as performing it once. It matters because every remote call has three outcomes rather than two — success, failure, and unknown — and the only safe response to unknown is to retry. Without idempotency, a retry after a timeout charges the customer twice. The standard mechanism is a client-supplied key recorded with the result, enforced by a uniqueness constraint at the point of persistence.

Not achievable across a network, and the phrase should be treated as a warning in a design discussion. What you can build is at-least-once delivery plus a consumer that recognises a repeat, and the combination behaves as though it were exactly once. The distinction matters because "the broker gives us exactly-once" usually means a vendor's guarantee that holds within one system's boundaries and evaporates at the first external side effect.

Strong consistency makes every read see the latest write, as if there were one copy, which requires coordination and costs latency. Causal consistency guarantees only that operations which depend on each other are seen in order — so a reply never appears before the comment it replies to — while unrelated operations may be seen in different orders by different observers. Causal is substantially cheaper and is enough for most user-facing behaviour, which is why it is worth knowing as an option between strong and eventual.

At-most-once, where a message may be lost but never duplicated, which suits telemetry. At-least-once, which is what nearly every practical system provides, where a message will arrive and may arrive repeatedly. The engineering consequence is that consumers must be idempotent, and that is not a refinement — it is the price of using a queue at all. Anything claiming exactly-once is doing at-least-once plus deduplication somewhere.

By partitioning: messages sharing a key go to the same partition and are ordered within it, while there is no ordering across partitions. So ordering is purchased with parallelism, because a partition is consumed by one consumer at a time. Choosing the key is therefore choosing both your ordering guarantee and your maximum concurrency, and it is effectively permanent, since changing it reshuffles which messages are ordered relative to each other.

Holding messages that have failed processing repeatedly, so a poison message does not block the partition behind it or loop forever consuming a worker. The part that matters is what happens next: a dead-letter queue nobody monitors is a silent data-loss mechanism with extra steps. It needs an alert on depth, a human owner, and a way to replay a message after the underlying bug is fixed.

The failure mode is the mirror image of a synchronous one, and it is the reason asynchronous design does not remove your monitoring obligation.

Synchronous            Asynchronous
-----------------      ------------------------------------------
Error rate rises       Error rate stays at zero
Latency spikes         Latency looks perfect
Alerts fire            Nothing fires
Users complain now     Users complain in four hours about a receipt
                       that never arrived

The queue absorbs the failure. Producers are still succeeding, the dashboard is green, and the only evidence is that the consumer lag is growing — a metric nobody added because the service looked healthy.

So the monitoring changes shape. For a synchronous service you watch error rate and latency; for an asynchronous one the primary signals are consumer lag, message age at processing, and dead-letter depth. Lag consistently near zero means consumers are keeping up; a rising floor means they are not and you are one traffic increase away from a backlog that takes hours to drain.

The second-order point is that a backlog is not just delay. Once the queue holds four hours of work, restarting the consumers does not help, adding consumers is limited by your partition count, and the business consequence — orders unshipped, refunds unissued — has already happened.

Two systems, no shared transaction, and a window in between where a crash loses data with no error recorded anywhere.

BEGIN
  INSERT INTO orders ...
COMMIT                      <- durable
                            <- crash here
broker.publish(OrderPlaced) <- never happens

The order exists. Nothing downstream ever hears about it. No exception was
thrown, no retry fires, and the discrepancy surfaces days later as an order
that was paid for and never shipped.

Reversing the order does not help — it produces the mirror bug, an event for an order that does not exist. The outbox removes the window by making the event part of the same commit:

BEGIN;
  INSERT INTO orders (id, customer_id, total) VALUES (...);
  INSERT INTO outbox (id, topic, payload, created_at)
       VALUES (gen_random_uuid(), 'order.placed', '{"orderId":"..."}', now());
COMMIT;

A separate relay polls the outbox — or tails the write-ahead log — publishes each row, and marks it sent. Now there is exactly one durable decision point, and everything after it is retryable.

Three consequences to state before you are asked. Delivery is at-least-once, because a crash after publishing and before marking sent republishes, so consumers must deduplicate on the event id. The table grows and needs pruning or partitioning, since it is a queue that happens to live in your database. And the relay is a single logical consumer, so ordering holds per aggregate but the relay itself needs to be highly available or events sit undelivered while everything looks healthy.

When a database write and a message publish must both happen or neither. They are two systems with no shared transaction, so a crash between them leaves the row written and the event lost, with no error anywhere. The outbox writes the event into a table in the same transaction as the data, and a separate process publishes from that table. It is at-least-once, so consumers still need to deduplicate, and the table needs pruning.

A distributed transaction attempts atomicity across systems with a coordinator and locks. A saga abandons atomicity: it is a sequence of local transactions, each committed independently, with compensating actions to undo earlier steps when a later one fails. That means intermediate states are visible to users and the compensations themselves can fail, so each must be idempotent and retryable, and the orchestration state must be persisted rather than held in memory.

Because without one, a slow dependency does not fail — it holds your thread, connection or worker indefinitely, and under load that exhausts the pool and takes down endpoints that never touched the dependency. Many client libraries default to no timeout, which means the default behaviour is the dangerous one. The related discipline is that the timeout should be shorter than your own caller's, or you will still be working on a request nobody is waiting for.

A timeout is local: this call gets three seconds. A deadline is a point in time that travels with the request through every hop, so a service that received a request with 800 milliseconds remaining does not start a two-second call downstream. Without propagated deadlines, work continues on requests whose callers have already given up, which is exactly the load you least want during an incident.

A retry is extra load applied at precisely the moment a dependency is struggling, so the intuitive fix amplifies the problem.

Normal:     1,000 req/s reaching a dependency
Dependency slows; clients retry 3 times with no backoff
Effective:  up to 4,000 req/s at the moment it is least able to cope
Result:     a slow dependency becomes a dead one

Worse, retries synchronise. Every client that failed at the same moment retries at the same moment, producing waves rather than a smooth load, which is why jitter matters as much as backoff.

The full set is worth naming together, because each element answers the previous one's failure:

timeout          stop waiting, so a slow call fails rather than hanging
bounded attempts stop retrying, so failure is finite
exponential backoff  give the dependency room to recover
jitter           desynchronise clients so retries do not arrive as a wave
retry budget     cap retries as a fraction of normal traffic
circuit breaker  stop calling a dependency that is clearly failing

The one candidates omit is the retry budget. Backoff limits how fast one client retries and says nothing about ten thousand clients each retrying politely, so a global cap — retries may not exceed some small percentage of normal traffic — is what actually protects the dependency.

And the precondition for any of this: retries are only safe if the operation is idempotent. Adding retries to a non-idempotent write does not improve reliability, it creates duplicate charges.

It converts a slow failure into a fast one across the whole client, rather than per request. After a failure threshold it stops calling the dependency altogether, failing immediately and freeing the resources that would have been held waiting. The half-open state is the design: after a cooldown it allows a few trial calls, so a recovering dependency is not immediately hit with full traffic — which is how a service that was about to recover gets knocked over again.

A limit on how much of your capacity any single dependency can consume, usually a separate connection or thread pool per dependency. Without it, one slow downstream drains the shared pool and endpoints that never touch it start failing. It is the mechanism behind the observation that an outage in a minor recommendation service can take down a checkout page: nothing was logically coupled, but the resources were.

Deciding in advance which parts of a response are essential and which can be omitted when a dependency is unavailable. A product page without live stock counts and without personalised recommendations is still a product page; a product page that returns 500 because the recommendation service is down is a lost sale. Making that explicit means each call site has a defined fallback, which is a product decision rather than an engineering default.

Latency, traffic, errors and saturation — the golden signals — measured as close to the user as possible. The reason they are chosen is that they cover the question "is the service doing its job" without requiring knowledge of the implementation. The most common mistake is alerting on causes rather than symptoms: high CPU is not a problem if users are being served, and a service where every instance is healthy can still be failing every request.

Nines are quoted so casually that the underlying budget disappears. Converted into time, the operational demands of each tier become obvious.

target     downtime/month   downtime/year   what it implies
---------  ---------------  --------------  ------------------------------------
99%        7h 18m           3d 15h          a person notices and fixes it
99.9%      43m              8h 45m          paging, runbooks, fast rollback
99.99%     4m 23s           52m             automated failover, no manual step
99.999%    26s              5m 15s          multi-region, and a deploy cannot
                                            be in the failure path

Four minutes a month is less than the time it takes a human to read a page and open a laptop, so four nines means no incident may require a human decision. Five nines means a bad deploy that takes thirty seconds to detect has spent the entire year's budget.

The framing that follows is the error budget: at 99.9% you may be unavailable for 43 minutes a month, and that is a resource to spend rather than a line never to cross. Spend it on releases and risky migrations while it lasts; when it is gone, the team stops shipping features and works on reliability until the window resets. That converts an argument about velocity versus stability into arithmetic.

The number to push back on in a design round is a target nobody costed. Asking "what is the availability target, and does the business actually need the difference between three and four nines?" is a senior question, because the jump usually multiplies infrastructure cost and rules out designs that were otherwise fine.

Because an average describes nobody. Ninety-five requests at 100ms and five at four seconds averages to under 300ms, which sounds fine and matches no actual request. The p99 is four seconds, and that is what your largest customers experience on their busiest page. Two further traps: percentiles do not average across machines, and a page making ten backend calls will hit somebody's tail on most page loads, so a per-service p99 is not the user's p99.

Not for precision, and nobody is checking your multiplication. Its purpose is to establish whether a requirement implies one machine or fifty, because that single fact determines whether the rest of your design is proportionate — a queue and a sharded store are over-engineering at a thousand writes a day and mandatory at a million an hour. It also surfaces assumptions early enough to be corrected: an interviewer who disagrees with your peak multiplier or your payload size will say so, and the conversation that follows is worth more than the arithmetic was. Skipping it means every later choice is unjustified, because nothing has established what the system is actually being asked to absorb.

Round aggressively, state every assumption, and carry the units — then read what the numbers say rather than just producing them.

Assume 10M monthly active users, each opening 20 documents a day.

Events per day     = 10,000,000 x 20                = 200,000,000
Average per second = 200,000,000 / 86,400           ~ 2,300/s
Peak, assume 5x    = 2,300 x 5                      ~ 12,000/s
Bytes per event    = ids, timestamp, small payload  ~ 200 bytes
Raw per day        = 200,000,000 x 200              = 40 GB/day
Raw per year       = 40 GB x 365                    ~ 14.6 TB
Replicated 3x      = 14.6 TB x 3                    ~ 44 TB

Now the part that scores. Twelve thousand writes per second at peak is beyond a single unsharded relational primary for this workload, so the write path needs partitioning or an append-only log rather than a table with an index on every column. Forty terabytes a year means retention is a design decision rather than an afterthought, so ask how long the events must stay queryable and propose rolling older data into aggregates or cold storage.

The 5x peak assumption is the one to say aloud and invite correction on, because if the real answer is 50x — a product with a hard daily spike — the design changes shape entirely.

Two numbers worth memorising so the arithmetic flows: there are roughly 86,400 seconds in a day, and 2.5 million seconds in a month. Everything else can be rounded to one significant figure without affecting the conclusion.

Ask about users, scale and the read/write mix. Name the two or three genuinely hard requirements. State what is out of scope and why. Estimate aloud. Then sketch the smallest design that meets the requirements, and say explicitly that you are starting simple and will add machinery where a specific requirement forces it. That framing gives every subsequent addition a reason, which is what you want when asked to defend it.

Because it turns an omission into a decision. An interviewer cannot tell the difference between a candidate who chose not to design the notification system and one who forgot it, unless you say. Scoping aloud — "I am going to treat authentication as solved and spend the time on the feed ranking, because that is where the interesting constraints are" — is the cheapest way to look senior in the first five minutes.

Whether you know what your choice cost, not whether you picked their preferred option. Several options are usually defensible, so the question behind "why not X?" is whether you considered it and can say why you did not. Abandoning a sound choice under mild pressure reads as having had no reason for it in the first place; defending a poor one past the point of evidence reads worse. The target is to state the trade and hold it while acknowledging the alternative.

Adding components that no stated requirement calls for — a queue, a cache, a search index and a service mesh on a problem that fits comfortably on one database. Every unnecessary component is another thing to defend and another failure mode to explain, and restraint is a scored behaviour rather than a lack of ambition. Sizing the problem first is what prevents it, which is another reason estimation comes before the diagram.

"How would you know this was failing?" It cannot be prepared for generically, because the answer depends on the specific design you just drew, and it requires having operated something rather than read about it. A strong answer names the metric, where it is measured, what the alert threshold is and who acts on it. A weak one describes adding monitoring, which is a category rather than an answer.