One celebrity account is ninety per cent of your write traffic on a single partition. What do you do about a key you cannot rebalance?
A hot partition from one account cannot be fixed by ordinary rebalancing because it is one key. Split the key into write-sharded sub-keys or aggregate writes before storage; both choices move cost to reads, ordering, rollups or approximation.
5 min readUpdated 2026-08-15
0
Answer
Short answer
A single hot key cannot be rebalanced across partitions; moving it only moves the hotspot. Either stop it being one key by write-sharding into numbered sub-keys, or reduce the write rate before storage with aggregation or batching. The trade-off is on reads: they now scatter and merge, read rollups, or accept approximate counters.
Was this answer helpful?
Likely follow-ups
→Your sub-key count is 32 and the account grows tenfold. How do you change the count without a migration that stops writes?
→What breaks if the writes to this key have to be read back in the exact order they were made?
→How would you handle this if the hot key is a counter that must be exact rather than a stream of events?
→Who decides an account is hot, and what stops that decision flapping every few minutes?
Rebalancing is the wrong tool, and saying so is the answer
A rebalance moves keys between partitions. You have one key. Move it and you have moved the hot partition, not removed it. Split the partition in two and the key lands wholly in one half, because a partition boundary can only fall between keys. Consistent hashing has the same limit: it decides where a key lives and has nothing to say about how much traffic one key attracts.
That is the first thing an interviewer wants to hear, and many candidates never say it. They reach for a bigger cluster instead. Adding nodes reduces load per node for every key except this one, which stays exactly where it was, taking ninety per cent of the writes on whichever single node now owns it.
So the problem restates itself. You cannot spread one key, so you have to stop it being one key, or stop the writes arriving one at a time.
Splitting the key, and what it costs to read
Write sharding appends a suffix to the partition key and picks the suffix at random on each write. The account's activity now lands across N partitions instead of one, and the arithmetic is direct: ninety per cent of writes divided by 32 suffixes is under three per cent per sub-key, which is the same order as your ordinary traffic.
# Before: every event for this account lands on one partition.PK = "account#nishaverma"# After: the same account, spread over 32 sub-keys.suffix = random.randrange(32) # write side picks at randomPK = f"account#nishaverma#{suffix}"# read side must visit all 32
The comment on the second line is the whole trade. Reads that used to be one partition lookup are now 32 lookups and a merge, which multiplies read cost, adds a tail-latency problem you did not have, and turns a single-partition transaction into something that cannot be one. Choose N deliberately. It is not a tuning knob you can raise later without deciding what to do about data already written under the old N.
There is a cheaper variant when reads are keyed by time. Put the suffix on writes but keep a rollup that a background job compacts into a single readable key on a schedule. Recent data costs a scatter; anything older than the rollup interval costs one read. You have accepted a freshness boundary in exchange for keeping the common read cheap.
Aggregate before it reaches storage
Before changing the data model, ask what the writes are. If they are counter increments, likes, view events or anything else whose individual identity nobody will ever query, the storage layer is the wrong place to fix this.
Buffer them in the service and write a total. A hundred increments a second becoming one write a second is a hundredfold reduction with no change to the schema and no read amplification at all. What you give up is stated plainly: a process that dies with a buffer in it loses those increments, so this is only available where the count is allowed to be approximate, and you should say which numbers in your product are allowed to be approximate before you propose it.
A supermarket queue makes the shape obvious. One customer with nine hundred items blocks a till no matter how many tills you open, and the fix is not another till. It is either splitting their trolley across several tills or scanning a case of soup once with a quantity of twelve. The analogy stops working at the receipt: the shopper accepts one combined bill, while your readers may need every individual line item, and if they do then aggregation is off the table and you are back to splitting.
Isolate it, so the blast radius is one account
Even after splitting or aggregating, the hot account is worth keeping away from everyone else. Give the known-hot keys their own partition set, their own connection pool, or their own cluster. The point is not efficiency. The point is that when this account has a bad day, the other million accounts do not.
Loading diagram…
Look at the two reader edges. They are different code paths with different latency profiles, and pretending they are one is how this design gets shipped and then surprises somebody.
Detection is the part that gets skipped
Every technique above needs to know which keys are hot, and that set changes. An account is unremarkable on Monday and the top of the site on Tuesday. A design that requires a human to add a name to a list will always be a day late.
Measure request rate per key at the layer that already sees every request, keep an approximate top-K rather than exact counts per key, and promote a key into the hot path automatically when it crosses a threshold. Promotion has to be sticky for a while, or a key hovering at the threshold flips between two code paths every minute, which produces the worst behaviour of both. Demotion should be slower than promotion for the same reason.
A strong candidate closes the loop out loud: "I would rather over-detect and put a few warm keys on the sharded path than have the detector be the thing that is late during an incident."
A hot key is a data-modelling problem wearing a capacity problem's clothes. No number of nodes divides one key, so either the key stops being one key or the writes stop arriving one at a time.