hardScenarioDesignSeniorStaffLead
The table is 800GB and the shard key turned out wrong. How do you reshard while writes continue?
Changing a shard key moves rows between shards, so it is a copy-and-delete across nodes rather than an UPDATE. Put a bucket-to-shard directory in front of the data, move one bucket at a time behind a freeze measured in milliseconds, verify each bucket before flipping its ownership, and keep every bucket independently reversible.
5 min readUpdated 2026-08-15
Answer
Short answer
Online resharding works when the table is moved as small virtual buckets, not as one 800GB operation. Copy and stream changes for one bucket, briefly freeze only that bucket, verify it, flip the directory entry, and keep rollback until the source rows can be safely cleaned up.
Was this answer helpful?
Likely follow-ups
- What happens to a row whose new shard key value changes while its bucket is mid-move?
- How do you keep a cross-shard read consistent during the window when some buckets have moved and others have not?
- Your bucket copy is 400 buckets in and the source shard runs out of disk. What is the recovery?
- Which queries get slower after the reshard, and how would you have known that before starting?
Related questions
- How do you choose a datastore, and then how do you choose its shard key?hardAlso on sharding and resharding6 min
- You are asked to design something distributed and your honest answer is one Postgres box. When is that the strongest answer in the room, and when does it end the interview?hardAlso on sharding5 min
- How would you autoscale a GPU inference service?hardAlso on capacity-planning6 min
- How do you decide whether to use a managed service or self-host a component, and which cloud costs catch teams out?hardAlso on capacity-planning6 min