You are moving a sharded datastore to Cassandra with no maintenance window. How do you cut over, and how do you know both copies agree before you do?
Replicate from the source's change log rather than dual-writing from application code, backfill with each row's original write timestamp so history can never overwrite live data, then run continuous range checksums and shadow reads until divergence is understood rather than merely small. The read flip and the write flip are separate steps, each reversible.
What the interviewer is scoring
- Whether change data capture is preferred to application-level dual writes, with the ordering argument given
- Does the candidate know that a backfill must carry the original write timestamp, and say what happens if it does not
- That reads and writes are cut over as two separate reversible steps rather than one switch
- Can they describe a verification that scales to the whole dataset rather than sampling a few rows
- Whether the data model is redesigned per query instead of being ported table for table
Answer
Short answer
For a Cassandra cutover with no downtime, start change-data-capture first, backfill with original write timestamps, then reconcile with range checksums and shadow reads. Flip reads and writes separately so each step is reversible, and do not treat a tiny unexplained mismatch rate as acceptable until every mismatch category is understood.
Mention data migration where it changes the risk, the owner, or the next check.
The cutover is easy; agreement is the work
Flipping a connection string takes a deploy. Knowing that the new store holds the same data as the old one, while both are being written to, is the part that takes weeks. So structure the answer around the verification and let the cutover fall out of it as the last two steps.
The plan has five phases, and each one is reversible on its own.
flowchart LR
A[Replicate live changes from source log] --> B[Backfill history behind the stream]
B --> C[Reconcile ranges continuously]
C --> D[Shadow reads compare and log]
D --> E[Flip reads to Cassandra]
E --> F[Flip writes and retain old store]Note that replication starts before the backfill rather than after it. Start the stream first and the backfill can run for a week without a gap opening behind it; start the backfill first and every change made during it is lost.
Dual writes from the application are the tempting wrong answer
The obvious move is to write to both stores in the service layer. It is easy to explain and it is where most migrations go wrong. Two writes are not one write, so a process that dies between them leaves the stores disagreeing with nothing to detect it. Worse, two concurrent updates to the same row can reach the two stores in opposite orders, and now both are internally consistent and different from each other.
Take the changes from the source's own change log instead. A log is already ordered and already durable, and consuming it gives you a stream you can stop, rewind and replay. No application code changes, which also means no service is blocked on the migration's schedule. If the source has no usable log, adding a trigger-populated outbox table in the same transaction as the write is the fallback, because it keeps the record of the change atomic with the change.
Say the trade-off out loud rather than treating change capture as free: the consumer must be idempotent, because a replay will deliver the same change twice, and the stream is asynchronous, so the new store lags the old by some measurable amount. Both are manageable. Neither is invisible.
The timestamp is the detail that makes a live backfill safe
Cassandra resolves conflicts by comparing timestamps at cell level, and the most recent write wins. That is what makes a concurrent backfill possible, and it is also the trap. A backfill job reading a row that was written last March and inserting it now stamps it with now, so it beats a live write that happened five minutes ago. You have quietly restored old data over new, in a way no error log will mention.
-- Carry the source row's own modification time, in microseconds since the epoch.
-- Without this clause the backfill stamps history with the present and
-- overwrites live writes that are genuinely newer.
INSERT INTO orders (order_id, customer_id, status)
VALUES (?, ?, ?)
USING TIMESTAMP 1743422400000000;
With the original timestamp supplied, the ordering between backfill and live traffic stops mattering. Old data can never win against new data, the two jobs can run in either order, and a backfill can be restarted from the beginning without damage. This one clause is what turns a migration from a race into an idempotent operation, and a candidate who reaches it without prompting has done this before.
Deletes need the same treatment and are harder. A row deleted in the source before your backfill read it will be inserted by the backfill and then look alive. Either take deletes from the change stream as tombstones with their own timestamps, or take the backfill from a snapshot that already excludes them, and say which.
Verifying agreement at a scale where reading every row twice is too expensive
Row counts are the first check and the weakest. They catch missing rows and nothing else, because a row can be present and wrong.
The workable method is range checksums. Split the key space into ranges of a manageable size, compute a digest over each range on both sides, and compare digests. Matching ranges are dismissed in one comparison; a mismatched range is subdivided until you have the individual rows that differ. Run it continuously rather than once, because a passing check the day before the cutover tells you about the day before the cutover. What matters is the divergence rate over time, and whether it is falling.
Then add shadow reads. Serve every production read from the old store as now, and asynchronously issue the same read against Cassandra, comparing the two results and logging any difference along with the key. This is the check nobody can argue with, because it exercises exactly the queries your product makes, in production proportions, including the ones nobody remembered to write a test for. It also surfaces the disagreements that are not bugs: a row that differs only because replication is 200 milliseconds behind is expected, and your comparator needs to know that before it drowns you in noise.
Do not accept a small residual divergence as background. Take a handful of reported keys and explain each one. Every migration worth trusting has a moment where a category of mismatch turns out to be a real defect in the transformation, found because somebody refused to round 0.002 per cent down to zero.
The model does not port, and the missing operations are the surprise
A relational schema sharded on a key does not become a Cassandra schema by being copied. Cassandra tables are designed per query: the partition key decides what one read touches, the clustering columns decide the order within it, and a query that does not match the primary key has no efficient plan. Expect to write the same data into several tables with different keys, and expect the application to maintain those copies, because nothing else will.
Some operations disappear, and enumerating them is what a senior candidate is being scored on.
- Multi-row transactions. There are none across partitions. Conditional writes exist within a single partition, using a consensus round that costs several network trips, so they are for the few places that genuinely need compare-and-set rather than for ordinary updates.
- Ad-hoc queries and joins. Not available. Anything analytical moves to a separate system fed from the same stream, which is a scope item the plan must name rather than discover.
- High-cardinality secondary indexes. A query through one contacts many nodes and gets slower as the cluster grows. Model a second table instead.
- Read-modify-write on a counter. Needs the dedicated counter type or an event-sourced total, not a fetch and an update.
Flipping reads and writes are two decisions
Move reads first, and move them by percentage. Ten per cent of read traffic to Cassandra, watch error rates and latency, then more. Writes are still going to the old store, so the old store is still the truth, and reverting is a configuration change with no data to reconcile.
Only when reads have been served from Cassandra long enough to be boring do you move writes. That step is the one-way door, so keep the change stream running in the other direction if you can, or accept a defined window during which rollback means replaying Cassandra's writes back into the old store. Keep the old store readable, and delete nothing, until the retention period you agreed in advance has passed. A migration is not finished when the traffic moves. It is finished when you delete the old copy on purpose.
Anybody can flip a connection string. What you are being interviewed on is the proof you had beforehand that both copies said the same thing, and whether your backfill could have quietly overwritten live data without telling you.
© 2026 Preptima. Originally published at preptima.com.
Likely follow-ups
- How do you migrate rows that were deleted in the source before the backfill reached them, without resurrecting them?
- Your reconciliation reports 0.002 per cent divergence and it is not falling. What do you do before the read flip?
- Which operations in the old store have no equivalent once the data is in Cassandra, and how do you replace them?
- After the write flip, how long do you keep the old store, and what would make you go back to it?
Related questions
- The clearing house has called far more initial margin than your own risk system predicted. How do you find out why before the deadline?hardAlso on reconciliation6 min
- You forecast every store and also the national total, and the store forecasts do not sum to it. What do you do?hardAlso on reconciliation5 min
- You need to reindex a ten-million-chunk corpus with a new embedding model. How do you do it without downtime?hardAlso on zero-downtime6 min
- You have to release a change to the order-routing path. Walk me through the release, and tell me what makes it different from deploying any other service.hardAlso on reconciliation6 min