When a transatlantic cable is severed and your multi-region database inevitably enters split-brain, how do you resolve the conflicting writes without data loss?

Short answer

An uncompromising examination of split-brain resolution, quorum consensus, fencing tokens, and why timestamp-based last-write-wins is fundamentally flawed.

Use distributed systems as the thread through the resolving split brain explanation: name the signal, then say what changes when it moves.

The active-active delusion

When a catastrophic network partition occurs between global datacenters, the naive assumption is that both regions can simply continue accepting writes independently, and the system will figure it out later. This active-active delusion leads straight to a split-brain scenario where divergent, irreconcilable states are generated. Relying on simple timestamp-based Last-Write-Wins (LWW) to resolve this upon network recovery is a fatal error; in distributed systems lacking a perfectly synchronized global clock (like TrueTime), clock drift guarantees that LWW will silently overwrite valid data.

Quorum and the illusion of safety

Maintaining availability during a hard partition forces the sacrifice of strict consistency, as dictated by the CAP theorem. To prevent isolated nodes from erroneously believing they are the legitimate leader, a fencing token mechanism—supplied by a coordination service like Zookeeper or etcd—must be enforced.

When a leader is elected, it is issued a monotonically increasing fencing token. All storage nodes are mandated to reject write requests bearing a token older than the highest one they have observed. If a European leader is isolated and a new leader in North America is elected with a higher token, delayed packets from the old European leader are mercilessly discarded by the storage layer.

To actively prevent split-brain before it requires resolution, a witness node must be deployed in a third, neutral region. This node does not store data but participates in quorum voting. The region that maintains connectivity with the witness retains the majority quorum and accepts writes, while the isolated region aggressively downgrades itself to a read-only state.

Causal ordering via Vector Clocks

If the architecture deliberately allows writes in partitioned regions to prioritize availability, the system must capture the causal history of every update to merge them later. Vector clocks are mandatory here.

flowchart TD
  A["Client Request"] --> B["European Leader"]
  A --> C["North American Leader"]
  B --> D["Vector Clock Update [EU:5, NA:2]"]
  C --> E["Vector Clock Update [EU:2, NA:5]"]
  D -.-> F["Network Partition"]
  E -.-> F
  F --> G["Conflict Detection Module"]
  G --> H["Resolution Strategy (CRDTs or Manual)"]

When a client performs a write, the local counter in the vector clock is incremented and stored alongside the payload. As the partition persists, the vector clocks diverge. Upon network healing, the database uses these clocks to definitively identify conflicts rather than guessing based on unreliable timestamps.

The reality of conflict resolution

Identifying a conflict is trivial compared to resolving it. For specific, commutative data structures, Conflict-free Replicated Data Types (CRDTs) like grow-only counters and observed-remove sets can automatically merge divergent data without human intervention.

However, for complex relational transactions or highly coupled business logic, automatic merging is often unsafe. The system must route conflicting records into a quarantine queue. These require deterministic, application-specific algorithms for resolution, or, in the worst-case scenario, manual review by an administrator.

Traffic shaping during recovery

When the network link is finally restored, dumping the entire backlog of replication payloads across the wire will likely overwhelm the link, triggering secondary timeouts and causing a false partition. The reconciliation engine must strictly govern replication lag and throttle batch synchronization payloads. Heartbeat intervals and election timeouts must be meticulously tuned to account for the jitter of intercontinental WANs, ensuring transient spikes during recovery do not trigger cascading, unnecessary leader elections.

Distributed consensus relies on majority quorum and fencing tokens to prevent split-brain scenarios, whilst vector clocks and CRDTs are essential for resolving data conflicts when availability is prioritised over strict consistency during network partitions.

© 2026 Preptima. Originally published at preptima.com.