Walk me through what happens between a customer order being captured and the service being live on the network.
The order is decomposed against the product catalogue into technical tasks on separate systems, each with its own latency and failure mode. Because nothing spans them transactionally, correctness comes from idempotent tasks, compensating actions, a fallout queue and jeopardy monitoring.
What the interviewer is scoring
- Does the candidate separate the commercial order from the technical tasks it decomposes into
- Whether partial success is designed for as the normal case rather than as an exception path
- That compensation and idempotency are reached for instead of a distributed transaction
- Whether orders that neither complete nor fail get an answer, not just the happy and sad paths
- Does the candidate say who owns the desired state after the order closes
Answer
From a commercial order to technical work
An order captured in a shop or a web journey is a commercial statement: this customer, on this contract, wants this offering at this address from this date, at this price. Nothing on the network understands that sentence. The first job of order management is therefore to decompose it, using the product catalogue, into concrete actions against named systems. For a fixed broadband order that typically means reserving an access port on the aggregation equipment serving the address, allocating subscriber credentials and writing them to the authentication store, pushing a policy profile so the line gets the speed that was sold, creating the subscription in the charging and billing systems, dispatching customer premises equipment, and possibly booking an engineer visit.
Those tasks have almost nothing in common operationally. Two are sub-second API calls. One is a file the mediation platform ingests on a schedule. One is a van that arrives on Thursday, or does not, because nobody was in. Any design that treats them as interchangeable steps will be wrong about precisely the ones that hurt.
Why this is not a database write
The instinct to describe activation as "write the subscriber row and flip a flag" is what the question is testing for. Three properties break it.
The systems have no shared transaction manager. A switch, a policy function, a charging system and a logistics provider cannot enlist in one commit, and even if two of them could, the third is a physical act. Every side effect is externally visible the instant it lands: a port marked assigned is a port other orders can no longer take, whether or not your order eventually completes. And the elapsed time is wrong by orders of magnitude for a transaction — an order that waits nine days for an appointment must survive orchestrator restarts, deployments and version changes, which means the order is durable state advanced by events, not a call stack.
What you can promise is not atomicity. It is convergence to intent, plus an explicit, inspectable answer for every state in between.
Making every task safe to repeat
Retries are unavoidable, so each task needs a stable identity that survives them. Derive the key from the order and the step, never from a timestamp or a fresh UUID per attempt, and require the downstream system to return the original outcome when it sees a key it has already processed.
Plenty of network-facing systems will not do that for you. For those, the task becomes read-then-write and treats "already in the desired state" as success rather than as a conflict.
// Stable across retries, redeploys and duplicate events: same inputs, same key,
// same side effect. A UUID minted per attempt would provision the port twice.
String taskKey = orderId + "#" + stepId;
// The element manager has no idempotency of its own, so converge instead of
// commanding: a second pass over an already-correct port must be a no-op.
if (!accessNetwork.assignmentFor(taskKey).equals(desired)) {
accessNetwork.assignPort(taskKey, desired);
}
This distinction is worth naming out loud in an interview, because retryable and idempotent are not the same word. A task is retryable if calling it again is safe; it is idempotent if calling it again leaves the same state. Only the second gets you through a network partition where you never learnt whether the first call landed.
Compensation is a business decision, not a technical one
When task four fails, tasks one to three have already changed the world, and the interesting part of the answer is what you do about it. Each task therefore declares a compensating action: release the port, delete the authentication entry, cancel the shipment.
The point that separates a strong answer here is that compensation is asymmetric and frequently lossy. A number that has been ported cannot be un-ported by your workflow. A personalised SIM cannot be un-personalised. An engineer who has already visited has already cost money. For those steps the compensating action is a business rule, not an inverse function: raise a returns case, write off the visit, or leave the resource in a quarantined state until someone decides. Candidates who describe rollback as though every step has a clean inverse have not worked on a real activation, and the interviewer knows it.
Orders that neither finish nor fail
The failure mode that actually generates complaints is not a task that errors. It is a task that goes quiet. A partner system accepts a request and never responds; a file is collected but never acknowledged; an appointment slot is booked with a date already in the past.
So every task carries an expected completion time, and a monitor raises a jeopardy alert before the date promised to the customer rather than after it, giving an operations team a window in which the promise can still be kept. Tasks that cannot self-heal park in a fallout queue with enough context for a human to act, and the crucial detail is that the human's resolution must re-enter the workflow rather than bypass it. An engineer who fixes the port by hand and closes the ticket without telling the orchestrator has created a subscriber whose service works and whose record says it does not, which surfaces months later as a billing dispute or as a resource nobody dares reclaim.
Two state models, deliberately kept apart
Keep the customer-facing order state coarse and stable — acknowledged, in progress, held, completed, cancelled, failed — and keep per-task state as detailed as operations need. They serve different readers. Collapsing them means either a shop assistant reads "policy push pending" or an engineer reads "in progress" for nine days. It also means every change to the network estate becomes a change to the customer-visible contract, which is exactly the coupling the OSS and BSS split exists to avoid.
The network keeps its half of the state whether or not your workflow finishes, so design for convergence and compensation rather than for a commit that was never available to you.
Likely follow-ups
- The customer cancels when three of five tasks have completed. What happens to the other three?
- How do you make an activation safe against a duplicate submission from an impatient retry?
- A field engineer resolves a stuck task by hand. How does your workflow learn about it?
- How would you show a customer meaningful progress without exposing network detail?
Related questions
- A subscriber's service is working on the network but no subscription exists in billing. How did that happen, and what do you do about it?hardAlso on provisioning and idempotency5 min
- Walk me through an order's journey from placement to delivery. How would you model its status?hardAlso on order-management and idempotency6 min
- The monthly bill run dies two thirds of the way through and the cycle closes tomorrow. What do you do?hardAlso on idempotency5 min
- A fill arrives for an order your system has already marked cancelled. What do you do?hardAlso on idempotency5 min
- The MES is unreachable for four hours and the line keeps producing. What happens to the production record, and how do you reconcile afterwards?hardAlso on idempotency5 min
- How would you design a data pipeline you can safely re-run?hardAlso on idempotency6 min
- You own the API test suite for a service from scratch. How do you structure it, and what do you mock?hardAlso on idempotency7 min
- An event for 23:58 yesterday arrives at 00:20 today, after you have already published yesterday's totals. What should your pipeline do?hardAlso on idempotency4 min