Usage records arrive duplicated, late and out of order, and the tariff changed in the middle of the month. How does mediation and rating cope?

What mediation is there to absorb

Mediation exists so that nothing downstream has to know what kind of equipment produced a usage record. It collects from many sources in many formats and on many schedules, normalises them into one internal representation, drops the records that are not billable, enriches them with the identifiers the business uses rather than the ones the network uses, and distributes the result to rating, to the data warehouse, to interconnect settlement and to fraud management. Each of those consumers wants a different subset, which is why mediation is a distribution point and not just a converter.

The reason it is worth a layer of its own is churn. Equipment is replaced on a hardware cycle, and every replacement brings a new record format. Absorbing that in one place keeps rating logic stable while the estate underneath it turns over.

Deduplication at volume

Duplicates are routine, not exceptional. A collection run is retried after a timeout, a file is redelivered because the acknowledgement was lost, a node is restored from backup and re-emits, an operator copies a directory to be safe. Charging the customer twice for one call is the sort of defect that reaches a regulator, so deduplication is a correctness requirement rather than tidiness.

The design decision is what constitutes record identity. Hashing the whole record is the wrong answer, and stating why earns credit: two deliveries of the same usage often differ in fields that are not part of the usage — a collection timestamp, a file sequence number, a padding field, a partial-record flag that was set on one pass and not the other. Use the identity the network itself assigns: the producing element, a charging or session identifier that is unique within that element, and the record sequence number within the session. Those together identify the usage, and they are stable across redelivery.

That key then needs a persisted index, because the process will restart. In practice you keep an in-memory probabilistic filter in front of a durable store so that the overwhelming majority of lookups never touch disk, and only candidates that the filter says might be duplicates are checked authoritatively. The window over which you retain keys is a business decision disguised as a technical one, and it should come from measurement: how late has a record ever arrived from your slowest source, plus the interval over which you would be willing to restate a bill. A window shorter than the worst observed lateness will silently accept a duplicate.

Late and out-of-order records

Two different things get conflated here. Out-of-order arrival within a period is mostly harmless, because rating a record does not depend on the records around it — with the important exception of anything cumulative, such as a bundle allowance, a tiered rate or a volume discount, where the order of consumption changes the price. For those, rate against a subscriber-period aggregate that is recomputed rather than incremented, so a record arriving out of order produces the same total as if it had arrived in order.

Lateness across a period boundary is the genuinely hard case. A record whose usage happened on the 29th and which arrives on the 3rd belongs to the closed period, not the open one. You need an explicit policy: hold the period open for a defined grace window, or accept the record into the next period while retaining its true event time so reporting and any restatement can attribute it correctly. Both are defensible. What is not defensible is letting arrival time decide which bill a call lands on, because then the same usage prices differently depending on how busy the collector was.

Sessions that outlive a period are a related case with a known shape. Long data sessions emit interim records, and the final record may restate cumulative volumes rather than incremental ones. If you sum interim and final naively you overcharge, so mediation must aggregate by session identifier and know, per source, whether counters are cumulative or delta.

Rating against a tariff that changed mid-period

The tariff is not a set of current values. It is a set of versions, each with a validity interval, and rating selects the version whose interval contains the event time of the record. That single rule handles the mid-month change, the retry that reprices a week later, and the record that arrives after the change but describes usage from before it.

Two consequences follow. First, tariff versions are immutable once any record has been rated against them; a correction creates a new version rather than editing the old one. Second, the rated output stores the identifier of the tariff version that produced it, alongside the rate applied and the inputs used. Without that stored lineage, nobody can answer "why is this charge 40 pence" six weeks later, and a dispute becomes an argument. It is also what makes rerating tractable: when a tariff is corrected retroactively, you select the records rated against the superseded version, reverse them as explicit adjustments, and rate them again against the new one, so the customer can see what changed rather than watching a number move.

Prepaid is a different problem, not a faster one

The answer that sounds right and misses the point is "postpaid rates in batch, prepaid rates in real time." Latency is a symptom. The structural difference is that prepaid must decide before the usage is allowed whether the subscriber can afford it, and that means reserving quota against a balance that is live, shared and mutable.

The flow is a credit-control conversation. Before a session starts, the charging function asks the balance manager for units; the balance manager decrements the available balance and returns a granted quota with a validity time. The network meters against that quota and reports back as it is consumed, requesting more. At the end it reports actual usage, and the unused part of the last reservation is returned.

Session start   -> request units for subscriber S
                <- grant 10 MB, valid 300s   (balance: 100 -> reserved 10, available 90)
Quota exhausted -> report 10 MB used, request more
                <- grant 10 MB               (used 10 charged, reserved 10, available 80)
Session end     -> report 4 MB used of the second grant
                <- 6 MB returned             (charged 14, available 86)

Everything hard about prepaid lives in that sketch. The balance is a contended resource, and a single subscriber can have several concurrent sessions — a voice call, a data session, a tethered device — each wanting a slice of the same money, so reservations must be serialised per subscriber rather than per session, which makes the subscriber the natural partition key and makes hot subscribers a real operational concern. Reservations must carry a validity time and be reclaimable, because the node holding one will sometimes die without ever reporting, and unreclaimed reservations look exactly like a customer whose credit has vanished. Grant sizing is a genuine trade-off: large grants reduce signalling but increase how much a subscriber can overspend if a node fails mid-session, and small grants do the reverse while multiplying traffic to the balance store.

Most pointedly, prepaid has no comfortable answer to a partition. Postpaid can always rate later, so if a component is unreachable you buffer records and catch up, and the only cost is a delay nobody sees. Prepaid must choose in the moment between refusing service to a customer who has credit and granting service to one who may not, and that choice is a commercial policy — often a small emergency grant with a strict cap — rather than something an architecture can make disappear. Saying that out loud is what distinguishes a candidate who has run a charging system from one who has read about one.