Your predictive maintenance system is live and the maintenance team has stopped acting on its alerts. How do you get that back?

You are almost certainly not measuring the thing you need

Ask what happened to the last hundred alerts and the usual answer is a count of alerts sent. That is a record of your system's activity, not of its effect, and it makes every subsequent conversation unfalsifiable — you cannot demonstrate the system works, and the maintenance team cannot demonstrate it does not, so the argument is settled by whoever is more senior.

So the first work is not modelling. It is closing the loop: every alert gets an identity, an owner, a state, and a recorded disposition when it closes. Until that exists you are tuning blind, and worse, you are asking the people who have stopped trusting you to accept a claim you cannot support.

The disposition categories have to be honest rather than convenient. A confirmed fault of the predicted mode. A confirmed fault of a different mode, which is a partial success worth counting separately because it tells you your features generalise further than your labels do. An explainable non-fault, where something genuinely changed in the signal but the cause was a different duty point, a cleaning cycle or a start-up transient. Inspected and nothing found. Not investigated. And the one that only resolves later: nothing found at the time, and the asset failed within the horizon anyway.

Not investigated is the number that decides everything

That last category is the one to compute first, because it dominates the arithmetic and it is the direct measurement of the problem you were asked about.

Delivered recall — the fraction of real failures the plant avoids because of your system — is not your model's recall. It is your model's recall multiplied by the probability an alert is acted on.

Model recall on held-out failure events        0.80
Alerts per week                                  40
Alerts a planner can triage per week              5
Action rate                                    5 / 40 = 0.125

Delivered recall                       0.80 x 0.125 = 0.10

Ten per cent. And the alerts that get triaged are not a random sample of the forty; they are the ones on assets the planner already worries about, which are the ones a person would have caught anyway. Under those conditions, improving model recall from 0.80 to 0.90 changes delivered recall from 0.10 to 0.11, which is invisible to everyone, while improving the action rate from 0.125 to 0.5 quadruples it with the same model.

That is the calculation to put in front of the steering group, because it reframes the request. You are not asking for time to retrain a model. You are asking to reduce alert volume until every alert delivered is one somebody opens.

No fault found is not a false positive

Before you score anything, resist the reflex that closes every "nothing found" as a false alarm. It corrupts the metric in a specific direction and it removes the incentive to fix the real gap.

An inspection only produces evidence about what it can observe. If the predicted mode is an inner-race bearing defect and the inspection was a walk past with a listen, a null result carries almost no information. If the predicted mode requires the machine to be opened and it was not opened, likewise. The correct label in those cases is unknown, and lumping unknowns in with confirmed negatives makes your precision look worse than it is while hiding the fact that the plant has no way to confirm your predictions even when they are right.

So each alert needs the predicted finding stated up front — which asset, which mode, and what an inspector should expect to see — and the inspection procedure needs to be capable of observing it. That pairing is what turns an alert from an assertion into a falsifiable claim, and it is what lets a null result mean something. It also has a side effect worth having: writing down what you expect to be found is a discipline that quietly kills the alerts you cannot justify.

Retuning the threshold is the wrong instinct

Here is the failure mode that keeps these projects in a loop for years. Trust drops, so the team responds by improving the model: more features, a different algorithm, a tighter threshold. It feels like the responsible engineering response and it addresses none of the binding constraints, because the alert's problem was never its probability estimate.

An alert as usually delivered is a demand for work with no supporting evidence, arriving in a channel nobody owns, addressed to no one in particular, with no way to record what happened. A maintenance planner receiving "Pump 214, high probability of failure, 0.87" is being asked to spend a technician's afternoon on the word of a system they cannot interrogate. Declining that is rational.

Three changes to delivery are usually worth more than any change to the model. Attach the evidence: the specific signal, its trace over a period long enough to see the change, the same asset's own history, and ideally a comparable asset that is behaving normally, so the recipient can form their own judgement rather than accepting yours. Route it into the system that already governs maintenance work, so it becomes an inspection request with an owner, a due date and a closure code, rather than an email or a dashboard tile. And make the confidence legible in units the recipient uses — expected lead time and what it costs to look versus what it costs to be wrong — instead of a probability whose calibration they have no reason to believe.

Narrowing the scope is how you buy trust back

Recovering trust costs recall, and you should be willing to spend it. The reason is asymmetric: an alert that turns out to be nothing costs a little credibility every time, while a missed failure on an asset that was never in scope costs almost none, because you did not claim it.

So the recovery shape is deliberately small. Pick the handful of assets where consequence is highest and where the reliability engineer agrees a signature exists. Suppress everything else outright rather than leaving it firing at low volume, because a trickle of unactioned alerts sustains exactly the impression you are trying to remove. Run at a threshold you would defend to the technician's face, and let the operating point be set by the triage budget rather than by an optimum on a curve. Then publish the outcomes weekly, including the ones that were nothing, and let the record accumulate.

Suppression is also where per-asset baselining earns its place. A single global rule alerts constantly on the two assets whose mounting or duty makes their normal levels different from the rest, and those two assets are the ones the maintenance team will cite when they say the system does not work.

Precision decays on its own

One thing to expect and instrument: precision falls over time with no change to your code. A sensor gets replaced and its mounting differs, so an asset's baseline shifts. A line is re-rated and machines run at a different duty point. A new product introduces a process step the model has never seen. A control loop is retuned. Each of these moves the input distribution while your thresholds stay where they were, and the result is an alert rate creeping up on a subset of assets.

Monitor the alert rate per asset and per mode rather than in aggregate, since aggregate volume hides a single asset that has started firing daily — and a single asset firing daily is enough to re-establish, in the mind of the person receiving them, that the system cries wolf.

Trust is lost in the aggregate and rebuilt one alert at a time, so the only recovery that works is fewer alerts, each carrying its own evidence and each with a recorded outcome.