Most predictive maintenance projects never reach production. Why, and what would you do differently?

The failure is upstream of the model

Ask what went wrong on a stalled predictive maintenance project and you will rarely hear that the architecture was inadequate. You hear that the model looked excellent in the notebook and produced alerts nobody trusted, or that six months of work went into a pipeline and the honest conclusion was that the data could not support the question. The modelling is the least uncertain part of this work, and the amount of attention it receives is inversely proportional to the amount of risk it carries.

Everything below is really one problem stated five ways: you are trying to learn a rare event from a record that was never kept in order to describe it.

Nobody wrote down when the failure started

The label you want is the onset of a degradation that led to a failure of a particular mode. What exists is a maintenance management system containing work orders. A work order tells you, at best, when someone closed a repair. That is days or weeks after onset, it is recorded by a technician whose priority was fixing the machine, and the description is free text — "bearing noise, replaced", "same as last time", "no fault found".

Several distinct problems hide in that. The date is the wrong date, so any model trained against it is being taught that the machine was healthy through the entire period you actually care about. The failure mode is usually absent or inferable only from a parts list, so you are pooling mechanically unrelated events into one class and asking a model to find a signature they do not share. Planned maintenance and reactive repair are frequently indistinguishable in the record, and a planned replacement is not a failure. And a well-run plant deliberately does not run assets to failure, so the events you would most like to learn from are the ones the maintenance regime exists to prevent.

The work that follows is label archaeology, and it is not glamorous: sitting with the reliability engineer and reconstructing, event by event, when each failure plausibly began and what mode it was. On a modest asset population that is dozens of events, not thousands, and doing it properly takes weeks. Budget for it explicitly, because a project that treats labelling as a preprocessing step has already lost.

Define the horizon before you define anything else

A prediction with no useful lead time is not useful. Before any modelling, get three numbers from operations. How much warning does acting require — sourcing a part, scheduling a technician, waiting for a window — which sets your minimum lead time. How far ahead is a signal plausibly present, which sets the maximum horizon. And how many alerts a week can a planner triage, which sets the operating point far more tightly than any statistical criterion.

That last number is the one engineers skip and the one that determines whether the system survives. If a planner can investigate five alerts a week and your model produces forty, precision does not need to improve — the alert volume does, and you should be tuning the threshold to that budget and reporting recall at it.

The positive class is nearly empty, so most metrics lie

With eleven failures in three years of readings, a classifier that always predicts "healthy" is well over 99% accurate. Everyone knows to reject accuracy. Fewer reject ROC-AUC, which is also misleading here, because with a vast negative class a large absolute number of false positives barely moves the false-positive rate. Precision-recall is the right frame, and better still is expressing the trade-off in the plant's own terms: how many false alerts per genuine catch, against the cost of an unplanned stop versus the cost of an unnecessary inspection.

The other consequence of a tiny positive class is that your confidence interval on any metric is enormous. Eleven events split into train and test leaves a handful of positives to score against, so the difference between 0.6 and 0.8 recall is one or two events. Report it as such, because claiming a precise figure from a handful of events is how projects get committed to targets that were noise.

Feature windows, and the four ways the outcome gets in

The usual construction is a sliding window: features computed over a window ending at time t, and a label saying whether a failure occurs in the interval from t plus the lead time to t plus the horizon. Stating that construction explicitly is worth doing in an interview, because most leakage is a violation of it that is invisible once the data is in a matrix.

Leakage arrives by four routes and it is worth naming the mechanism rather than the category. First, splitting rows at random. Consecutive windows from the same failure event overlap heavily, so a random split puts near-identical rows on both sides of the line and your test score measures memorisation. Split by time, and group by failure event and by asset. Second, statistics fitted on the whole dataset — a scaler, a normalisation constant, a rolling mean that is centred rather than trailing — each of which lets information from after t into the features at t. Third, readings from during or after the repair. The machine is stopped, then it runs differently because it has a new bearing, and both are trivially separable from normal operation; a model that finds them has learned to detect maintenance, not to predict it. Fourth, an operational artefact that encodes the outcome: a diagnostic sensor switched on only once someone suspected a problem, a sampling rate that changed when the asset went on watch, an asset that appears in the data only because it failed.

The tell for all of them is the same, and it is the most reliable heuristic in this domain: an unexpectedly strong first result is evidence of leakage until proven otherwise. Genuine early-degradation signal is faint. If your first model separates the classes cleanly, go and find out what it is actually reading.

The baseline you owe before any model

No model result means anything without something to compare it to, and "no prediction at all" is not the honest comparator, because the plant is not doing nothing today. Establish, in this order:

• The existing regime. There are almost certainly threshold alarms on vibration, temperature or current, and condition monitoring on the critical assets. Measure their precision and recall against your reconstructed labels. This is frequently a shock, in both directions.
• An age-based baseline. Time since last intervention, or a reliability distribution fitted per asset class, using no sensor data at all. Preventive maintenance intervals already embody this, and it is a surprisingly strong competitor.
• A single-feature rule. A trailing mean or RMS of the one signal the reliability engineer says matters, with a threshold tuned on the training period. If a gradient-boosted model on 300 features cannot beat this, the extra 299 features are not carrying signal.

Commit to those numbers before modelling, in writing. Their real function is political: they are what stops a project being judged against an imagined perfect system, and they are what lets you say honestly at week six that condition monitoring already does this well enough — which is the correct outcome more often than the field admits.

What the first six weeks should contain

No model. Data provenance for every tag you intend to use, including which are change-based and at what deadband. Clock alignment checked across controllers, historian and the maintenance system, because a few seconds of skew ruins a join and nobody notices. Tag semantics confirmed against something other than the tag name. The label reconstruction described above. The baselines measured. A written statement of horizon, lead time and alert budget, agreed by the person who will receive the alerts. Then modelling, on a small feature set you can explain, evaluated on held-out failure events by time.