A dashboard has been showing the same temperature for two days and nobody noticed. Walk the path and tell me what would have caught it.

Silence and stability are the same bytes

The whole failure rests on one ambiguity. Industrial data collection is dominated by reporting on change, because a tank temperature sampled every second is almost entirely repetition. But the moment you stop sending a value that has not moved, you have made "nothing arrived" and "nothing changed" indistinguishable at the receiver. Two days of an unchanged reading is exactly what a healthy slow signal looks like, which is why nobody noticed.

That ambiguity is not a bug in any one product. It is the direct consequence of an efficiency everyone wants, and it means liveness is a property you have to transmit on purpose. Nothing in the shape of the data will supply it for you.

Walk the hops, because each freezes differently

The temperature on that dashboard passed through several components, and it is worth being able to enumerate them, because the diagnosis differs at every one and so does the fix.

The instrument itself can fail to a plausible value. A thermocouple or transmitter that loses its input may hold its last reading or settle on a fixed output, and from the network's perspective that is a perfectly healthy device reporting a perfectly plausible number. Nothing above it can distinguish this from a stable process; only a comparison against something else physically related to it can.

The controller can stop updating the register. The logic that writes the value lives in a program, and a routine that is no longer being called — a mode change, a disabled section, a first-scan initialisation that never got past initialisation — leaves the last value sitting in memory. Every read of that register succeeds and returns it.

The collector's subscription can end without the collector noticing. Where data comes from a subscription rather than a poll, the subscription is state on the server, and state on the server can be dropped: a session timing out, a resource limit reached, a server restart after which the client's subscription no longer exists. If the client is not checking that it is still receiving what it asked for, it simply stops receiving and carries on.

Configuration can be the cause with nothing broken at all. A deadband widened during a tuning exercise to reduce load will genuinely suppress a signal whose movement is smaller than the deadband, indefinitely and correctly.

And the last hop can be the culprit: a broker holding a retained value, a cache in the API, or a dashboard whose query returns the newest point without regard to how old it is. This is the cruellest one, because the freeze is downstream of a perfectly healthy data path.

What each protocol lets you assert

The reason to know the protocols is that each of them expresses liveness at a specific layer, and only that layer.

OPC UA is the richest here, because every value carries a status code and a source timestamp alongside the number. That is the mechanism that distinguishes a good reading from a stale, uncertain or failed one, and a client that discards the status and stores only the value has thrown away the answer to this question at the first hop. Subscriptions also have a keep-alive behaviour, so a server with nothing to report still tells the client it is alive, which is what turns a dead subscription into a detectable event rather than an absence.

MQTT gives you liveness about the connection, not about the value. The keep-alive between client and broker detects a dead client, and the will message the broker publishes on an ungraceful disconnect is how subscribers learn a device has gone away without polling it. What MQTT does not give you is any statement about payload freshness, and its retained-message feature actively works against you: a new subscriber is served the last published value with no indication of its age, so a dashboard restarted during an outage comes up showing pre-outage numbers that look current. Sparkplug is the usual remedy, because its birth and death certificates make a node's presence explicit and its sequence numbering lets a subscriber detect that it missed messages rather than assuming continuity.

Modbus offers nothing at all. A read either succeeds or times out, and a successful read of a register the controller stopped writing to returns success with a stale number. Device reachability is all you can establish; value freshness has to be constructed above the protocol.

Carry a counter, not a timestamp of arrival

The measure most teams reach for is a last-received check at the consumer, and it is weaker than it looks. Anything on the path that republishes a cached value satisfies it — a gateway that emits its current snapshot on a timer, a retained message replayed to a reconnecting subscriber, a poll loop that reads and forwards a frozen register. The check goes green while the value is two days old.

The robust signal comes from as far upstream as you can put it: a value that changes because the source is executing. A counter in the controller that increments every scan, or a heartbeat tag toggling on a fixed cadence, is trivial to add and cannot be faked by anything downstream, because a stale copy of a counter is visibly not advancing. Monitor that counter and you are monitoring the source, not the pipe.

Where you cannot get a source-side heartbeat, use the source timestamp rather than the arrival timestamp, and store both. Two stored times let you measure the lag on every hop and turn "the data looks odd" into a number you can chart.

Thresholds have to be per tag

The staleness alert itself is where a well-intentioned implementation creates the alarm flood that gets it switched off. Tags have wildly different natural update rates. On change-based collection, an ambient temperature might legitimately go an hour without moving while a motor current updates several times a second, so a single global threshold either misses the slow tags or drowns you in alerts about them.

Derive the threshold from each tag's own history. Take the distribution of intervals between consecutive points over a representative period, and set the expected maximum gap generously above its high percentile — if a tag's 99th-percentile gap is forty minutes, an alert at two hours is meaningful and an alert at five minutes is noise. Recompute periodically, since the distribution changes when the deadband or the process does.

-- Per-tag expected gap, learned from the tag's own inter-arrival history,
-- instead of one global staleness threshold for every series.
WITH gaps AS (
  SELECT tag_id,
         source_time - LAG(source_time) OVER (PARTITION BY tag_id
                                              ORDER BY source_time) AS gap
  FROM readings
  WHERE source_time > now() - INTERVAL '30 days'
)
SELECT tag_id,
       percentile_cont(0.99) WITHIN GROUP (ORDER BY gap) AS p99_gap
FROM gaps
WHERE gap IS NOT NULL
GROUP BY tag_id;

Two smaller habits close the gap. Alert on the disappearance of a tag from the expected set, not only on individual tags going quiet, because a gateway that resubscribes to most of its tags after a restart produces a silent partial loss that no per-tag rule fires on if the tag was never seen again. And make staleness visible where the number is read: a dashboard that greys the value, shows its age, or refuses to render it past the expected gap converts an invisible two-day failure into something an operator reports in the first hour.