Skip to content
QSWEQB
hardScenarioDesignSeniorStaffLead

The electronic record will be down for a four-hour upgrade, and it also fails without warning. How does a ward keep working?

Plan for four modes rather than one: a continuously fed read-only viewer on independent infrastructure, printed ward-level essentials, paper ordering with pre-allocated temporary identifiers, and a recovery process that backloads events at their original times and reviews everything that queued during the window.

5 min readUpdated 2026-07-26

What the interviewer is scoring

  • Does the candidate distinguish planned, unplanned, partial and degraded outages instead of designing for one scenario
  • Whether the downtime viewer is placed on genuinely independent infrastructure and authentication, and its contents scoped deliberately
  • That backloading records the original time of the clinical event rather than the time of data entry
  • Whether queued inbound results are treated as an active safety risk at recovery rather than as a backlog
  • Does the answer treat identity created during downtime as a merge problem that has to be worked afterwards

Answer

Four modes, not two

Treating this as up-or-down is the first error, because the modes have different plans and the awkward ones are in the middle.

A planned outage gives you notice, so activity can be scheduled around it, staff briefed, and paper forms printed in advance. An unplanned outage gives you none, which means everything the plan relies on must already be in place and already tested rather than assembled when it happens. A partial outage is the one that causes the most confusion: the record is up but the laboratory interface is down, or the medication administration devices cannot reach the server, or the record is available read-only. Staff can see a working system, so they do not believe there is a problem, and the failure is discovered when something that should have arrived did not. And degradation — everything works but takes twenty seconds — is worse than a clean outage in practice, because there is no moment at which anyone declares anything, and clinicians silently start skipping steps that have become unaffordable.

So the plan needs a declaration process: who is authorised to declare downtime, on what evidence, how the wards are told through a channel that is not the system that is down, and equally who declares recovery. Degradation needs a defined threshold that triggers the same process, otherwise it never does.

The read-only viewer earns its cost or it is theatre

The core continuity control is a read-only copy of the clinically essential record, continuously fed, hosted so that it does not share the failure domain of the record it copies: separate infrastructure, separate network path, separate authentication, and a location staff can reach from the ward. If it authenticates against the same directory that is down, it is not a continuity control.

Scope it deliberately rather than mirroring everything. What a clinician needs to keep a patient safe for four hours is a small set: active problems, current medications with the last administration time, allergies and adverse reactions, recent results, code status and treatment escalation decisions, the care plan, and enough identity to be certain which patient this is. A viewer holding the entire chart is slower to build, slower to use under pressure, and no safer. Behind that sits the last resort, which is paper: a ward-level pack regenerated on a schedule frequent enough that it is worth reading, because a printed summary from last Tuesday will be ignored and should be.

Paper capture and the recovery that follows

During the window, ordering, prescribing and documentation move to pre-printed forms designed for the purpose, not to blank paper. The forms matter because they determine whether recovery is possible: they must capture the patient identifiers, the ordering clinician, and above all the actual time of each event.

Recovery is where the clinical risk concentrates, and it is the part that gets three lines in most continuity documents.

sequenceDiagram
    participant W as Ward staff
    participant P as Paper forms
    participant E as EHR
    participant I as Integration engine
    participant R as Recovery queue
    Note over W,I: Downtime window
    W->>P: Orders, doses, notes with real clock times
    I->>I: Inbound results queue, store and forward
    Note over W,R: Recovery
    W->>E: Backload with original event times
    I->>E: Release queued results in order
    E->>R: Unmatched items and duplicate candidates
    Note over E,R: Criticals from the window need a targeted review, not the normal inbox

The edge to look at is the last step: releasing the queue restores the data but not the attention, because the acknowledgement obligations for results that arrived during the window were never surfaced to anyone.

Two rules make backloading safe. Every entry records the time the event happened, not the time it was typed, with the entry time recorded separately as metadata. A medication administration record that says a dose was given at recovery time when it was given three hours earlier is a false clinical record that will be read by the next nurse deciding whether a dose is due, and it is a straightforward route to a double dose. And every backloaded item is attributed to the clinician who did the thing, not to the person entering it, with the transcription itself attributable.

Duplication is the other predictable failure. An order written on paper and also entered by a second person on recovery produces two orders for one intention, so recovery needs an explicit reconciliation step: a named owner per ward, a checklist of forms accounted for, and a system-side duplicate-candidate queue matching on patient, item and time window. Nobody should be relying on individual vigilance for this while also catching up on four hours of work.

Identity created during the window

Registration is usually part of what is down, so patients arriving during the window cannot be given a real record number. The workable answer is a pre-allocated block of temporary identifiers held physically on each unit, issued sequentially with the allocation logged, so two units cannot issue the same one and every temporary identifier can be accounted for afterwards.

Afterwards, each temporary identity has to be resolved to the correct permanent record, and that is a patient-matching exercise with all the usual asymmetry: merging two people is far worse than leaving a duplicate for a day. So temporary identities go into the same adjudication process as any other candidate link, with human review, and anything ordered, administered or resulted against the temporary identifier has to move with it — which is only tractable if the merge was designed to be reversible in the first place.

What has to be practised

The controls that fail are the ones nobody has used. A viewer that has never been failed over, a form nobody has filled in, a printed pack the ward does not know where to find, a phone tree with a number that changed. The counter to that is a drill: switch to downtime procedures deliberately, on a schedule, in a real clinical area, and measure what people could not do. Expect it to be unpopular and to find something every time.

Two adjacent controls deserve naming because they carry so much of the load. First, the integration engine's durability is what makes recovery possible at all — inbound results, discharge notifications and orders from other systems accumulate rather than being lost, and selective replay is what releases them in order once the record is back. A continuity plan that does not know whether its interfaces retry and for how long is guessing. Second, prescribing loses its automated checking during downtime, so the paper process needs a compensating human step, typically a pharmacist review of everything written on paper before or immediately after recovery. The checks did not become unnecessary because the computer was off.

Downtime plans are usually written for the outage and not for the recovery, which is where the harm actually occurs. If backloaded events carry the time they were typed rather than the time they happened, and if the results that queued during the window land in an inbox nobody was told to look at, the system came back and the record did not.

Likely follow-ups

  • Two clinicians independently enter the same paper order during recovery. How does your process catch it?
  • What would you put in the printed ward pack, and how often does it have to be regenerated to be worth having?
  • The record is up but responding in twenty seconds. Who is allowed to declare that a downtime, and on what evidence?
  • How do you test a downtime plan without harming patients or exhausting the staff you are testing it on?

Related questions

ehr-downtimebusiness-continuityclinical-safetyrecoveryintegration