Loading...
Loading...
Browse 9 real-world technical and behavioral interview questions about Model governance. Review scenarios, edge cases, and architectural best practices.
It cost a customer, a complaint, and a training example you have mislabelled as clean. The fix is to keep the score ranking an investigation queue rather than gating a payment, to time-box every referral with an interim payment path, and to record why a referral was cleared instead of merely that it was.
Gate on absolute thresholds, on a comparison against the incumbent evaluated on identical data, on per-slice regression checks so an aggregate gain cannot hide a subgroup loss, and on operational limits like latency and artefact size. Keep a human approval where a wrong decision is legally consequential.
A statement of intended and out-of-scope use, the provenance and lawful basis of the training data, evaluation results broken down by segment rather than only in aggregate, the known limitations and failure modes, and a signed approval record naming who accepted the risk against which evidence.
Establish who needs the explanation and what they will do with it, because a global account of model behaviour and a per-decision reason serve different needs. Attribution methods give you defensible local reasons within limits, but a post-hoc explanation is a second model of the first, not its reasoning.
Credit decisioning combines policy rules, affordability checks and score cut-offs, then stores the exact inputs, policy version, model version and adverse-action reason so a loan decline is explainable later.
It has to be available at quote time for a known share of risks, obtainable as it stood historically so a backtest means anything, defensible as a factor rather than a proxy, and stable enough that the vendor cannot change your filed rate for you. Everything else is a modelling detail.
The prediction record must name the model version, the exact feature vector as served with its values, the identifier of the code that computed those features, the decision and any thresholds or overrides applied. Reconstruction after the fact is not possible, because feature pipelines and reference data have moved on.
Decide which groups you are testing across and how you will obtain that attribute, choose one fairness definition and justify it - because the common definitions are mathematically incompatible and cannot all be satisfied - then report performance and error rates per group with sample sizes and intervals attached. It also connects bias testing to the point an interviewer is testing.
It is the system of record for which artefact is authoritative and why. It binds each version to the data snapshot, code commit and evaluation that produced it, records stage transitions and who approved them, and gives serving a stable identifier so any past prediction can be traced to the exact model that made it.