Available is not the same as hidden
The difference between a fair exercise and a gotcha is not whether information is withheld at the start. It is whether a competent person could reasonably get it.
An answer is hidden when only one phrasing unlocks it, when the interviewer pretends not to know, or when the missing fact is something no one in the real job would think to ask. That kind of interview design measures luck and prior exposure. An answer is available when any sensible question on the subject gets a truthful reply. The candidate's job is to notice that the subject matters, and that is the skill you want to sample.
So the interview design starts with a rule for the interviewer: never refuse, never mislead, never make the candidate guess the exact words.
Write the fact sheet before the first candidate
Pick a brief that sounds complete and is not. Here is one for a senior backend role: "Build a nightly export of customer orders for the finance team."
Then write down what you know and the candidate does not, tiered by whether each fact changes the work.
| Fact | Tier | What it changes |
|---|
| About 40 million order rows per night | Load-bearing | Streaming rather than building the file in memory |
| Finance opens the file in Excel | Load-bearing | An Excel worksheet holds 1,048,576 rows, so one CSV is unusable for them |
| Orders are in several currencies | Load-bearing | Totals cannot be summed across rows without a rate |
| Customer emails must not leave the order system | Load-bearing | An explicit column allowlist, not "select everything" |
| Orders keep arriving during the export | Load-bearing | A consistent snapshot, or a stated cut-off time |
| The team writes Kotlin | Neutral | Nothing about the design |
| Files are named by date | Neutral | Nothing |
Writing this down in advance is the part of the interview design that is easiest to skip, and it does two jobs. It stops interviewers improvising different answers for different candidates, after which their scores can no longer be compared. And it lets the rubric be specific: there are five facts that matter, so you can record which ones surfaced and when.
Open with something like: "The brief is short on purpose. I have more detail, so ask for anything you need and I will answer honestly. If you would rather assume something, tell me the assumption."
Some interviewers worry this gives the game away. It does not. As the interviewer psychology guide argues from the candidate's side, naming the signal does not hand anyone the skill. Knowing that questions are welcome does not tell a candidate that the Excel row limit matters. What the sentence removes is the anxiety that asking will look like weakness, which is noise you did not want in the signal anyway.
Give non-askers a second half
Some strong engineers start building and ask later. If the exercise only rewards questions at minute two, it penalises a working style rather than a skill, and your hiring loop loses people it should have kept. Two mechanisms fix that.
First, let the material surface the facts. If the sample data includes orders in two currencies, a candidate who notices while building has shown something real, if later than one who asked. Record the moment, not just the fact.
Second, add a fixed midpoint disclosure. At about half time, the interviewer volunteers one load-bearing fact the candidate has not found: the first unfound one in an order fixed in advance, so every interviewer makes the same choice. Now every candidate has a second half in which to show what they do with new information.
The rubric then records each load-bearing fact as asked, noticed in the work, disclosed by the interviewer, or never surfaced.
Grading the questions instead of what they changed
This is the failure that makes the exercise useless, and it is easy to fall into. Interviewers enjoy good questions, so they start scoring the questions themselves: how many, how articulate, how senior they sounded. That rewards candidates coached to ask a rehearsed set and then build the same thing regardless.
Score the effect instead. A candidate who asked about volume and then built an in-memory export did not use the answer, and asking earns nothing. A candidate who asked two questions, both load-bearing, and redesigned twice in response scores higher than one who asked ten. The anchor for the top band should read something like: "surfaced most load-bearing facts before committing, and each one visibly changed the design." Volume belongs nowhere on the form. This fits the coverage-matrix approach in designing a senior engineer loop, where each round has a single named signal and its own anchors.
Pilot it before you trust it
Treat the interview design as untested until it has met real people. Run the exercise on two or three colleagues before it touches a hiring decision. If none of them ever asks about the Excel limit, the fact is not available, it is hidden, and you need to change the brief or the sample data so the subject can be noticed. If everyone asks the same three questions in the first minute, the brief is too leading and you are measuring reading, not judgement. Keep the fact sheet versioned, and when a candidate asks a reasonable question it does not cover, give a truthful answer and add it to the sheet for everyone after them.
The candidate should be able to get every answer that matters. What you are grading is whether they noticed it mattered, and whether the work changed when they did.
© 2026 Preptima. Originally published at preptima.com (opens in a new tab).