You have read access to the customer's production database and there is no staging environment. How do you develop and test without putting their system at risk?
Pull a bounded, representative extract into an environment you control and develop against that, keeping production for read-only verification under agreed limits, and treat every write path as something proven against a restored copy before it runs anywhere near live.
What the interviewer is scoring
- Whether the candidate moves development off production onto a controlled copy rather than working carefully in place
- Does the candidate consider the cost their read queries impose, including locks, replication lag and plan cache effects
- That they raise data protection and residency before copying customer data anywhere
- Whether the extract is chosen for representativeness rather than convenience
- Does the candidate design write paths to be reversible and rehearsed rather than merely reviewed
Answer
Short answer
Pull a bounded, representative extract into an environment you control and develop against that, keeping production for read-only verification under agreed limits, and treat every write path as something proven against a restored copy before it runs anywhere near live.
Careful is not a strategy
The instinct when handed production credentials and no lower environment is to be careful: add LIMIT, avoid writes, run the big queries at night. That is necessary and it is nowhere near sufficient, because it makes safety a property of your attention rather than of the setup. Over ten weeks you will run several thousand queries, some of them at the end of a long day, and one of them will be the join you thought was filtered. A design that depends on never making that mistake is a design with no margin.
The correct move is structural: get a copy of enough data that development happens somewhere you can break things, and reduce production to a small number of deliberate, bounded, mostly read-only interactions. Everything else in the answer follows from that separation, and an interviewer is largely listening for whether you reach for it or whether you propose a discipline of caution.
Get an extract, and choose it for representativeness
The extract is the piece candidates usually get wrong, because the easy version is a LIMIT 10000 off the head of each table. That gives you the oldest rows, or whatever the storage engine happens to return, and it systematically excludes the cases that will break your code. What you want instead is a slice chosen to contain the awkward shapes: a full recent period rather than a random sample so that referential integrity holds within it, plus deliberately included examples of every category, source system and status value you found during profiling, plus the outliers — the largest claim, the row with the null everyone said was impossible, the record type that appears twice a year.
Pull it with the relationships intact. Take a set of parent keys first, then pull children by those keys, so that the extract is internally consistent and your joins behave as they will in production.
-- Step 1: pick the driving population, including deliberate awkward cases.
CREATE TEMP TABLE sample_keys AS
SELECT policy_no FROM claims.policy
WHERE created_at >= DATE '2026-05-01'
UNION
SELECT policy_no FROM claims.policy
WHERE policy_status NOT IN ('ACTIVE', 'A') -- keep the rare statuses
UNION
SELECT policy_no FROM claims.policy
ORDER BY total_value DESC LIMIT 50; -- and the extreme rows
-- Step 2: pull children by those keys so the extract stays joinable.
SELECT c.* FROM claims.claim c
JOIN sample_keys k ON k.policy_no = c.policy_ref;
Size it so that a full run of your pipeline takes seconds rather than minutes, because the value of a local copy is the loop speed, and an extract too large to iterate against quickly has lost most of its point.
Settle the data protection question before you copy anything
Copying customer data to a laptop or to your own company's cloud account is a legal and contractual matter, not a technical convenience, and asking afterwards is not available to you. Establish before the first extract what the data contains, what the engagement contract permits, whether there is a residency constraint, and who on the customer's side is entitled to approve it. If the data includes personal information, the default answer should be to pseudonymise on the way out — replace names, addresses and identifiers with stable surrogates that preserve the join behaviour and the cardinality without carrying the content.
Note what pseudonymisation costs you, because you have to keep it in mind while debugging: you can no longer eyeball a record and recognise it, and you cannot reconcile a specific case with the business by name. That is usually an acceptable trade, and where it is not, the answer is to work on the real data inside their environment rather than to take unmasked data out of it.
Reduce production to a small number of deliberate interactions
With development moved off production, what remains against live should be short, agreed and observable. Verification queries that confirm your local conclusions still hold at full scale. A profiling pass, once, in an agreed window. The eventual production run. Each of those deserves the same preparation: know roughly how much data it touches, run it with a bounded date range first, and agree with whoever operates the database when and how you will do it.
The technical hazards worth naming, because they distinguish someone who has worked in this position from someone who has not, are that a long-running read on a busy system can hold shared resources and lag replication; that an unbounded scan can evict useful pages from the buffer cache and slow everything else for minutes after your query has finished; and that reading from a replica is safer for the primary but gives you data that is behind by an unknown amount, which will make your reconciliation figures disagree for reasons that have nothing to do with your logic.
Write paths are a different category of risk
Reads have a cost. Writes have a blast radius, and no amount of local testing removes the fact that the first execution against their production system is the first execution. Three things make it survivable. The write must be reversible, which usually means writing to your own schema or table rather than into theirs, and where you must touch theirs, capturing the prior state first. It must be rehearsed against a restored copy of production rather than against your curated extract, because the difference between the two is exactly where the surprise lives. And it must be run with their operator present, in a window agreed in advance, with a stated stopping condition.
Idempotency deserves particular attention on an embedded engagement, because the person who will re-run your job after a failure is not you, and the safest system to inherit is one where running it twice is indistinguishable from running it once. Design for the re-run before you design for the happy path.
A nightly restore is the thing to ask for first
If the customer's team takes backups, and they almost certainly do, then a restore of last night's backup into a separate instance is a staging environment in everything but name, and it is far cheaper for them to provide than a properly maintained lower environment. It gives you real data at real volume with real distributions, inside their network, where you can break anything. It is the single most valuable ask in your first week and it is frequently granted, because the DBA who would refuse you production write access is often quite happy to hand you a restore.
Be clear about what it does not give you. It is stale by up to a day, so it cannot be used to reconcile a live figure. It may lack the production configuration, the volume of concurrent load and any external integrations, so performance conclusions drawn there are indicative rather than reliable. And unless the restore is repeated, it drifts, which matters over a twelve-week engagement where the schema may change under you.
The failure mode: a system that only ever ran in one place
The deeper hazard in this situation is not a damaged production database, which most engineers avoid. It is building something that has only ever been executed by you, from your machine, against connections you configured by hand, with credentials in your environment. It works, you demonstrate it, and it cannot be run by anyone else because the environment it depends on is your laptop. The absence of a staging environment quietly encourages this, because there was never a second place the code had to work.
Guard against it by making the deployment target part of the work from early on, even when it is unglamorous: the configuration lives in a file in their repository, the credentials come from wherever their other systems get credentials, and someone on their team runs it end to end well before the handover. A pipeline that has executed in two places is a pipeline; one that has executed in one place is a demonstration.
Never make production your development environment. Get a representative extract and a restored copy, keep live access to a few bounded and agreed interactions, and make sure the thing you build has run somewhere other than your own machine.
© 2026 Preptima. Originally published at preptima.com.
Likely follow-ups
- The customer refuses to let any data leave their network. How does your approach change?
- What would you insist on before running the first write against their production database?
- How do you keep a local extract useful as production data drifts over a twelve-week engagement?
- Their DBA offers you a nightly restore of last night's backup. What does that solve and what does it not?
Related questions
- The customer's schema has no documentation and the person who designed it has left. How do you work out what the tables actually mean?hardAlso on customer-data-integration6 min
- Go gives you no project layout and no dependency-injection container. How do you structure a service so it stays testable as it grows?mediumAlso on testing5 min
- The business told you a field is always populated, and in production it is null for a third of the rows. How do you work out what is going on and what do you do about it?hardAlso on customer-data-integration7 min
- How do you test a service that calls a third-party API, without calling it?mediumAlso on testing4 min