Top 5 Data pipelines Interview Questions
Browse 5 real-world technical and behavioral interview questions about Data pipelines. Review scenarios, edge cases, and architectural best practices.
Events for a daily revenue table arrive up to three days late. How do you design the pipeline and the backfill?
Land raw events unchanged by arrival date, then rebuild each revenue day from that raw layer by event date. Each run overwrites only the days its new arrivals touched, a day is provisional until the three-day window closes, and the backfill is the same job run over a date range, never a separate script.
A transform has been writing wrong revenue figures for three days and six downstream tables have consumed it. How do you backfill the corrected data without double-counting anything?
Make the write idempotent before you make it correct: partition-level replace rather than append, so rerunning a day produces the same result whether it runs once or five times. Then work out how far the contamination spread, because a downstream table that accumulated rather than recomputed will not fix itself when the source is repaired.
A DAG that finished in an hour now takes nine and blocks everything behind it. How do you find out where the time went?
Sum the task durations and compare that with the wall-clock span, because nine hours of waiting and nine hours of work are different problems. If the sum is still an hour, you are queueing on a bounded worker pool, usually behind sensors holding slots; if a task genuinely grew, its own duration history over the last thirty runs tells you whether it stepped or sloped.
An upstream team renames a field in the events you consume and nobody tells you. What should your pipeline have done?
Validate every record against a declared schema at the ingestion boundary, quarantine what fails instead of dropping it, treat additive changes as safe and renames as breaking, and alert on a field that has gone all-null rather than waiting for a dashboard to look wrong.
An event for 23:58 yesterday arrives at 00:20 today, after you have already published yesterday's totals. What should your pipeline do?
Decide it deliberately, because there is no correct default: you can hold the window open longer, accept the record and restate the figure, or drop it and report how much you dropped. What you must not do is let arrival order silently decide, which is what a pipeline keyed on ingestion time does.