Top 3 Cost Interview Questions
Browse 3 real-world technical and behavioral interview questions about Cost. Review scenarios, edge cases, and architectural best practices.
Design an autoscaling scheme for a self-hosted CI runner fleet. What are you actually trading off, and how do you decide the numbers?
Runner autoscaling trades idle compute cost against queue time, and the two levers that decide the trade are a warm pool sized to your usual concurrency and a scale-down delay long enough to absorb the next job without a fresh cold start.
Models take a million tokens of context now. Does that remove the need for retrieval?
No. Cost and latency scale with what you send, accuracy over a long context is uneven rather than flat, and a corpus has no upper bound while a window does. Long context did change the job - it absorbs whole documents and forgives loose recall - but it replaces chunking, not retrieval.
When is an agent the wrong shape for a problem?
Whenever you can draw the flowchart. A fixed pipeline with a model at the two steps needing judgement is cheaper, testable and debuggable, and per-step error compounds across a planning loop. Reach for an agent when the tool sequence is open-ended and the environment verifies the work.