Top 6 Cost optimisation Interview Questions
Browse 6 real-world technical and behavioral interview questions about Cost optimisation. Review scenarios, edge cases, and architectural best practices.
You serve millions of prompts an hour and the LLM bill doubled. Where does the money go, and what do you cut?
Split LLM cost by feature and by line: uncached input, cached input, output, retries and hidden callers. Find which line moved before cutting anything. A collapsed prompt-cache hit rate or longer replies can double a bill with no change in traffic; fix those before you downgrade a model.
How do you decide whether to use a managed service or self-host a component, and which cloud costs catch teams out?
Managed service vs self-hosted decisions compare operational load, control, lock-in, availability and real cloud cost. The hidden costs are usually egress, cross-zone traffic, idle capacity and the team needed to run the component well. It also connects managed services to the point an interviewer is testing.
How would you cut the cost of a GPU training fleet without slowing the team down?
Measure utilisation first, because most fleets pay for idle accelerators rather than for too few. Then right-size to whatever each job is bound by, move interruptible work onto spot capacity behind tested checkpointing, and pool the fleet behind one queue instead of reserving per team.
How do you architect an automated lifecycle management system to identify, quarantine, and terminate orphaned cloud resources without accidentally destroying critical infrastructure?
Zombie cloud infrastructure should be detected with billing, utilization, ownership, and access signals, then quarantined before deletion so FinOps automation saves cost without destroying critical standby resources. Use this FINOPS answer to show the decision, trade-off, and evidence rather than a memorised definition. It also connects cloud infrastructure to the point an interviewer is testing.
Your inference bill tripled last month and traffic was flat. How do you find out why?
Flat traffic and a tripled bill means cost per request tripled, so hunt what changed per request: a larger model version, more feature reads per prediction, retries multiplying calls, or warm capacity left running. Per-request cost attribution is what makes this answerable.
Do you use one large model, or a smaller model with retrieval? How do you decide?
Decide from the task. For grounded work over your own corpus, quality is set by what reaches the context, so retrieval dominates model size and a smaller model usually suffices. Keep the larger model for long multi-step reasoning, and treat routing as a second system to evaluate and monitor.