Top 4 Llmops Interview Questions
Browse 4 real-world technical and behavioral interview questions about Llmops. Review scenarios, edge cases, and architectural best practices.
Design a chat service that streams model output to users and lets them press stop. What has to happen when they do?
Stream tokens over server-sent events or a WebSocket, but treat stop as a server-side command routed to whichever worker owns the generation. That worker aborts the upstream model call, saves the partial reply with a stopped status, and sends a final event so the client and the stored history agree.
Your provider announces the model you depend on retires in six weeks. Walk me through what you do.
Inventory everywhere the model is coupled to behaviour, then use the eval set as your migration harness, shadow the candidate on live traffic to compare without user exposure, and re-measure cost and latency because both shift. Use the migration to delete instructions that only compensated for the old model.
How do you ship a change to a production prompt safely?
Treat the prompt as a versioned artefact under review, run your eval set in CI against a pinned model version, canary on a slice of traffic with the version recorded per request, and keep rollback a config change rather than a deploy. Pinning is what makes a regression attributable to your edit.
What do you instrument and alert on for an LLM feature running in production?
Trace every request with prompt and model versions, token counts and cost, then add sampled groundedness grading, refusal and retry rates, validation failures and implicit user signals. Alert on countable rates, because an averaged quality score moves too little and too late to page anyone.