The loop for building LLM features in a product: prompting and structured output, retrieval, agents and tool use, evaluation, and the cost and safety questions that follow.
Enforce step, token, wall-clock and monetary budgets in the orchestrator rather than in the prompt, detect repetition and lack of progress as separate conditions, and require an approval bound to specific arguments before any irreversible action. Exhausting a budget should hand back partial work, not vanish.
Whenever you can draw the flowchart. A fixed pipeline with a model at the two steps needing judgement is cheaper, testable and debuggable, and per-step error compounds across a planning loop. Reach for an agent when the tool sequence is open-ended and the environment verifies the work.
Revoke once in the authorisation store, then propagate to the index ACLs, every cache holding text or results, any derived summaries and embeddings, and the conversation transcripts still carrying passages. Answers already delivered cannot be recalled, so the guarantee you offer is a bounded window rather than erasure.
Say yes if it is true, give a stage and a rough timeline without naming companies, and use it to set a decision date rather than to imply a bidding war. Inventing a process you do not have is the one answer that can end badly.
Ask the index directly whether the old text is still there, because that single check splits the problem into change detection, queue or write failure, a retire step that left duplicate chunks, or a downstream cache. Then fix the reason you learned about it from a user rather than an alert.
Source the questions from real users and from randomly sampled documents rather than from the team, label each with a quotable answer string anchored to a document rather than a chunk id so the set survives re-chunking, and hold part of it back so you are not tuning against your own measurement.