hardScenarioCase StudyMidSeniorStaffLead
Your inference bill tripled last month and traffic was flat. How do you find out why?
6 min readUpdated 2026-08-15
Answer
Short answer
Flat traffic and a tripled bill means cost per request tripled, so hunt what changed per request: a larger model version, more feature reads per prediction, retries multiplying calls, or warm capacity left running.
Was this answer helpful?
Likely follow-ups
- The bill rose as a step on one day. What does that rule in and out compared with a gradual rise?
- How would you tell a retry storm apart from a genuine traffic increase in the cost data?
- What would you put in a request log so that cost per model version is queryable?
- Which of your findings would you fix with a guardrail rather than a code change?
Related questions
- Users say the LLM feature in your product feels slow. How do you work out what to fix?mediumAlso on inference and observability5 min
- How do you decide whether to use a managed service or self-host a component, and which cloud costs catch teams out?hardAlso on cost-optimisation6 min
- Your largest customer costs more to serve than they pay you. What do you do about that architecturally?hardAlso on cost-attribution6 min
- You serve millions of prompts an hour and the LLM bill doubled. Where does the money go, and what do you cut?hardAlso on cost-optimisation5 min