Why agents cost 1,000x more than chat, and how to add budget enforcement to any agent loop — throttle at 80%, rollback at 100%, circuit-break on retry storms.
Pi has no budget enforcement. An agent without budget controls is a budget fire — a single runaway loop can burn thousands of dollars in an afternoon. This deep-dive adds the missing layer: the @token_meter interceptor that tracks tokens at the provider-call boundary, throttles at 80%, rolls back at 100%, and circuit-breaks on retry storms. Budget as a first-class constraint.
The multiplier stack. Agents cost 1,000x more than chat because five mechanisms compound: re-sent context (62% of bills), tool-call fan-out, retry loops, quadratic attention, and context rot (which amplifies cost by triggering more retries).
The cost iceberg. LLM inference is only ~20% of TCO. The majority lives in context management, retrieval/RAG, and orchestration/eval/governance (the hidden 80%). Optimizing only Layer 1 addresses a fraction of the spend.
The @token_meter interceptor. Wraps every provider call. Throttle at 80% (switch to cheaper model, reduce context). Rollback at 100% (abort + restore). Circuit-break on retry storms (stop the loop). Budget checked BEFORE the call, not after.
Prompt caching is the highest-ROI optimization. 90% discount on cached tokens, zero quality loss. The barrier is prefix stability — dynamic content (timestamps, IDs) in the first tokens kills the cache. Audit your prefix.
The prime directive. Cutting cost while dropping success rate produces a broken-cheaper agent, not an optimized one. Cost optimization must hold quality constant. RouteLLM and prompt caching do this correctly; blind throttling does not.