Cost-Aware Agents: Budget as a First-Class Constraint
Why agents cost 1,000x more than chat, and how to add budget enforcement to any agent loop. The @token_meter interceptor pattern: throttle at 80%, rollback at 100%, circuit-break on retry storms. Prompt caching (90% discount). The prime directive. With a real TypeScript cost-aware loop wrapper.
Agents cost up to 1,000x more than equivalent chat (Stanford-MIT, arXiv 2604.22750) because five mechanisms compound: re-sent context (62% of inference bills per Stanford Digital Economy Lab), tool-call fan-out, retry loops, quadratic attention (O(n²)), and context rot (30%+ degradation at mid-window, which triggers more retries, which costs more).
LLM inference is only ~20% of total cost of ownership. The majority lives in context management (the 62% re-sent-context problem), retrieval/RAG, and orchestration/eval/governance — the hidden 80% that teams don't budget for. Optimizing only Layer 1 (switching models) addresses a fraction of spend.
The @token_meter interceptor pattern (Microsoft EvalAgentic): wraps every provider call, tracks cumulative token usage against a per-task budget, enforces throttle at 80% (switch to cheaper model, reduce context), rollback at 100% (abort + restore prior state), and circuit-break on retry storms (stop the loop). Budget is checked BEFORE the call — prevention, not detection.
Prompt caching is the highest-ROI optimization: up to 90% reduction on cached tokens (Anthropic), zero quality loss. The barrier is prefix stability — dynamic content (timestamps, session IDs, framework-injected UUIDs) in the first tokens invalidates the cache. Audit the prefix; move dynamic content after the stable portion.
The prime directive: cutting cost 80% while dropping success rate 50% produces a broken-cheaper agent, not an optimized one. Cost optimization must hold success rate constant (RouteLLM: 85% cut at 95% quality; prompt caching: 90% cut at 100% quality). Blind throttling without quality monitoring is a prime-directive violation.
The cost-aware loop is a wrapper, not a rewrite. You add the @token_meter interceptor at the provider-call boundary. The loop structure stays the same. Pi's loop + meter.check + meter.record = cost-aware loop. This is how you turn a personal-scale harness into an enterprise-scale harness without redesigning the loop.
HITL at cost thresholds converts uncontrolled spending into controlled spending: log at 50%, warn+choice at 80%, hard stop at 100%. But HITL must be at meaningful decision points (one or two per session), not every turn — frequent alerts produce fatigue and rubber-stamping.
The circuit breaker prevents runaway loops — the highest-risk cost failure. Four trip conditions: consecutive retries (e.g., >= 5), total session retries (e.g., >= 20), fan-out depth (e.g., >= 3), tokens-per-minute. Reset must be MANUAL (automatic reset produces a flapping breaker that never stops the storm).