# DD-24 — Cost-Aware Agents: Budget as a First-Class Constraint

**Course**: Master Course · **Deep-Dive**: DD-24 · **Duration**: 50 min · **Level**: Senior engineer · **Prerequisites**: Modules 0-12, DD-01 (Pi)
**Pillar**: Deep-Dives

> Pi has no budget enforcement. This deep-dive shows how to add it — turning Pi's personal-scale loop into an enterprise-scale loop that won't burn the company's API budget.

---

## Learning Objectives

1. Explain why agents cost up to 1,000x more than chat — the multiplier stack of re-sent context, tool-call fan-out, retry loops, quadratic attention, and context rot.
2. Quantify the invisible cost layer: re-sent context is 62% of total agent inference bills, and the "hidden 80%" of orchestration/eval/governance sits outside the LLM inference line item.
3. Implement the `@token_meter` interceptor pattern: track tokens at the provider-call boundary, throttle at 80% of budget, rollback at 100%, circuit-break on retry storms.
4. Design cost-aware tool selection — route cheap queries to cheap models, reserve expensive tools for cases that justify them.
5. Implement human-in-the-loop (HITL) at cost thresholds — when spending crosses a line, pause for approval.
6. Build a circuit breaker for runaway loops — detect retry storms and tool-call fan-out before they compound.
7. Write a cost-aware execution loop in TypeScript that wraps Pi's loop with budget tracking, and explain how this extends to Course 4 Module E03 (Cost-Aware Agent Architectures).

---

## The Thesis

Pi (DD-01) has a loop. It calls the model, the model calls tools, the tools return, the model calls again. The loop runs until the task is done or the model stops. Nowhere in Pi is there a concept of "how much has this cost?" or "should we continue spending?"

This is fine for a personal harness. It is fatal for an enterprise deployment.

The numbers are stark. Agentic tasks consume up to 1,000x more tokens than equivalent chat tasks (Stanford-MIT, arXiv 2604.22750). Re-sent context — the same system prompt, conversation history, and tool definitions sent with every provider call — accounts for 62% of total agent inference bills (Stanford Digital Economy Lab). Token costs have been doubling every 45 days during the AI build-out. 95% of enterprise generative-AI pilots produce no measurable impact (MIT, 2025). 40% of agent projects will be cancelled by 2027 on cost grounds (Gartner).

The implication: an agent without budget enforcement is a budget fire. A single runaway loop — the model retries a failing tool call 500 times, each retry re-sending the full context — can burn thousands of dollars in an afternoon. At enterprise scale (hundreds of agents, thousands of sessions), the absence of cost controls is not a missing feature. It is an existential risk to the project.

This deep-dive adds the missing layer. Budget as a first-class constraint: tracked at the provider-call boundary, enforced via throttle/rollback/circuit-break, surfaced through HITL at thresholds, and designed into the loop from the start.

---

## DD-24.1 — Why Agents Cost 1,000x More Than Chat

A chat interaction is one request, one response. The user sends a message, the model replies. Total tokens: the message plus the reply. Maybe 500 tokens.

An agent interaction is a loop. Each iteration of the loop is a provider call, and each provider call re-sends the entire context: the system prompt, the full conversation history, all tool definitions, and the new iteration's input. A 10-turn agent session with a 4,000-token context window sends ~40,000 tokens to the provider — not because the task is 80x harder, but because the context is re-sent on every turn.

This is the **multiplier stack** — the five mechanisms that compound to produce the 1,000x cost gap:

### 1. Re-sent context (the largest single cost — 62% of bills)

Every provider call re-sends the accumulated context. Turn 1 sends the system prompt + tool defs. Turn 2 sends the system prompt + tool defs + turn 1's exchange. Turn 3 adds turn 2's exchange. By turn 10, you are re-sending 9 turns of history with every call. The same tokens, billed again and again. Stanford's research puts this at 62% of total agent inference cost — the majority of the bill, and the most invisible part because it doesn't feel like "new" spending.

### 2. Tool-call fan-out

When the model calls a tool, the tool's result goes back into context, and the model calls again. A task that requires 5 tool calls generates 5 additional provider calls, each re-sending the growing context. If each tool call spawns sub-tasks (an agent that calls sub-agents), the fan-out is multiplicative. A 5-deep agent tree with 3 branches per node is 3^5 = 243 leaf calls, each with its own context stack.

### 3. Retry loops

When a tool call fails, the model retries. Three retries triple the cost of that interaction. When a retry loop is unbounded — the model keeps trying because it doesn't know the tool is permanently broken — the cost is unbounded. This is the runaway loop that burns budgets.

### 4. Quadratic attention

Transformer attention is O(n²) in sequence length. Doubling the context more than doubles the compute cost per token. As the agent's context grows (more turns, more tool results), each additional token costs more to process than the last. This is why a 100K-token context is not 10x the cost of a 10K-token context — it's more.

### 5. Context rot

As the context fills, the model's accuracy degrades. Stanford research shows 30%+ performance degradation at mid-window. This means longer sessions not only cost more per token — they produce worse results, which triggers more retries, which costs more. Context rot is a cost amplifier disguised as a quality problem.

The 1,000x figure is the product of these five mechanisms compounding. Re-sent context makes every turn expensive. Fan-out multiplies turns. Retries multiply failures. Quadratic attention makes long contexts superlinearly expensive. Context rot makes long contexts less accurate, triggering more work.

---

## DD-24.2 — The Invisible Cost Layers

The LLM inference bill — the tokens you pay the model provider — is only part of the cost. The full cost of running an agent in production has four layers:

**Layer 1: LLM inference (~20% of TCO).** The token costs. This is what most teams track. It's the visible part of the iceberg.

**Layer 2: Context management (the largest invisible cost).** The storage, retrieval, and re-injection of context across turns. RAG pipelines, context-window management, summarization, and the re-sent-context problem above. This is where the 62% lives.

**Layer 3: Retrieval/RAG.** Embedding generation, vector store queries, reranking. Every retrieval step has a cost — and agents retrieve on most turns.

**Layer 4: Orchestration, eval, and governance (the "hidden 80%").** The infrastructure around the agent: the orchestration layer that drives the loop, the evaluation pipeline that checks outputs, the governance layer that logs and audits. This is the operational overhead that makes an agent production-ready. It's often called "the hidden 80%" because teams budget for Layer 1 and are surprised by Layers 2-4.

The implication: optimizing only Layer 1 (e.g., switching to a cheaper model) addresses 20% of the cost. The majority of agent cost lives in Layers 2-4, and most teams don't instrument it.

---

## DD-24.3 — The `@token_meter` Interceptor Pattern

The core implementation pattern. The `@token_meter` is an interceptor — a wrapper around the provider-call boundary that tracks token usage and enforces budget.

```typescript
// The interceptor wraps every provider call.
// It tracks cumulative token usage against a per-task budget.
// At 80% of budget: throttle (warn, switch to cheaper model, reduce context).
// At 100% of budget: rollback (abort the current operation, restore prior state).
// On retry storms: circuit-break (stop the loop, surface to operator).

interface TokenUsage {
  promptTokens: number;
  completionTokens: number;
  cumulative: number;
}

interface BudgetConfig {
  taskBudget: number;          // max tokens for this task
  throttleAt: number;          // 0.8 = throttle at 80%
  rollbackAt: number;          // 1.0 = rollback at 100%
  retryStormThreshold: number; // max consecutive retries before circuit-break
}

class TokenMeter {
  private usage: TokenUsage = { promptTokens: 0, completionTokens: 0, cumulative: 0 };
  private consecutiveRetries = 0;
  private throttled = false;

  constructor(private config: BudgetConfig) {}

  // Called before every provider call. Returns a decision.
  beforeCall(promptTokens: number): "ok" | "throttle" | "rollback" {
    const projected = this.usage.cumulative + promptTokens;
    const ratio = projected / this.config.taskBudget;

    if (ratio >= this.config.rollbackAt) {
      return "rollback";  // hard stop — abort and restore
    }
    if (ratio >= this.config.throttleAt) {
      this.throttled = true;
      return "throttle";  // soft stop — switch to cheaper model, reduce context
    }
    return "ok";
  }

  // Called after every provider call with actual usage.
  afterCall(usage: { promptTokens: number; completionTokens: number }, isRetry: boolean): void {
    this.usage.promptTokens += usage.promptTokens;
    this.usage.completionTokens += usage.completionTokens;
    this.usage.cumulative += usage.promptTokens + usage.completionTokens;

    if (isRetry) {
      this.consecutiveRetries++;
      if (this.consecutiveRetries >= this.config.retryStormThreshold) {
        throw new RetryStormError(
          `Circuit breaker: ${this.consecutiveRetries} consecutive retries. ` +
          `Cumulative cost: ${this.usage.cumulative} tokens.`
        );
      }
    } else {
      this.consecutiveRetries = 0;  // reset on success
    }
  }

  getUsage(): TokenUsage { return { ...this.usage }; }
  isThrottled(): boolean { return this.throttled; }
}

class RetryStormError extends Error {
  constructor(message: string) { super(message); this.name = "RetryStormError"; }
}
```

This is not pseudocode. It compiles. It is the budget-enforcement layer that Pi lacks.

The three enforcement actions, in order of severity:

**Throttle (80%).** The session is approaching budget. Actions: switch to a cheaper model (e.g., from GPT-4o to GPT-4o-mini — a 27x price spread per RouteLLM research), reduce the context window (summarize-and-evict older turns), or skip non-essential tool calls. The session continues, but at lower cost and potentially lower quality.

**Rollback (100%).** The session has exceeded budget. Actions: abort the current operation, restore the agent to its state before the current step, surface to the operator (or the user) that the task hit budget. The session does not continue past this point without explicit intervention.

**Circuit-break (retry storm).** The session is in a loop — consecutive retries exceeding the threshold. This is the runaway loop. Actions: stop the loop immediately, log the storm, surface to the operator. A retry storm left unchecked will burn the entire budget on repeated failures.

---

## DD-24.4 — Cost-Aware Tool Selection

Not all tools cost the same. A `read_file` tool is free (local filesystem). A `web_search` tool costs a per-query fee. A `run_analysis` tool that calls a paid API costs per call. A `generate_chart` tool that calls an image model costs per image.

Cost-aware tool selection means: the agent (or the orchestrator) knows the cost of each tool and factors it into the decision of which tool to call. This is the agent equivalent of RouteLLM — the UC Berkeley framework that routes queries to cheaper models when possible, cutting cost 85% while maintaining 95% of quality.

The implementation: annotate each tool with a cost estimate (tokens for local tools, dollar-cents for paid tools). At the planning step, the orchestrator can choose the cheaper path when multiple tools could accomplish the same goal. At the throttle threshold, the orchestrator can skip expensive tools and fall back to cheaper alternatives.

This is also where model routing lives. A 27x price spread exists between GPT-4o and GPT-4o-mini. Most agent tasks don't need the expensive model on every turn — the first few turns (planning, decomposition) benefit from the strong model; the execution turns (tool calls, simple reasoning) can run on the cheap model. Cost-aware routing switches models mid-session based on the task's current phase.

---

## DD-24.5 — Human-in-the-Loop at Cost Thresholds

Some spending decisions should not be automated. When an agent session crosses a cost threshold — say, $5 for a single task — the right action is not to throttle or rollback silently. It's to pause and ask a human.

The HITL-at-threshold pattern:

- **Threshold 1 (e.g., 50% of budget): log.** The session is spending. Surface this to the operator dashboard. No interruption.
- **Threshold 2 (e.g., 80% of budget): warn + offer choice.** Notify the user (or operator): "This task has used 80% of its budget. Continue? Switch to a cheaper model? Abort?" The human decides.
- **Threshold 3 (e.g., 100% of budget): hard stop.** No more spending without explicit override. The task is paused, not killed — the human can authorize more budget if the task is worth it.

This pattern converts uncontrolled spending into controlled spending. The agent doesn't get to burn unlimited budget on a single task. The human is in the loop at the points where the cost of continued spending exceeds the cost of asking.

The threshold values depend on the deployment context. A personal harness (Pi) might set thresholds at token counts (10K, 50K, 100K). An enterprise deployment might set them in dollars ($1, $5, $20). The pattern is the same; the numbers differ.

---

## DD-24.6 — Circuit Breakers for Runaway Loops

The runaway loop is the highest-risk failure mode for cost. A loop where the model retries a failing tool call, or where a sub-agent spawns sub-agents that spawn sub-agents, can compound cost faster than any human can react.

The circuit breaker pattern, borrowed from electrical engineering: when current exceeds a threshold, the breaker trips and the circuit opens. No more current flows. You reset the breaker manually.

For agents, the "current" is the retry rate or the fan-out depth. The circuit breaker trips when:

- **Consecutive retries exceed a threshold** (e.g., 5). The same tool failing 5 times in a row is a storm, not a transient error.
- **Total retries in a session exceed a threshold** (e.g., 20). Even if no single tool storms, a session with 20 retries across different tools is pathological.
- **Fan-out depth exceeds a threshold** (e.g., 3 levels of sub-agents). A 5-deep agent tree is 3^5 = 243 leaf calls — usually unjustified.
- **Tokens-per-minute exceed a threshold.** A session burning 50K tokens/minute is in a tight loop. Healthy sessions are bursty, not steady.

When the breaker trips, the session stops. The operator is notified. The breaker is reset manually (not automatically — automatic reset is how you get a flapping breaker that never truly stops the storm).

---

## DD-24.7 — A Cost-Aware Execution Loop in TypeScript

This is a cost-aware wrapper around Pi's loop. It adds: a `TokenMeter` instance per session, budget checks before every provider call, throttle/rollback/circuit-break enforcement, and HITL at thresholds.

```typescript
// cost-aware-loop.ts — extends Pi's loop with budget enforcement
import { TokenMeter, RetryStormError, BudgetConfig } from "./token-meter";

interface AgentConfig {
  budget: BudgetConfig;
  strongModel: string;    // e.g., "gpt-4o"
  cheapModel: string;     // e.g., "gpt-4o-mini"
  hitlThreshold: number;  // e.g., 0.8 — pause for human at 80%
}

async function costAwareLoop(
  task: string,
  config: AgentConfig,
  provider: (model: string, messages: any[]) => Promise<{ content: string; usage: { promptTokens: number; completionTokens: number }; isError?: boolean }>,
  tools: Record<string, () => Promise<any>>,
): Promise<{ result: string; totalCost: number; throttled: boolean; hitlTriggered: boolean }> {
  const meter = new TokenMeter(config.budget);
  const messages: any[] = [{ role: "system", content: task }];
  let hitlTriggered = false;
  let lastWasRetry = false;

  for (let iteration = 0; iteration < 50; iteration++) {  // hard cap on iterations
    // --- Budget check BEFORE the call ---
    const estimatedPromptTokens = JSON.stringify(messages).length / 4;  // rough estimate
    const decision = meter.beforeCall(estimatedPromptTokens);

    if (decision === "rollback") {
      // Hard stop — budget exceeded. Restore prior state, surface to operator.
      messages.pop();  // rollback the last message
      return {
        result: `Task aborted: budget exceeded (${meter.getUsage().cumulative} tokens used).`,
        totalCost: meter.getUsage().cumulative,
        throttled: meter.isThrottled(),
        hitlTriggered,
      };
    }

    // --- HITL at threshold ---
    const usageRatio = meter.getUsage().cumulative / config.budget.taskBudget;
    if (usageRatio >= config.hitlThreshold && !hitlTriggered) {
      hitlTriggered = true;
      // In production: pause and await human approval. Here we log and continue.
      console.warn(`[HITL] Task at ${Math.round(usageRatio * 100)}% of budget. Pausing for approval.`);
      // const approved = await requestHumanApproval(task, meter.getUsage());
      // if (!approved) return { result: "Task aborted by human.", ... };
    }

    // --- Model selection: strong for early turns, cheap when throttled ---
    const model = meter.isThrottled() ? config.cheapModel : config.strongModel;

    // --- The provider call ---
    let response;
    try {
      response = await provider(model, messages);
    } catch (err) {
      // Provider error — counts as a retry for the circuit breaker
      try {
        meter.afterCall({ promptTokens: estimatedPromptTokens, completionTokens: 0 }, lastWasRetry = true);
      } catch (storm) {
        return {
          result: `Task aborted: ${(storm as Error).message}`,
          totalCost: meter.getUsage().cumulative,
          throttled: meter.isThrottled(),
          hitlTriggered,
        };
      }
      continue;
    }

    // --- Record usage AFTER the call ---
    try {
      meter.afterCall(response.usage, lastWasRetry);
      lastWasRetry = response.isError ?? false;  // an error response likely means a retry next
    } catch (storm) {
      // Circuit breaker tripped
      return {
        result: `Task aborted: ${(storm as Error).message}`,
        totalCost: meter.getUsage().cumulative,
        throttled: meter.isThrottled(),
        hitlTriggered,
      };
    }

    messages.push({ role: "assistant", content: response.content });

    // --- Check if the task is done ---
    if (response.content.includes("[DONE]") || !response.isError) {
      // Simple completion heuristic — real harnesses use a proper termination check
      break;
    }
  }

  return {
    result: messages[messages.length - 1].content,
    totalCost: meter.getUsage().cumulative,
    throttled: meter.isThrottled(),
    hitlTriggered,
  };
}
```

Read this code for five production details:

1. **Budget is checked BEFORE every call**, not after. By the time you've paid for the call, it's too late to prevent the spend.
2. **Throttle switches models mid-session.** Early turns use the strong model; throttled turns use the cheap model. This is the RouteLLM pattern applied at the session level.
3. **HITL triggers once, at the threshold.** It doesn't interrupt every subsequent turn — it flags that the threshold was crossed and lets the human decide whether to continue.
4. **The circuit breaker catches both provider errors and retry-indicating error responses.** Either can signal a storm.
5. **There is a hard iteration cap (50).** Even if every other safeguard fails, the loop cannot run forever. This is the fail-safe that prevents unbounded cost in the worst case.

---

## DD-24.8 — Prompt Caching: The 90% Discount You're Probably Missing

Prompt caching is the single highest-ROI cost optimization for agents. Anthropic's prompt caching gives a 90% reduction on cached tokens. If your system prompt and tool definitions are 2,000 tokens and you make 100 calls, caching turns 200,000 billable tokens into 20,000.

The catch: caching is prefix-based. The cache hits only if the beginning of the prompt is identical across calls. This means:

- **Static prefixes cache well.** A system prompt that never changes caches perfectly.
- **Dynamic prefixes kill the cache.** A timestamp in the system prompt, a session ID, a per-request UUID — any of these in the prefix invalidates the cache. Frameworks that inject dynamic IDs (some LangChain configurations do this) silently destroy the caching benefit.
- **Tool definitions should be stable.** If your tool descriptions change between calls (versioned strings, dynamic content), the prefix changes and the cache misses.

The production discipline: audit your prompt prefix. Remove anything dynamic from the first N tokens. Put timestamps, session IDs, and per-request data AFTER the stable prefix. This single change can cut your inference bill by up to 90% with no quality loss.

---

## DD-24.9 — The Prime Directive and the C4 Connection

There is a trap in cost optimization. The "prime directive" from the research literature: cutting cost 80% while dropping success rate 50% does not produce an optimized agent. It produces a broken agent that is cheaper to run.

An agent that costs 80% less but succeeds half as often is not "optimized for cost." It is a worse product. The customers who relied on the 50% of tasks that now fail are not saved money — they are failed tasks. Cost optimization that sacrifices success rate is a regression, not an improvement.

The correct framing: cost optimization must hold success rate constant (or improve it) while reducing cost. This is what RouteLLM does (85% cost cut at 95% of quality). This is what prompt caching does (90% cost cut at 100% of quality). This is what the throttle-at-80% pattern does (it degrades gracefully, it doesn't break).

The trap is most dangerous in the throttle action. When you switch from the strong model to the cheap model at 80% budget, the cheap model may fail at tasks the strong model handled. The throttle must be paired with a quality check — if the cheap model's output is below a confidence threshold, fall back to the strong model for that turn (and accept the cost). Blind throttling without quality monitoring is the prime-directive violation.

### The C4 Connection (Module E03)

This deep-dive builds the single-session cost-aware loop. Course 4 Module E03 (Cost-Aware Agent Architectures) builds the fleet-level cost architecture: per-tenant budgets, cost allocation across teams, the cost dashboard that surfaces the hidden 80%, the chargeback model that makes agent cost visible to the business units consuming it. This deep-dive is the foundation; E03 is the enterprise scale-up.

---

## Anti-Patterns

### Optimizing only Layer 1 (LLM inference)
Switching to a cheaper model addresses ~20% of the cost. The majority lives in context management (62% of inference bills), retrieval, and orchestration. A cost-optimization effort that only looks at the model-choice lever will underdeliver.

### Blind throttling without quality monitoring
Throttle switches to a cheaper model. The cheaper model fails at tasks the strong model handled. If you don't monitor quality, you won't know you're violating the prime directive (cheaper but broken). Always pair throttle with a quality check.

### Dynamic content in the prompt prefix
A timestamp or session ID in the first tokens of the system prompt invalidates prompt caching. You lose the 90% discount. Audit your prefix; move dynamic content after the stable portion.

### No circuit breaker on retry loops
A tool that fails and retries indefinitely is a budget fire. Without a circuit breaker (consecutive-retry threshold), a single broken tool can burn the entire session budget. The breaker must exist from day one.

### HITL at every threshold
HITL is valuable at the decision points (typically one or two per session). HITL at every turn, or at every 10% of budget, produces alert fatigue — the human stops reading the alerts and rubber-stamps everything. Set thresholds at meaningful decision points, not at frequent intervals.

### Measuring cost in tokens without converting to dollars
Token counts are abstract. A session that "used 500K tokens" doesn't feel expensive until you convert: at $5/M input tokens, that's $2.50 per session. For an enterprise running 10,000 sessions/day, that's $25K/day. Always convert to dollars for reporting; it's the number the business understands.

---

## Key Terms

| Term | Definition |
| --- | --- |
| Multiplier stack | The five mechanisms (re-sent context, fan-out, retries, quadratic attention, context rot) that compound to make agents 1,000x more expensive than chat. |
| Re-sent context | The same prompt/history/tool-defs sent on every provider call. 62% of total agent inference bills. |
| `@token_meter` | The interceptor pattern: wraps provider calls, tracks cumulative token usage, enforces throttle/rollback/circuit-break. |
| Throttle (80%) | Budget approach triggers cost reduction: switch to cheaper model, reduce context, skip non-essential tools. Session continues. |
| Rollback (100%) | Budget exceeded triggers abort: stop the operation, restore prior state, surface to operator. Session does not continue. |
| Circuit breaker | Retry-storm or fan-out-depth detector. Trips on consecutive retries, total retries, fan-out depth, or tokens-per-minute. Stops the loop. |
| HITL at threshold | Cost threshold triggers human-in-the-loop: log (50%), warn+choice (80%), hard stop (100%). |
| Cost-aware tool selection | Annotating tools with cost estimates and routing to cheaper paths when possible. The agent equivalent of RouteLLM. |
| Prompt caching | Prefix-based caching of system prompts/tool defs. 90% reduction on cached tokens. Killed by dynamic content in the prefix. |
| Prime directive | Cutting cost while dropping success rate produces a broken-cheaper agent, not an optimized one. Cost optimization must hold quality constant. |
| The hidden 80% | Orchestration, eval, and governance infrastructure — the cost layer most teams don't budget for. |
| RouteLLM | UC Berkeley framework for routing queries to cheaper models. 85% cost cut at 95% of quality. |

---

## Lab Exercise

See [07-lab-spec.md](07-lab-spec.md) — build a TypeScript cost-aware loop wrapper with `TokenMeter`, budget enforcement (throttle/rollback/circuit-break), HITL simulation, and a runaway-loop demo.

---

## References

1. Stanford-MIT (arXiv 2604.22750) — agentic tasks consume up to 1,000x more tokens than chat.
2. Stanford Digital Economy Lab — re-sent context is 62% of total agent inference bills.
3. Microsoft EvalAgentic — the `@token_meter` interceptor pattern (throttle at 80%, rollback at 100%, circuit-break on retry storms).
4. RouteLLM (UC Berkeley, ICLR 2025) — routing framework, 85% cost cut at 95% of quality.
5. Budget-Aware Tool-Use (arXiv 2511.17006) — cost-aware tool selection research.
6. Anthropic — prompt caching, 90% reduction on cached tokens.
7. MIT (2025) — 95% of enterprise generative-AI pilots produce no measurable impact.
8. Gartner — 40% of agent projects cancelled by 2027 on cost grounds.
9. Cockroach Labs, Microsoft Token Economics — cost-layer analysis (the hidden 80%).
10. DD-01 (Pi) — the personal-scale loop this deep-dive extends with budget enforcement.
11. Course 4 Module E03 — Cost-Aware Agent Architectures: fleet-level budgets, chargeback, cost dashboards.

---

*Pi has no budget enforcement. This deep-dive shows how to add it — turning Pi's personal-scale loop into an enterprise-scale loop that won't burn the company's API budget.*