The Day My Autopilot Ran Out of Tokens

This week my autopilot ran out of tokens.

Not crashed. Not stuck on a hard bug. It exhausted the weekly ceiling of my Claude Max subscription two full days before the reset, and went quiet. The board had issues marked ready. The scheduler polled on time, found the work, and could do nothing about it, because the thing that turns issues into code had no fuel. A system I have described in three posts as shipping while I sleep was suddenly a very elaborate way of waiting for a usage window to reset.

If you have read the earlier posts, you know this series runs on honest numbers and honest failures. This one is a failure of mine, not the machine's, and it taught me more than most of the features did.

Flat rates hide the meter

I put the autopilot on a subscription because it felt obviously smart. A flat fee, an always-on coder, no per-request anxiety. And it was smart, right up until it wasn't, because a flat rate does something quietly corrosive: it makes cost invisible. For four weeks I could not have told you what a single pull request costs, and I never felt the need to ask. The subscription answered the question before I could ask it.

Hitting the wall converts that abstract comfort into a very concrete problem. The machine is idle, the backlog is live, and the reset is two days away. You discover that your autonomous system has exactly one fuel line, and you do not even have a gauge on it.

I got my gauge the expensive way. With the subscription capped and the backlog waiting, I did the naive thing: bought $250 in API credits, pay-as-you-go, and felt clever doing it, because a twenty percent discount made it look like a bargain. I turned the machine back on at two in the afternoon. By nine that evening the credits were gone. Two hundred and fifty dollars in one working afternoon. The flat rate had not been hiding a small number.

The remedy, once I stopped staring at the empty balance, turned into a list: get a real gauge, get more fuel lines, and stop spending premium tokens on non-premium work.

The gauge

First, the involuntary measurement. That $250 afternoon was not wasted; it was the most informative money this project has spent, because for the first time the meter was visible while the machine worked. Between two and nine the autopilot opened 17 pull requests across four repositories and merged 15 of them the same day: implemented, CI fixed, argued through review, the whole pipeline. Divide it out and you get the number readers have asked about since the very first post, when all I could offer was a shrug: about $15 per pull request at pay-as-you-go prices, $17 if you only count the merged ones. Compare that to what an hour of a human contractor costs and it is absurdly cheap. Compare it to a flat subscription you believed was the whole bill and it is a cold shower.

The measurement is no longer involuntary, either. The system now tracks the cost of every pull request it ships: each agent job reports its token usage, the dashboard rolls it up per PR, and next to every merged change there is a dollar figure at API list prices. The meter I lacked for four weeks is now a column in a table.

The part that still makes me pause: I did not write that feature. I filed the issue. The autopilot implemented cost tracking for the autopilot, opened the PR, argued with its reviewer, and I merged it. It is one of the 17 PRs from the $250 afternoon, which means the machine spent my credits building the instrument that measures how it spends my credits. This was also the week the system started taking pull requests on its own codebase, which deserves and will get a post of its own. For today's story, one detail matters: when the machine's own metabolism became the problem, the machine wrote the instruments.

The fuel lines

The first decision was ending the single-vendor diet. The pipeline can now run on five providers: Claude, Codex, Gemini, Kimi, or Copilot, configurable per project and per role. The coder can come from one vendor while the adversarial reviewer comes from another, which I already wanted for a different reason: when the same model family writes and reviews the code, they can share blind spots, and independence is the whole point of review.

Underneath that sits the least glamorous change of the week: the instructions every workspace runs on moved from a Claude-specific file to a vendor-neutral AGENTS.md. It took the famine to make me see the layers clearly. The accumulated value of this system is the process: the board conventions, the review gauntlet, the guardrails, the escalation rules. The model executing that process is a part. Parts should be swappable, and now they are.

Making every token count

Then I went after waste, because at $15 a PR every wasted token is a line item.

The biggest offender was tool output. An agent working on an issue runs dozens of shell commands, and some of them, test suites, build logs, dependency installs, dump thousands of lines into the conversation. The model pays to read every one, and pays again to carry them forward for the rest of the session. Now a wrapper bounds every high-volume command: the agent sees the head, the errors, and the tail; the full output lands in a local evidence file it can open only if it actually needs to; and when a command produces identical output twice, the repeat is replaced with a reference to the first run. The dashboard shows bytes saved per job, so even the savings have a gauge.

The prompts got the same treatment. Policy text that every flow carried a private copy of was collapsed into one canonical contract, written once instead of pasted everywhere. And the instructions themselves went through what I call caveman mode: short words, imperative sentences, no diplomacy. "Do X. Never do Y. If Z, stop and ask." The polite, flowing paragraphs I originally wrote read nicely to a human, but the agent follows the caveman version at least as well, at a fraction of the tokens. It is a strange feeling to edit your own writing down to grunts and watch the system get cheaper without getting dumber.

The right model for the job

The most expensive realization was that I had been paying one price for very different kinds of thinking. So the flows are now separated by model class, enforced in config rather than by my discipline:

• Task creation is the highest-leverage step: turning a vague request into a precise, buildable spec. It runs only on the top reasoning tier, Claude Fable with extended thinking or Codex SOL at high effort, and the system flatly refuses to author specs with anything less.
• Implementation runs on the middle tier: Claude Sonnet or Codex Terra. Writing the spec wrong is expensive; typing the code from a good spec is not.
• PR review gets the strong tier again, Fable with a hard thinking budget or SOL, because a reviewer that misses things is worse than no reviewer.
• CI fixes and applying review suggestions, the mechanical ends of the pipeline, go to the cheapest models in each family: Haiku on the Claude side, Luna on the Codex side.

The budgets are caps as well as assignments: an implementation session gets at most 200 turns, a review 60, a CI fix 30. Nothing is allowed to burn indefinitely anymore.

What this looks like in practice today: the main product runs on Claude, Sonnet implementing and Fable reviewing. And the autopilot's own codebase, the system born on Claude and raised on my Claude Max subscription, is currently implemented by Codex Terra and reviewed by Codex SOL. The machine is not loyal to its birth vendor, and after this week neither am I.

And on top of the separation sits the first version of a router. A flow can opt into cheap-first routing: the inexpensive tier gets exactly one attempt, and if it fails or the change looks risky, a deterministic escalation hands the job to the full tier, with the reason logged on the dashboard. Today it is a two-step ladder. The ambition is a real router that reads each task and picks the model the way a good tech lead assigns tickets: hard problems to the senior, routine ones to whoever is free.

What the wall was for

The machine that went hungry on one vendor's tokens learned to eat from five menus, learned to count its own calories, and did most of the cooking itself, on money I handed over at a discount I was briefly proud of. I wrote the issues, merged the PRs, and paid the bill, which is starting to look like a fair summary of my whole job.

I used to think the scary dependency in this project was trusting AI-written code. Four weeks of gates and reviewers handled that better than I expected. The dependency that actually bit me was economic and boring: one subscription, one vendor, no meter. If you run an autonomous system on somebody's flat-rate plan, you have not eliminated cost, you have deferred the conversation. Have it before the wall does it for you.

The full scoreboard, and the strange story of the day this system started rewriting itself, come next post. The numbers are ready. The system counted them itself this time.