Autopilot, One Week Later: The Numbers You Asked For

A week ago I published a story about Autopilot, the system where AI agents implement GitHub issues, fix their own CI, argue with reviewers, and open pull requests while I sleep.

The feedback arrived quickly, and the most useful piece was this:

"Nice story. Where are the numbers? How many PRs? What percentage merges fully autonomously? How many escalations? Escaped defects? Average cost?"

Fair. This post answers it in full.

Numbers first, then what changed in the system this week, because the biggest change deserves its own section: the last human ceremony is gone.

The scoreboard

Autopilot has been running against a real production product for three weeks. Everything below comes straight from the GitHub API, not from memory or vibes. All PRs it produces carry an autopilot label, so the accounting is trivially reproducible.

Lifetime, three weeks:

• 155 pull requests opened
• 144 merged, 10 still open, 1 closed without merging
• That is a 93% merge rate for AI-authored PRs

The past seven days alone:

• 97 PRs opened, 87 merged. For comparison, the first two weeks produced 58 in total. The system is accelerating, and not because I got faster.
• Busiest day: 25 PRs in one day.
• Roughly 26,000 lines added and 4,300 deleted in the main repo, in one week.
• Median time from PR opened to PR merged: 39 minutes. A quarter of PRs merged within 14 minutes. 52 of the 87 merged within the hour.

That last number deserves a pause. The median lifetime of a change, from "an agent starts writing code" to "it is on the main branch, gates green, reviewed," is under 40 minutes, around the clock, including the hours when I am asleep. My old personal median was "whenever I got around to it, plus CI, plus finding a reviewer."

The autonomous-merge rate, honestly

Here is where I owe you precision, because "autonomous" hides a spectrum.

For most of these three weeks, the loop worked like this: the agent writes the code, fixes its own CI, and responds to review feedback autonomously, but a human (me) clicks the actual merge button. Call it "autonomous authorship, supervised landing."

The fully hands-free pipeline, where a second AI with its own GitHub App identity reviews the diff, issues a formal APPROVE, and the PR auto-merges with no human in the loop at any point, went live in the middle of this week. Since it went live:

• 34 autopilot PRs merged
• 7 of them merged with zero human involvement: bot-authored, bot-reviewed, bot-approved, auto-merged
• That is a 21% fully autonomous merge rate, one merge in five

Why not higher? Because the reviewer is doing its job. This week the approver bot issued 12 REQUEST_CHANGES verdicts. Every one of those knocks a PR out of the hands-free lane and back into a revision round. The deterministic policy layer also yanks anything touching CI workflows, migrations, or secret-adjacent paths out of the fast lane regardless of how confident the reviewer feels. A 21% hands-free rate with a hostile reviewer is a number I trust. A 95% rate would have meant the reviewer was a rubber stamp, and I would be writing a very different post, probably an incident report.

Escalations: what actually needs a human

This week: 9 new escalation issues filed by the system, and 6 PRs labeled needs-human. Lifetime: 18 escalation issues.

The surprise is what they are about. I expected the machine to escalate when it got confused by code. Almost none of them are that. The actual list reads like a systems administrator's todo: verify a DNS record, provision a secret in two environments, grant an IAM role to a service account, apply a DDL change to the production database. The agents escalate at the boundary of their permissions far more often than at the boundary of their intelligence.

That is by design, and it is the design I would tell you to copy. The agent does not hold production credentials, so production-touching steps become tickets for me instead of actions by it. Roughly one work item in ten ends up needing a human hand, and nearly every time, the hand is doing something I would never have delegated anyway.

Escaped defects

The number the skeptics actually want: how much broken code got through the gates, the adversarial reviewer, and the policy layer, and landed somewhere it could hurt?

Three weeks, 144 merged PRs: 3 defects escaped to a deployed environment. Two of them this week:

1. A custom-domain routing change passed every test and served the wrong site in the deployed environment, because the hosting layer rewrites the Host header in a way no local test exercised.
2. A consent-email path silently refused to send because a missing API key was treated as a hard requirement on a flow that should not have needed it.

Both were caught in the dev environment or in early production smoke testing, both were fixed by the same pipeline that caused them, and both spawned follow-up P0 issues that an agent picked up within minutes. The ugliest incident of the week was a signup-redirect bug that took five PRs over two days to fully extinguish, because each fix revealed the next assumption baked into a cross-domain auth flow.

Notice the shape of all three: none of them are "the AI wrote bad logic." All of them are "the tests could not see the deployed environment." Escapes cluster exactly where the gates are blind: DNS, Host headers, cross-origin redirects, environment configuration. The lesson from the first article holds at the defect level too. The system is only as honest as its feedback loops, and the feedback loops end where the environment begins.

For calibration: 2 escaped defects against 87 merged PRs this week is about a 2% escape rate. I have shipped worse ratios entirely by hand, with fewer excuses.

What it costs

The receiver's panel prices every session from actual token usage, so these are observed figures, not brochure figures.

• A CI-fix session typically lands around 50 cents.
• A full implementation session, from reading the issue through design doc, TDD, gates, and opening the PR, typically costs 2 to 3 dollars on the heavy model.
• Failed attempts, review rounds, and revision sessions are the tax on top. Amortizing the tax across everything that merged, a merged PR comes out to roughly 3 to 5 dollars, all in.

Multiply it out: 87 merged PRs at 3 to 5 dollars each is somewhere around 300 to 450 dollars of API spend for the week, in exchange for 26,000 lines of gated, reviewed, shipped change. I have paid more than that for a single day of contractor time that produced one PR and a Slack thread.

The honest caveat from the first article still applies: the money is the cheap part. The expensive part is the attention the escalations and escapes demand. That cost is real, and it is the one I optimize now.

What changed this week: the start button is gone

Now the part the feedback did not ask for but the week delivered anyway.

When I published the original article, Autopilot was event-driven but not self-driving. Webhooks handled everything reactive: CI failed, review arrived, issue labeled. But the proactive half, "look at the board, decide what to work on next, start it," ran inside an interactive session that I personally launched. The system shipped code while I slept, but only if I remembered to turn the key before going to bed.

As of this week, the receiver drives itself. There is now a scheduler in the receiver: every N minutes, per project, it launches a short headless sweep session. The sweep does board hygiene, looks at what is in flight, and picks the next work items, returning a strict JSON list of assignments. The receiver then dispatches an implementation agent per issue, under a set of new controls that turned out to be the hard part:

• Whole-epic locks. When an agent works an issue, its entire epic is locked. A second agent cannot grab a sibling issue and create two PRs that collide in the same files. Webhook-triggered jobs that hit locked work are not dropped; they defer into a queue and drain in order once the lock frees.
• Agent caps, per project and global. The board does not care that 30 issues are ready. The fleet has a size, and the P0 lane can preempt it.
• Questions never block. In the old interactive loop, an agent that needed a decision could ask me, because I might be watching. A headless agent asks no one. So decisions it cannot make become human-task issues on the board, and the agent moves on to work that is not blocked. The board absorbed another job that used to be mine: being available.

The identity story also completed this week. Every session, including interactive ones, now authors commits as the coder bot through per-command token minting with a one-hour TTL, the reviewer app became organization-aware, and the loop now keeps open PRs current with their base branch, rebasing the ones that fall behind or hit conflicts instead of letting them rot until merge day.

Put together: I used to be the clipboard, then I was the person who started the machine each morning. Now my remaining mechanical role is answering tickets the machine files about me.

The bug of the week, in the proud tradition

The first article had the garbage collector that spent weeks successfully deleting nothing. This week's entry in the same genre: the sweep session kept reporting an empty backlog for a project whose board was visibly full.

The cause: the board read used a single API page. The item list was truncated, the code treated the truncated page as the whole board, and the sweep cheerfully concluded there was nothing to do. No errors. Clean logs. A scheduler, built to remove the human starter, politely doing nothing on schedule.

Same lesson as last time, now with a corollary I am writing on the wall: in an autonomous system, silence is not success, and an empty result is a claim that requires evidence. Any query that returns "nothing to do" now has to prove it saw everything before the system is allowed to believe it.

Where this leaves the argument

Last week I wrote that I did not lose my job, I lost my excuses. The week of data sharpens that.

The numbers say the machine now produces roughly 90 merged changes a week at coffee prices, with a 2% escape rate concentrated precisely where automated feedback cannot reach, escalating one item in ten, almost always at a permission boundary rather than a competence boundary. Every one of those numbers points my actual job at the same target: extend the feedback loops deeper into the environment, shrink the blind spots, and answer the board's tickets faster.

The machine writes the code and, one merge in five, ships it without me entirely. That fraction will grow, because the reviewer gets stricter and the gates get deeper, not looser.