Why a factory, not a copilot
The industry has measured the copilot era. DORA's 2025 report finds AI adoption now correlates with higher throughput — and still with lower delivery stability; its summary is that AI amplifies what a team already is (Google Cloud, 2025 DORA report). Faros sees high-AI teams open 47% more pull requests with 91% longer review time. Speed appears where the code is typed and disappears where it is reviewed and shipped. We wanted to know what happens when the whole line changes, with a person still responsible at both ends.
Six design rules
- Deterministic code does the infrastructure. Git, CI, budgets, routing, artefacts are plain code in the harness. Agents never touch a repository directly; the builder may edit only the files named in the approved plan.
- Fail closed, everywhere. Every verdict is a strict JSON contract. If a planner, verifier or reviewer cannot say yes or no unambiguously, the workflow halts and a human looks.
- Two human gates, zero auto-merge. Intake: a person approves the plan before anything is built. Inspection: a person merges, with the evidence trail open. Minutes per ticket; neither optional.
- One ticket, one worktree. Isolation with file-level reservations, and a merge queue that re-verifies every waiting ticket when main moves — added after semantic conflicts appeared at four parallel tickets.
- Different models for different roles. A frontier model plans, a faster model builds, a different model reviews, the verifier can be local. Cost and independence in one setting.
- Economics built in from day one. Per-ticket budgets, a circuit breaker on spend per hour, plan reuse, full metrics per invocation. The week cost $145 because someone could see what every call cost while it ran.
The week in numbers
July 20–27, 2026: 50 tickets processed, 48 completed, two rejected by the factory itself. 390 engine invocations, about 18.6 hours of engine time, $145 at API-equivalent rates — about $2.90 per ticket, excluding project setup and human time.
One night run, 01:12 to 05:47 on July 22: 20 tickets claimed, 18 completed, 2 rejected, 166 engine invocations, $31 — about $1.70 per completed ticket. 42 tests run, all passed, including one correct BLOCKED verdict for a ticket that was already implemented.
Role invocations without error, timeout or restart: planner 94 of 95, builder 78 of 87, reviewer 76 of 76. These are invocation-validity rates, not ticket completion or PR approval rates.
What broke, and where it was fixed
- Semantic conflicts at four parallel tickets → a merge queue with re-verify when main moves.
- Oscillating review loops → prior-round memory and oscillation detection.
- Zombie runs at 3am → hard timeouts and orphan adoption.
- Budget timeouts killing a frontier model mid-plan → per-model limits; planning window raised from 5 to 12 minutes.
All four were fixed in the harness, not in the prompts. That is the point of having a harness.
What this is not
One person, two repositories, one week. It is not a benchmark, not a productivity study and not a price list — the $2.90 is inference only. Review capacity, not model capacity, remains the real constraint: every plan and every merge still needs a person who understands the system, and we say so in our weekly AI delivery reports to clients. Anthropic's guidance on agent systems says the same thing from the other side: keep the design as simple as it can be, test in sandboxes, and let the agent pause for a person at checkpoints (Building effective agents).
What we'd tell a 200-person organisation
Start with one repeatable workflow, not an enterprise transformation. Measure it from day one — cost per ticket, time per stage, first-pass rate per role — because without numbers adoption theatre and real advantage look identical. Then add the next workflow. Mission Brief 005 lets you assemble the line yourself and see what each block is worth; Brief 009 helps you decide whether your organisation is ready for one.
The experiment was run and written up by Bartosz Rychlicki, our AI R&D lead.
