Four days ago I wrote that Claude plans my code changes and Xiaomi's MiMo writes them, and that MiMo costs about 25× less for the same tokens. I promised a before-and-after once I'd fixed the slow parts. Here it is, and the "after" got worse.
Orders created 11 to 23 October: 62 merged, 24 of them on the first round (39%). Orders created 25 to 28 October, after I added an automatic reviewer and a self-review checklist: 14 merged, 2 on the first round (14%). Small sample, and one of those orders took nine rounds on its own. But the direction is clear. A stricter reviewer finds more problems, so more orders go back.
So this week I stopped asking "how do I review harder" and asked where the problems come from.
Should the worker be a subagent at all?
On 22 October Anthropic's Applied AI team published a post arguing that one agent loop with skills usually beats an orchestrator plus specialist subagents. They keep a subagent only when one of three things is true: the single loop degrades measurably from context bloat; tasks run in parallel and each needs a fresh context it can throw away; or one agent can't pick the right tool, or its instructions conflict.
My worker passes the second test. Each order runs in its own git worktree with a fresh context, up to three at once, and the context is discarded when the branch merges. It also passes a test the post doesn't cover: the worker is a different, much cheaper model. Folding it into the Claude loop would put every token at Claude prices.
I kept it. But the post's first failure mode, "context loss" at the handoff, describes my bill exactly. Everything the worker knows comes from one Markdown work order I write.
Whose fault is the rework?
I took 10 orders that came back for rework, across six repos, and pulled the "must fix" list from the first review. That gave 61 items. For each one I asked: did the order say what to do here?
- 55 items (90%): the order was silent or ambiguous. A worker following it word for word could still get it wrong. Example: an endpoint that re-reads a file before writing back. The order never said that a 20-second model call sits between the read and the write, so a second click silently overwrote the first.
- 6 items: the order was explicit and the worker didn't follow it. Example: the order said full-width colons in a message, the code used ASCII.
I used Claude Sonnet to sort the items and checked one order by hand; it matched. The split is the useful part. Reviewing harder only catches the 90% after a full round has already been spent.
Moving the check before the handoff
So now every order gets a "falsifier" pass before it goes out. The idea comes from a prompt someone shared on r/ClaudeAI. I rewrote it for work orders. The model gets only the order and read access to the repo, and it has to return at most seven ranked findings. Each finding quotes the line of the order that triggers it (or says "not mentioned"), gives a concrete input that breaks it, and names the section of the order where the fix belongs. It doesn't write code.
On the same 10 orders, with no repo access:
| Order gaps caught (of 55) | |
|---|---|
| Run 1 | 33 (60%) |
| Run 2 | 31 (56%) |
| Both runs | 25 |
| Either run | 39 (71%) |
One run costs about $0.07 and 34 seconds on Sonnet. Two runs disagree a lot, which is why I run it twice. The Anthropic post makes the same point about evals: run each task more than once, because a failure may show up in only one of two runs.
Compare that with a rework round. From my earlier numbers, a two-round order uses about 9M tokens against 3.6M for a one-round order, plus another review and about 20 minutes of wall-clock.
I used it on the four implementation orders I sent out today. Every one came back with at least two HIGH findings: a lock that didn't cover the read-modify-write, a half-written JSON line that would swallow the next record, a frontmatter value in quotes that a naive parser would reject. All of them went into the orders before MiMo saw them. Three of the four then passed the automatic reviewer on their first delivery; the fourth needed two more rounds. By that same measure (no feedback round needed), 63% of the merged orders from 11 to 23 October passed, so four orders tell me nothing yet. MiMo's API also stalled all afternoon, which added restarts to every order and makes today's round counts useless.
The one order I didn't run it on was the eval order itself. It pointed the worker at files that existed in my main checkout but weren't committed, and a fresh worktree can't see uncommitted files. MiMo spent 23 minutes confirming it couldn't read anything. The falsifier flagged the same mistake in a later order.
Two more things from the post that applied
Progressive disclosure. The skill that runs my daily publishing (four scheduled runs a day) had grown to 53 KB, and every run loaded all of it. I cut it to a 14 KB core and moved each step's details into files the agent reads only when it reaches that step. Before changing anything, I wrote 12 scenarios with 29 do-or-don't decisions and ran both versions twice. The old version scored 28/29 both times, the new one 29/29 both times. The question the old one got wrong was a case of conflicting instructions. One section said "for dispatching, this section wins", and a newer dispatch skill said otherwise.
Hard limits belong in tools. My rule about never printing credentials lived in the prompt, and it was broken four times in September. A rule broken twice needs a structural fix, not louder wording. It's now a pre-tool hook that blocks cat, sed -n and Read on credential paths before the call runs.
If you run subagents at work
The checklist I'd use on any team's setup:
- Count the handoffs. For every subagent, check it against the three conditions above. "Each team owns one" isn't one of them.
- Sort the rework. Pull the last 10 send-backs and split them into "the brief didn't say" and "the worker didn't do". If the first group is large, add a check before the handoff.
- Build 10 golden cases before you touch the architecture, and run each one twice.
- Look for instructions that contradict each other across files. I found two such pairs in my own setup this week.
Limits
- One person, mostly Python. Ten orders is a small sample.
- The judge and the falsifier are the same model family, so the hit rate may be generous. In production the falsifier also reads the repo, which should catch more.
- I wanted a MiMo-run falsifier as a cheaper comparison. MiMo's API kept stalling today and I cancelled those runs.
I do this kind of review for teams as an architecture audit: a 10-case baseline run twice, a keep-or-fold verdict for each subagent against the three conditions, and measured cost and latency before and after. Write to info@zhuojun.li. If you've sorted your own rework, I'd like to compare numbers.
Top comments (9)
@lizhuojunx86 Naming a concrete input that breaks the order is the sharpest form of this. It forces the order to carry the context: the slow model call between the read and the write goes into the spec, or the test never fails.
A stricter automated reviewer changes the effective price per merged task. When the reviewer rejects on stylistic ambiguities instead of broken invariants, the 25x token discount on generation disappears into rerun context. The compound token bill and human triage on nine rounds usually exceed the cost of doing it once under a looser check that only tests functionality.
Agreed that price per merged task is the number to watch, not price per token. A two-round order here used about 9M tokens against 3.6M for a one-round order, so one send-back eats a big part of the 25x.
I'm less sure about the style part. The example I gave from the 61 items was a second click silently overwriting the first, which a functionality-only check should also reject. The closest thing to style was the full-width colons, and the order had asked for those explicitly. I didn't tag all 61 as style vs broken invariant, though, so I can't give you a split. Worth doing. If a chunk turns out to be style, those rules belong in a linter, not in the reviewer.
55 of 61 must-fix items tracing back to orders that were silent or ambiguous is a number every team handing work to agents, or to people, should see. The 20 second model call between a read and a write is a perfect example: nothing in the order said the read could go stale, so the worker had no reason to guard it.
Running the falsifier twice because the runs disagree, then catching 71% of gaps across either run for about $0.07, is far cheaper than a rework round at roughly 9M tokens against 3.6M. Sal Parvez's comparison to estimating callbacks fits too. In fixed scope work the expensive surprises usually trace back to a sentence the brief never had.
Paul Irolla's point that without an acceptance test the agent decides what done means might be the shortest summary of the post. Do you now require every work order to name a concrete input that would prove it finished?
Indirectly. Every falsifier finding has to name an input that breaks the order, and those go into the order before MiMo sees it. It's not a hard rule for every order yet. Four orders isn't enough to tell whether it pays off.
Checking the work order before the crew starts is cheaper than any inspection after. Coming from estimating, most callbacks I saw traced back to a vague scope, not bad work. That's where we're putting the review at ML Systems.
Same pattern here. A vague scope costs a full round before anyone notices, and the order is the cheapest place to fix it.
The rework usually sits in the scope line, not in the code. An order with no acceptance test lets the agent decide what done means. Does a missing test cause most of your rework?
Partly, but mostly it was missing context rather than a missing test. Take the overwrite bug: the order never said a 20-second model call sits between the read and the write. A test the worker wrote from that order would probably have passed. I didn't count how many orders lacked an acceptance test, so I can't say it's the biggest single cause. The falsifier comes at it from the other side: every finding has to name a concrete input that breaks the order.