How four embarrassing bugs in v1, two sharp comment threads, and Randal L. Schwartz's Synthetic Scars architecture pushed a 1,031-line test guardrail into a 1,783-line governance engine—and nearly blocked its own release along the way.
In Antigravity Field Notes #1, I wrote about auditing my daily Google Antigravity setup, trimming the fixed per-turn prompt prefix by 42.4%, and moving test-tampering checks out of Markdown and into a 1,031-line (39,144-byte) standard-library Python script.
Shipping that first version felt clean for about forty-eight hours. Then two things happened at once: readers in the comments started poking at the exact joints I had taped over, and I started throwing multi-agent workloads at v1 every day.
In the comment thread on Field Notes #1, @max_quimby put a finger straight on the softest spot in the design: if an agent with shell access can unlock a Merkle-sealed test tree by passing renegotiate --reason "fixed tests", a prose rationale is just an honor system with extra steps. Any language model can invent a plausible excuse in 200 milliseconds. Around the same time, howcani highlighted a subtler problem on the red side of the cycle: witnessing a non-zero exit code proves that a command failed, not that the failure was informative or tied to the right invariant.
Meanwhile, v1 outgrew its original single-file Gist. As the tool gained a full automated test harness, changelog, and stricter contracts, I moved it into a standalone MIT repository at FranciscoGrandon/ag-ops. That repository exists because v1 developed four concrete cracks once real multi-agent sessions (a main orchestrator coordinating parallel subagents) hit it every day.
Four Cracks That Showed Up Once v1 Met Daily Work
1. Receipt collisions inside .ag_state/merkle_guard.json
Version 1 wrote a single hardcoded file (.ag_state/merkle_guard.json) per state directory. That worked fine when one agent sealed all of tests/. It fell apart the afternoon two subagents worked concurrently on separate modules in the same folder (tests/test_a.py and tests/test_b.py), because the second witness-red run quietly overwrote the first agent's receipt. Worse, v1 excluded *.tmp globally during Merkle traversal, so a stray fixture dropped at tests/sub/datos.tmp went completely unnoticed. Running witness-red against an empty directory also happily sealed zero files and issued a valid receipt over nothing.
2. An unanchored regex in distill that panicked over polite success logs
To extract pytest failures without dumping thousands of lines into context, v1 scanned output with an unanchored E\s{3} pattern meant to catch pytest's E AssertionError: lines. Then a batch runner printed this status banner near the top of a job:
Step 1 DONE everything fine
Look closely at the end of DONE followed by four spaces: E. My error extractor saw "DONE ", panicked over the trailing E plus three spaces, and spent its entire root-cause line budget preserving a cheerful progress banner 2,000 lines above the real crash. I had built a failure filter that was allergic to well-tabulated optimism.
3. Premature funeral notices in state
Our lexical runlevel tracker (00_plan.json, 10_build.json) lets long-running sessions survive context compaction. In v1, if Step 10 registered an expected output path (--artifacts dist/schema.json) while marking the phase EN_PROGRESO before the build finished, state write recorded "NOT_FOUND". On the next check, state status reported FILE_DELETED for an artifact that had never been born; once dist/schema.json did appear on disk, state status never sealed its SHA-256 hash, leaving later mutations invisible. Reusing step 10 with a new slug also unlinked the previous step file without warning.
4. Blind spots and false accusations in ast-lint
In v1, TK002 caught forbidden bg= / fg= arguments on themed ttk widgets using naive prefix heuristics. If an agent wrote import tkinter.ttk without as ttk and called tkinter.ttk.Button(root, bg="red"), the linter missed it completely. Conversely, if a developer wrote a plain function named def ttk_helper(bg="blue"):, startswith("ttk_") accused it of being a broken Tkinter widget. Beyond that, AUTH001 authorship checks were hardcoded instead of opt-in, and the linter had nothing to say when an agent took its two favorite shortcuts under pressure: silencing failures with except Exception: pass or importing from unittest.mock import MagicMock straight into a production module.
Taking Notes from Randal L. Schwartz's Synthetic Scars
My first instinct after finding those cracks was the bad habit most of us learned over the last two years: open GEMINI.md and type four new paragraphs of sternly worded rules. Never use bare except: pass. Never import unittest.mock outside tests/. Always clean up async streams.
Before doing that, I spent several evenings studying Randal L. Schwartz's ongoing Synthetic Scars series (especially Part 12, Giving AI Pain: The Architecture of Synthetic Scars & The Rapid-Regret Miner) and testing how his systems model mapped onto what I was seeing inside Antigravity.
I am approaching this as a working systems engineer, not a theorist, so what grabbed me in Randal's writing was how directly three of his observations diagnosed my own bugs:
- The Quota vs. Prompt-Compliance Death Loop: Every time you add negative prose rules ("DO NOT swallow exceptions", "DO NOT mock production services") to a system prompt or skill file, you inject the exact tokens of that anti-pattern into the Transformer's KV-cache. During autoregressive generation, pretraining gravity from millions of internet tutorial snippets overpowers the thin semantic negation (the classic pink-elephant trap). The agent emits the banned pattern anyway, you burn 30 turns pasting 50k-token diffs to argue about it, your weekly token quota evaporates by Wednesday, and you respond by adding three more negative rules to the prompt.
-
Why the Bitter Lesson builds Cathedrals on Quicksand in open-world repos: Rich Sutton's Bitter Lesson holds in closed games like Go or chess because the board is fully observable and the reward signal is exact. Open-world repositories have neither property. Give an unconstrained coding agent 10x more test-time compute when a test fails, and it rarely discovers your undocumented architectural invariants; instead, it uses that extra compute to construct what Schwartz calls a Cathedral on Quicksand: 400 lines of adapter shims, tautological mocks, or silent
exceptblocks engineered to turn pytest green while hollowing out the system underneath. -
I_goalvs.I_path(π)across 588 production scars: When Schwartz audited 588 real production scars, more than 95% of critical failures did not violate the explicit ticket goal (I_goal). They broke path-induced invariants (I_path(π))—the secondary obligations that only exist because the agent chose implementation pathπ, such as wiring an error/cancel exit out of an async fetch state or guarding a callback against late arrivals after teardown. Humans learnI_path(π)because getting paged at 3:00 AM hurts. An LLM has no nervous system; asking it politely in Markdown to care about 3:00 AM is a category error.
If I wanted my Antigravity sessions to respect I_path(π) without bloating GEMINI.md back up to 20 KB, the pain had to live in exit codes, file hashes, AST nodes, and Git history.
Inside ag_ops v2.1.0: Turning Field Notes into Deterministic Gates
Translating those ideas into standard-library Python pushed ag_ops.py from a 1,031-line test wrapper to a 1,783-line governance engine (v2.1.0), still running on zero third-party packages.
| Dimension |
ag_ops v1.0 |
ag_ops v2.1.0 |
Delta / Engineering Notes |
|---|---|---|---|
| Core Script Size |
1,031 lines (39,144 B) |
1,783 lines (74,674 B) |
+72.9% lines; single-file architecture preserved |
| External Dependencies |
0 (14 stdlib modules) |
0 (15 stdlib modules) |
Added fnmatch for receipt-bound --ignore globs |
| CLI Subcommands |
4 (guard, distill, state, ast-lint) |
5 (+ regret-mine) |
Added Stage-1 AST/Git Rapid-Regret Miner |
Static Rules (ast-lint) |
3 (.py only) |
9 (5 .py + 4 .dot) + --context-map
|
TK001, TK002, PATH001, PATH002, opt-in AUTH001 + Graphviz DOT001, DOT002, SCAR-PROC-152, SCAR-ASYNC-01
|
| Guard Receipts & Renegotiation | Single merkle_guard.json, prose --reason
|
merkle_guard_<sha256[:12]>.json, SCAR-PROC-150
|
Target isolation, persistent collisions array, --require-witness
|
| Automated Test Suite | Ad-hoc script checks |
tests/test_ag_ops.py (333 lines) |
6/6 integration suites using isolated temp dirs |
Here is how each piece works in practice, using real invocations and verbatim JSON payloads from ag_ops.py v2.1.0.
1. guard and SCAR-PROC-150: Isolated Receipts, Collision Telemetry, and Constructive Witnesses
First, _receipt_path_for_target killed the single-file collision bug by hashing the normalized resolved target path (os.path.normcase(str(test_path.resolve()))) into .ag_state/merkle_guard_<sha256[:12]>.json. Two subagents sealing tests/test_a.py and tests/test_b.py in parallel now write independent v2 receipts.
Exclusions in _compute_merkle_tree were stripped down to surgical minimums: only the exact state_dir, the receipt file itself, __pycache__, .pyc/.pyo/.pyd, root-level .pytest_cache, and explicit globs passed via --ignore (which get persisted inside ignored_patterns in the receipt so verify-green enforces the exact same boundary). Empty directories fail immediately with EMPTY_TEST_TARGET (exit 1), and witness-red blocks non-assertion runner failures (exit codes 2, 3, 4, 5, 126, 127, 9009), returning INVALID_TEST_INVOCATION unless overridden via --expected-exit.
Every time verify-green catches a violation (COMMAND_MISMATCH, EACCES_TEST_TAMPERING, or EACCES_TEST_TAMPERING_DURING_RUN at exit 2; COMMAND_TIMED_OUT or TEST_STILL_FAILING at exit 1), _record_guard_collision appends a UTC-stamped event into the receipt's collisions list before exiting. Even if an agent backs out its unauthorized test edit two minutes later, the collision stays on the permanent record.
Then came @max_quimby's question: how do you stop lazy prose renegotiations when subagents run unattended without an interactive TTY? You cannot call blocking input("Allow test edit? [y/N]") inside a background pipe without hanging or raising EOFError. Instead, v2.1.0 implements Schwartz's SCAR-PROC-150 constructive witness gate via --require-witness:
# Agent tries to unlock a modified test with a hand-wavy prose excuse:
python ag_ops.py guard renegotiate \
--test-dir tests/test_billing.py \
--reason "fixed tests" \
--require-witness \
--json
{
"ok": false,
"error": "LAZY_RETRACTION_REJECTED: --require-witness is active; specify --witness-type (spec-collision | api-lock | arch-fork)."
}
With --require-witness active, prose alone gets exit 1. To obtain a RENEGOTIATED_RED receipt, the agent has to pass a proof that _verify_constructive_witness can verify mechanically against disk or compiler output:
-
spec-collision: requires--conflict-a file:lineand--conflict-b file:line, verifying via_validate_file_line_refthat both files exist on disk, both line numbers are in bounds, and the two locations are distinct. -
api-lock: inspects the failing test run'sstdout/stderr(case-insensitively) for concrete boundary/type lock diagnostics (TypeError,AttributeError,ImportError,Final,Protocol,sealed,cannot assign,visibility,private,readonly). -
arch-fork: requires--fork-optionswith at least two pipe-separatedLabel@file:lineblast-radius anchors verified on disk.
python ag_ops.py guard renegotiate \
--test-dir tests/test_billing.py \
--reason "RFC-104 mandates UUIDv4 strings while legacy parser returns int" \
--require-witness \
--witness-type spec-collision \
--conflict-a docs/rfc_104.md:3 \
--conflict-b src/billing.py:2 \
--json
{
"ok": true,
"status": "RENEGOTIATED_RED",
"reason": "RFC-104 mandates UUIDv4 strings while legacy parser returns int",
"new_tree_hash": "c3704260ab99c04b3f7316f3fb075eb305853816466d3155762eff89714174dd",
"previous_tree_hash": "de4dbd3b859429a6c711918cd0af3bc8c41095e0e0409f9173b122f662e8e69d",
"cmd": "python tests/test_billing.py",
"red_exit_code": 1,
"history_count": 1,
"constructive_witness": {
"type": "spec-collision",
"conflict_a": "D:/Users/francisco.grandon/AppData/Local/Temp/ag_ops_lab/docs/rfc_104.md:3",
"conflict_b": "D:/Users/francisco.grandon/AppData/Local/Temp/ag_ops_lab/src/billing.py:2"
},
"receipt_file": "D:\\Users\\francisco.grandon\\AppData\\Local\\Temp\\ag_ops_lab\\tests\\.ag_state\\merkle_guard_c4d80759f832.json"
}
Asking a Transformer in Markdown to respect test integrity invites creative lawyering; handing it exit 2 and a SHA-256 receipt leaves nothing to negotiate.
2. ast-lint: Path Invariants, Vector 4 (reflex), --context-map, and Pre-Flight Graphviz .dot
In ast-lint, _AntigravityVisitor replaced prefix guessing with full _dotted_name AST resolution across two ast.walk passes. Calling tkinter.ttk.Button(root, text='Sync', bg='red') is now caught reliably even without an as ttk alias, while def ttk_helper(bg='blue'): and tk.Button(root, bg='green') pass untouched. AUTH001 only runs when --author is explicitly passed.
To guard Python path invariants (I_path(π)), v2.1.0 added two rules targeting classic Cathedral on Quicksand maneuvers:
-
PATH001: flags broadexcept:orexcept Exception:/BaseException:handlers whose body is solelypassor.... -
PATH002: flagsunittest.mockormockimports anywhere outside test paths (test_*.py,*_test.py,conftest.py, ortest/,tests/,testing/directories).
Every violation also carries Vector 4 from Schwartz's 4-vector scar anatomy: a "reflex" field containing the concrete orthogonal fix so the model doesn't stall in an apology loop:
python ag_ops.py ast-lint src/invoice_ui.py --json
{
"ok": false,
"elapsed_ms": 0.4,
"files_checked": 1,
"violations_count": 3,
"violations": [
{
"file": "D:\\Users\\francisco.grandon\\AppData\\Local\\Temp\\ag_ops_lab\\src\\invoice_ui.py",
"line": 2,
"rule": "PATH002",
"message": "Mocking primitives from 'unittest.mock' imported in production module.",
"reflex": "Remove unittest.mock imports from production modules; pass dependencies via constructor or callable parameters."
},
{
"file": "D:\\Users\\francisco.grandon\\AppData\\Local\\Temp\\ag_ops_lab\\src\\invoice_ui.py",
"line": 9,
"rule": "TK002",
"message": "Direct 'bg=' argument forbidden on ttk.Button. Use ttk.Style() or tk equivalent.",
"reflex": "Configure 'bg' via ttk.Style().configure('<Style>.TButton', ...) or switch to tk.Button."
},
{
"file": "D:\\Users\\francisco.grandon\\AppData\\Local\\Temp\\ag_ops_lab\\src\\invoice_ui.py",
"line": 12,
"rule": "PATH001",
"message": "Silent broad exception swallowing ('except: pass') hides broken path invariants.",
"reflex": "Catch a specific exception type and log/handle the failure context, or re-raise the error."
}
]
}
Static scars also age poorly when dependencies change, a failure mode Schwartz calls Scar Ossification (autoimmune drift). Passing --context-map context_map.json pins any rule ID to an anchor file's SHA-256 signature (for example, pyproject.toml). If a library upgrade alters pyproject.toml, _apply_context_signatures automatically downgrades that rule from ENFORCED (exit 1) to PROBATIONARY_REVERIFY ("autoimmune_drift": true, exit 0), warning the orchestrator to re-verify the rule rather than blocking valid builds:
{
"ok": true,
"elapsed_ms": 1.041,
"files_checked": 1,
"violations_count": 0,
"violations": [],
"probationary_count": 1,
"probationary_warnings": [
{
"file": "D:\\Users\\francisco.grandon\\AppData\\Local\\Temp\\ag_ops_lab\\src\\sync_button.py",
"line": 3,
"rule": "TK002",
"message": "Direct 'bg=' argument forbidden on ttk.Button. Use ttk.Style() or tk equivalent.",
"reflex": "Configure 'bg' via ttk.Style().configure('<Style>.TButton', ...) or switch to tk.Button.",
"severity": "PROBATIONARY_REVERIFY",
"autoimmune_drift": true
}
]
}
Finally, ast-lint now validates pre-flight Graphviz .dot state machines (_lint_dot_state_machine, or --include-dot on directories) before an agent writes async code. Four topological rules run in sub-millisecond time: DOT001 (deadlock freedom: non-terminal states need out-degree >= 1), DOT002 (reachability: non-initial states need in-degree >= 1), SCAR-PROC-152 (async states marked via parallelogram, async=true, or Awaiting*/Fetching*/Stream*/Loading* prefixes must have out-degree >= 2 including an explicit error|cancel|dispose|fail|abort|timeout|close exit edge), and SCAR-ASYNC-01 (every non-teardown resolution edge from an async state must declare a [guard: ...] predicate against late-arrival zombie callbacks):
digraph SyncStream {
Idle -> Fetching
Fetching -> Active [label="onPayload"]
Active -> Closed [label="onClose"]
}
python ag_ops.py ast-lint sync_stream.dot --json
{
"ok": false,
"elapsed_ms": 0.436,
"files_checked": 1,
"violations_count": 2,
"violations": [
{
"file": "D:\\Users\\francisco.grandon\\AppData\\Local\\Temp\\ag_ops_lab\\sync_stream.dot",
"line": 1,
"rule": "SCAR-PROC-152",
"message": "Async state 'Fetching' lacks explicit teardown/cancel/error exit edge (out-degree=1).",
"reflex": "Add an explicit transition from 'Fetching' to a Disposed, Cancelled, or Error state."
},
{
"file": "D:\\Users\\francisco.grandon\\AppData\\Local\\Temp\\ag_ops_lab\\sync_stream.dot",
"line": 1,
"rule": "SCAR-ASYNC-01",
"message": "Late-arrival zombie race on 'Fetching -> Active': missing '[guard: ...]' predicate on edge label.",
"reflex": "Add a cancellation/epoch guard on the resolution edge, e.g. Fetching -> Active [label=\"onResolve [guard: !isClosed && reqId == activeReqId]\"]."
}
]
}
3. distill and state: Anchored Deltas and Artifact Lifecycle Sealing
Fixing distill (--cmd, --max-lines, --log-dir, --timeout exiting with 124 after killing the isolated process group) meant anchoring _ERROR_LINE_RE strictly to line boundaries: ^\s*E\s{3} for pytest assertion lines, paired with ^\s*error(?:\[[A-Z0-9]+\])?: for compiler diagnostics like Rust's error[E0382]:. " Step 1 DONE everything fine" now scrolls past in peace, while the real E AssertionError anchors the critical delta.
In state (write and status, requiring --dir alongside --step, --name, --status, --summary, --artifacts, --replace, and --strict), reusing an existing step number with a different slug now aborts with STEP_CONFLICT (exit 1) unless --replace is explicit. Artifact tracking follows a clean three-stage lifecycle:
-
Pre-birth: An artifact registered before creation starts as
"NOT_FOUND". Callingstate status --dir .ag_state/runlevels --strictflagsARTIFACT_NEVER_EXISTED(exit 2), accurately distinguishing "not built yet" from "deleted." -
Automatic seal on birth: The first time
state statussees that file materialize on disk, it computes_sha256_file, persists the digest atomically intoNN_<slug>.json, and returnshas_drift: false(exit 0). -
Post-seal integrity: Any later tampering or removal triggers
DRIFT_MODIFIEDorFILE_DELETED(exit 2under--strict).
4. regret-mine: Stage-1 Deterministic Rapid-Regret Miner
Inspired by Target 90 in Synthetic Scars, regret-mine (--repo, --window-min 60, --limit 100, --state-dir) audits recent Git commits to find where an agent (or human) wrote a function and had to rush back into the exact same AST node minutes later to fix what broke.
For consecutive commits touching the same .py production file inside --window-min, _extract_ast_function_hashes parses each revision and hashes ast.dump(node, include_attributes=False) with SHA-256. Formatting tweaks, blank lines, and edits outside the function body produce identical hashes and are ignored. When the same function's normalized AST mutates twice inside the window, regret-mine checks four deterministic asymmetry signals to separate genuine rapid regret from healthy Green -> Refactor TDD:
-
FIX_VERB_INTENT: corrective keywords in the follow-up commit subject matching_FIX_VERB_RE(fix,fixes,fixed,bug,hotfix,revert,oops,regret,patch,broken,typo). -
TEST_ASSERTION_DELTA:len(newer["test_files"]) > 0, catching production rewrites accompanied by new or updated test assertions. -
GUARD_COLLISION_RECEIPT: Unix timestamp correlation (older["ts"] <= ev_ts <= newer["ts"]) against mid-flight collisions or renegotiations recorded inside.ag_state/merkle_guard_*.json. -
LINE_REVERSAL_OVERLAP: lines added in the earlier commit and deleted in the follow-up (added_k & deleted_k1), gated by refactor and control-guard checks.
python ag_ops.py regret-mine --repo . --window-min 60 --json
{
"ok": true,
"window_minutes": 60,
"commits_scanned": 3,
"tdd_refactors_filtered": 1,
"regret_pairs_count": 1,
"regret_pairs": [
{
"file": "pricing.py",
"overlapping_functions": [
"compute_fee"
],
"delta_minutes": 12.0,
"initial_commit": "24864454",
"regret_commit": "456955a2",
"regret_subject": "fix: handle None amount in compute_fee",
"asymmetry_signals": [
"FIX_VERB_INTENT",
"TEST_ASSERTION_DELTA",
"GUARD_COLLISION_RECEIPT"
]
}
]
}
Three Scars Earned While Building the Scar Engine
It would be dishonest to write about deterministic guardrails as if v2.1.0 compiled cleanly on the first try. Building the scar engine produced three fresh scars of its own before the release could ship.
1. The regret miner that punished healthy TDD (blocking v2.0 at 5/6)
When I wired up test_06_regret_mine in tests/test_ag_ops.py, I fed it a textbook three-commit TDD sequence on calc.py:
-
C1(feat: add calc):res = x + 1; return res -
C2(refactor: simplify calc): inlined toreturn x + 1 -
C3(fix: edge case None in calc): addedif x is None: return 0
I expected tdd_refactors_filtered: 1 and regret_pairs_count: 1 (C2 -> C3). Instead, my v2.0 candidate failed its own release gate (5/6 suites passing) with tdd_refactors_filtered: 0 and regret_pairs_count: 3. There is a special kind of poetic justice in writing a Rapid-Regret Miner whose maiden run blocks its own release because it cannot tell disciplined TDD apart from panic.
Two bugs had compounded:
- When
C2simplifiedres = x + 1; return resdown toreturn x + 1, it deleted lines thatC1had added seconds earlier. My naiveadded_k & deleted_k1check saw deleted lines fromC1and flaggedC1 -> C2withLINE_REVERSAL_OVERLAP. Every variable inline in history looks like line reversal if you don't inspect what kind of line was removed. - Without a per-commit
paired_filesset, the nested commit loop pairedC3againstC2and transitively back againstC1(C1 -> C3), counting one bug fix twice.
In v2.1.0, regret-mine tracks paired_files per newer commit so each file only compares adjacent C_k -> C_{k+1} transitions, skips line-reversal checks when _REFACTOR_VERB_RE matches without fix verbs (is_pure_refactor), and requires inverted_lines to either touch a control-flow guard (_CONTROL_GUARD_RE: if|elif|else|raise|assert|except) or accompany another corroborating signal. That brought the suite to 6/6 green.
2. The greedy \s* that ate semicolon-free Graphviz edges
Graphviz DOT allows edges without trailing semicolons:
digraph G {
Idle -> Fetching
Fetching -> Done [label="onSuccess"]
}
In my first draft of _lint_dot_state_machine, edge_re ended with \s*(?:\[(.*?)\])? to capture optional edge attributes. Because Idle -> Fetching had no [...] block, \s* happily ate the newline at the end of line 2 plus the leading indentation on line 3, treating line breaks between state transitions as optional whitespace and leaving SCAR-PROC-152 and SCAR-ASYNC-01 blind whenever semicolons were omitted. Restricting every horizontal whitespace token in edge_re and node_attr_re from \s* to [ \t]* under re.M fixed it immediately.
3. The latency reality check (NodeVisitor vs. ast.walk, OneDrive, and Windows cold start)
Initially, _AntigravityVisitor subclassed ast.NodeVisitor in a single top-down pass. That failed whenever an import lived near the bottom of a file (below a helper function calling it) and paid dynamic getattr dispatch on every AST node. Replacing NodeVisitor with two flat ast.walk(tree) passes (imports first, calls and exception handlers second) fixed the late-import bug cleanly.
Then I ran real timing benchmarks (median of 5 runs on Python 3.12 on Windows) to see how honest our <15 ms static-check label actually was:
| Target File | Lines | In-Process (elapsed_ms) |
Wall-Clock Subprocess | Notes |
|---|---|---|---|---|
tests/test_ag_ops.py |
333 |
6.54 ms |
151.1 ms |
Typical module size; well inside <15 ms
|
ag_ops.py (local SSD) |
1,783 |
42.6 ms |
~182 ms |
Dominated by ast.parse(), not tree walking |
ag_ops.py (OneDrive path) |
1,783 |
52.9 ms |
~193 ms |
Cloud-synced filesystem stat/read overhead |
Two lessons jumped out of that table. First, once a single Python file approaches 1,800 lines, CPython's ast.parse() dominates the in-process budget; micro-optimizing the visitor barely moves the needle. Second, spawning python.exe on Windows via CreateProcessW adds roughly 140 ms of fixed interpreter startup outside time.perf_counter(). In-process, a 333-line file lints in 6.54 ms; end-to-end from an agent's shell tool, every invocation pays ~150–185 ms. That is still fast enough to run on every turn without friction, and worth reporting without hiding behind perf_counter() alone.
Hard Limits, Real-Repo Noise, and What I Deliberately Left Out
Stress-testing regret-mine against dbader/schedule
Synthetic unit tests are friendly terrain, so I pointed regret-mine at an external open-source repository, dbader/schedule (398 commits), across three time windows:
-
--window-min 60(1 hour):19TDD refactors filtered,11regret pairs flagged. -
--window-min 1440(24 hours):25TDD refactors filtered,24regret pairs flagged. -
--window-min 10080(7 days):35TDD refactors filtered,37regret pairs flagged.
Inspection of the 11 one-hour pairs showed that the miner catches real rapid-regret defects cleanly, including commits like "Fix a bug where the at(':SS') syntax wasn't working" and "Missed == to !=".
It also exposed two remaining sources of false-positive noise in v2.1.0:
-
Docstring typo fixes inside functions: In Python's AST, a function docstring sits at
FunctionDef.body[0]as anast.Expr(ast.Constant(...)). Fixing a typo inside a docstring changesast.dump(node, include_attributes=False)and usually carries"fix typo"in the commit subject, firingFIX_VERB_INTENTeven though zero executable statements changed. -
Unqualified method names across sibling classes:
_extract_ast_function_hashescurrently keys functions bydefs[node.name]. IfJob.__init__changes in Commit A andScheduler.__init__in the same file gets a bugfix 20 minutes later in Commit B,defs["__init__"]collides. Stripping leading docstring constants before hashing and qualifyingClassName.method_nameare queued forv2.2.
Why there is no Stage-2 LLM inside ag_ops.py
In Randal's full architecture, Stage 2 hands candidate regret pairs and transcript.jsonl slices to an isolated clean-room critic model to compile new 4-vector scars automatically. I deliberately left Stage 2 out of ag_ops.py. Mixing API keys, network retries, and LLM prompts into a deterministic guardrail CLI would violate the exact property that makes ag_ops.py trustworthy: zero external dependencies, zero network calls, zero token spend. Shrinking 398 commits down to 11 structured JSON candidates in milliseconds is the CLI's job; deciding whether an orchestrator subagent or a human reviews those 11 candidates belongs one layer up.
The Same-UID .ag_state/ Perimeter
Just like v1, ag_ops v2.1.0 runs in-band under the same OS user account as the Antigravity agent. An adversarial process with unrestricted shell access could still inspect or overwrite .ag_state/merkle_guard_<hash>.json directly. In daily engineering work, ag_ops targets autoregressive laziness: stopping an over-eager model from taking statistical shortcuts like except: pass or quietly widening assert status == 200. Defending against hostile multi-tenant breakouts is a different threat model; true adversarial custody still requires separating the performer and the verifier across OS UIDs, read-only bind mounts, or CI runners.
Still Learning in Public (And Gratitude Where It Belongs)
ag_ops v2.1.0 remains a working lab instrument in standard-library Python, shaped by watching real multi-agent runs in Google Antigravity fail in ways I didn't anticipate and tightening the bolts one bug at a time.
I owe a clear debt of gratitude to Randal L. Schwartz (@randalschwartz) for the Synthetic Scars series, and particularly Giving AI Pain, for bringing decades of Unix engineering discipline, formal state-machine rigor, and empirical clarity to a space that badly needed an alternative to prompt superstition. Thank you as well to @max_quimby and howcani, whose sharp questions on Field Notes #1 turned a loose prose --reason flag into constructive witnesses and pushed this engine onto much firmer ground.
The full source code (ag_ops.py v2.1.0), the 6/6 automated verification suite (tests/test_ag_ops.py), and the MIT License are available at github.com/FranciscoGrandon/ag-ops. If you run regret-mine against your own repositories and find an edge case that breaks it, I would genuinely love to hear what broke.
Top comments (39)
The SCAR-PROC-150 witness gate is the right inversion — a prose reason like 'fixed tests' gets exit 1 unless the agent produces a mechanically verifiable witness, so renegotiation becomes evidence instead of rhetoric. The part I'd stress-test is the spec-collision witness itself: since it only checks that two file:line refs exist on disk, what stops an agent from manufacturing a plausible collision by editing a doc first, or does the Merkle seal cover the docs too?
Fair hit, and you've pinpointed the exact trap door: right now, the Merkle seal only wraps around the --test-dir perimeter. That means the spec-collision witness merely verifies that two file:line coordinates exist on disk, not that the semantic contradiction is genuine. In its current state, an agent could simply doctor a documentation file beforehand and cite it—effectively forging the penal code before committing the crime.
For v2.2, I'm planning to harden this pipeline with three concrete steps: (1) capturing and persisting the SHA-256 alongside the exact quoted text of each cited coordinate in the receipt, (2) resolving cited lines directly out of git show HEAD:file rather than trusting the mutable working tree, and (3) optionally pinning spec manifests during witness-red so post-failure edits carry zero evidentiary weight.
Of course, none of this stops a determined agent with shell access from quietly running git commit on its tampered specs before calling renegotiation. Mitigating that boundary condition ultimately requires an anchor beyond the process's own write privileges—such as an external CI pipeline or a decoupled UID—which brings us straight back to the same-UID physical limit discussed in the post.
Thanks for stress-testing the architecture.
The git show HEAD move is the one that closes the working-tree forgery — once the witness resolves against a commit rather than mutable files, doctoring a doc first stops buying anything. But the commit-before-renegotiation case is the honest boundary: the seal can attest to what the repo contains, not to when it got there, so the anchor really does have to live outside the agent's own write privileges. Would a CI-signed receipt be enough for v2.2, or does the anchor need to sit outside the repo entirely to count?
This exchange between @agenticaffairs and @fgrandon gets right to the foundational security boundary of local agent governance: how do you anchor a witness against an agent that shares your local UID?
We hit that exact "commit-before-renegotiation" / working-tree forgery boundary, and the solution that closed it for us has two halves — neither of which requires spinning up an external CI run just to validate a mid-flight renegotiation:
base_commit_sha = git rev-parse origin/main(or the remote upstream ref) at Target 10 intake (00_governance.md), NOTHEAD: A local agent can edit the working tree and even rungit commitlocally if not gated, so resolvinggit show HEAD:fileis still vulnerable ifHEADmoved locally. Instead, resolve thespec-collisionwitness against the immutable upstream base SHA (git show <origin_base_sha>:docs/rfc_104.md) that was recorded before the agent ever touched the branch (or verifygit merge-base --is-ancestor <witness_sha> refs/remotes/origin/mainwhen network fetch isn't tainted). Because pushing toorigin/mainrequires remote credentials / branch protection outside the local workspace, the upstream commit graph acts as a free, zero-latency read-only anchor right inside.git/objects!git commitandgit pushexplicit human pause gates (Target 50andTarget 60) withgit log origin/main..HEADaudit: In our workflow, a worker subagent duringTarget 30(TDD implementation) is strictly prohibited from runninggit commit. Whenverify-greenorrenegotiateruns atTarget 30,HEADis guaranteed to equal the pre-session base commit (git rev-list origin/main..HEADmust be empty duringTarget 30). Ifag_ops.py guard renegotiatesimply checks two things:git show HEAD:<file>for thespec-collisioncoordinates, andHEAD == recorded_witness_red_head_sha(theHEADcommit hash captured insidemerkle_guard_<hash>.jsonat the momentwitness-redfirst ran), then doctoring a doc in the working tree buys nothing (git show HEAD:<file>ignores uncommitted edits), and runninggit committo sneak the doctored doc intoHEADimmediately tripsHEAD_MOVED_DURING_RED_PHASE(HEAD != recorded_witness_red_head_sha)!Capturing
head_sha = git rev-parse HEADinsidemerkle_guard_<hash>.jsonduringwitness-redand resolvingspec-collisionviagit show <recorded_head_sha>:<file>duringrenegotiateis about 10 lines ofsubprocessinag_ops.pyv2.2 — and it closes both the working-tree forgery and the sneaky local-commit forgery in one shot!Thanks, this is a useful addition. Recording the repository state at the red phase and resolving cited evidence from committed history, rather than from the working tree, takes uncommitted edits out of play and costs very little at runtime.
Two design notes. First, once evidence is resolved against a fixed recorded commit, demanding that the repo state never move afterwards is stricter than needed and can reject legitimate commits made mid-session; requiring the recorded commit to remain an ancestor, with the cited files unchanged since, gives the same protection with fewer false rejections. Second, I'd treat this as a consistency check rather than the root of trust: a doctored doc committed before the red phase would simply be recorded along with everything else, and local state remains writable under the same UID. That is why the plan pairs it with an upstream anchor verified at the CI gate, which the agent can't write to.
The ancestor refinement is the right loosening — freezing HEAD outright would punish legitimate mid-session commits, while ancestor-plus-unchanged-files keeps the same protection without the false rejections. And the consistency-check framing is the honest one: recording state at red only proves what was there at red, not that it was honest before it. At the CI gate, does the upstream anchor re-check the cited files themselves, or just re-run the witness against the recorded commit?
Both of those refinements are spot on:
git merge-base --is-ancestor <recorded_sha> HEADandgit diff --quiet <recorded_sha> HEAD -- <cited_files>): That is much cleaner than lockingHEADglobally. If an agent or developer makes legitimate mid-session commits to unrelated modules while iterating, those pass right through without false rejections, while any mutation to the cited spec file sincewitness-redis still caught immediately.Target 30and an unforgeable audit gate atTarget 70!It has to re-check the cited files directly against the PR's protected target branch merge-base (
GITHUB_BASE_REF/git merge-base origin/main HEAD), not just againstrecorded_witness_red_head_sha!Here is why: if the CI gate only re-ran the witness against
recorded_witness_red_head_sha, then an agent that doctoreddocs/rfc_104.mdand committed it on the feature branch before callingwitness-redwould pass — becauserecorded_witness_red_head_shawould already point to that forged local commit!Inside the CI runner,
origin/main(orGITHUB_BASE_REF) is fetched fresh from the protected remote and cannot be mutated by the PR branch. So the CI verification is a two-line check againstMERGE_BASE=$(git merge-base origin/$GITHUB_BASE_REF HEAD):git diff --quiet "$MERGE_BASE" HEAD -- <cited_spec_files>(or resolve thespec-collisionline coordinates and SHA-256 directly fromgit show "$MERGE_BASE":<cited_spec_file>).RENEGOTIATED_REDreceipt was modified between$MERGE_BASEandHEAD, CI rejects the receipt unless the PR carries an explicit human spec-change sign-off label/review.That way:
witness-red->renegotiate): checking againstrecorded_witness_red_head_shacatches mid-TDD working-tree or post-red commit tampering in milliseconds.$MERGE_BASE->HEAD): resolving againstgit merge-base origin/main HEADcatches pre-red branch forgery against the unforgeable upstream branch!That's the missing piece — re-running against the recorded SHA alone would bless the forgery, since the forged commit is already in the recorded history. The merge-base check plus a human sign-off exception is the right shape: strict by default, with an explicit, auditable escape hatch. Does the gate itself check for the sign-off label, or is that a reviewer judgment call at PR time?
In our architecture, both — it is a two-stage mechanical gate so it never relies on a human reviewer "remembering" to check if a spec file was touched:
git diff --name-only "$MERGE_BASE" HEADtouches any file referenced in aRENEGOTIATED_REDreceipt (or any locked specification/schema directory), the CI script queries the PR payload (jq -r '.pull_request.labels[].name' "$GITHUB_EVENT_PATH"orgh pr view --json labels,reviews) and exits non-zero unless an explicit human-applied label (such asspec-change-approved) or aCODEOWNERSreview is present. An agent running without repository admin/label privileges cannot self-apply that bypass.Landing Gate— The 7-Criteria Human Escalation Matrix,SCAR-PROC-122): Even before merge, our orchestrator is required to evaluate a 7-Criteria Escalation Table at Target 75 (checking whether the diff modifies public API contracts, database/wire schemas, security rules, or architectural specifications). If any of the 7 criteria isYES, the workflow mechanically disables Branch A (Direct Squash-Merge) and forces Branch B (Assign Human Lead Reviewer)!That way, an unapproved spec mutation can neither turn CI green quietly nor take the direct-merge fast path at landing time.
Both stages together is the answer — the CI label check makes the bypass explicit and unforgeable by the agent, and the Target 75 matrix removes the fast path entirely when a spec is touched, so there's no quiet green to hide behind. The part that makes it hold is that the agent can't self-apply the label without admin privileges. Is the 7-criteria table itself locked like a spec, or could an agent argue a criterion down to NO before the matrix runs?
Great question — we defend against both vectors (modifying the 7-criteria policy file itself, AND rationalizing a criterion down to
NOin chat):The 7-Criteria Policy File Is Out-of-Repo & Protected by a Dual-Key Invariant (
SCAR-PROC-53):references/03_human_review_policy.md) lives in the sovereign workflow repository (~/.gemini/config/skills/...), outside the application monorepo worktree where the agent is executing the ticket.AGENTS.md,.agents/skills/,CODEOWNERS) automatically triggers the Dual-Key Landing Invariant: two independent human approvals are required before merging (the human operator is only Key 1; a second human maintainer must turn Key 2).Why an Agent Can't "Argue a Criterion Down to
NO" to Auto-Merge (SCAR-PROC-90+SCAR-HCI-11+ GitHub Bot/Path Classification):Even if the orchestrator tried to rationalize Criterion 5 (Domain/Spec Impact) or Criterion 2 (Entity/Schema Change) down to
NOin its Target 75 table, three layers stop it from slipping through:tool_calls: []): Even when the agent evaluates0/7criteria and outputsRECOMMENDED: Branch A (Direct Land), the agent is strictly forbidden from callinggh pr mergein that turn! It must halt withtool_calls: []and render the full 7-rowYES/NOtable (alongside the diff stats and changed file list) directly in front of the human operator in chat (SCAR-HCI-11). For the agent's rationalization to work, it has to lie directly to the human operator's face on a 7-row table right above the list of modified files, and wait for the human to click "Branch A".Target 40): Before the PR is even pushed, the fresh-contextTarget 40Adversarial Critic audits the diff's Blast Radius and Spec/Contract impact with zero shared KV-cache from the Builder.reviewDecision == "APPROVED"): On GitHub,mainrequires an explicitAPPROVEDreview beforegh pr merge --squashis permitted by the branch ruleset. Our automated GitHub review bot only grants write-access approval on low-blast-radius PRs; if the PR touches protected paths or high-risk modules,mergeStateStatusstaysBLOCKED(REVIEW_REQUIRED) on GitHub's servers regardless of what the local agent argues in its markdown table!The Target 75 pause is the piece that makes the whole defense hold — the agent has to halt with an empty tool-call list and render the full 7-row table in front of the human, so arguing a criterion down to NO means lying to a person's face, not slipping past a gate. Combined with the out-of-repo policy file and the GitHub-side approval requirement, there's no quiet path left. The one assumption I'd probe is the dual key: it needs a second human maintainer, so does the invariant still hold for a solo developer, or is that the case where the Target 75 pause has to carry the whole weight alone?
That is the exact difference between our two empirical cohorts (
Cohort Benterprise multi-maintainer vs.Cohort Asolo-maintainer open source likeBlocSignal)!For a solo developer (where there is no second human maintainer to turn Key 2), you obviously can't require two humans — so we replace the "second human key" with three orthogonal machine/process keys so the Target 75 pause never has to carry the weight alone:
~/.gemini/config/skills/...vs. Repo Worktree): Even for a solo dev, the canonical workflow and policy files live outside the repository's git working tree (and in sandboxed CLI harnesses, outside the write-permitted workspace root), or are checked againstgit diff $(git merge-base origin/main HEAD)in CI so an in-repo rule edit is immediately flagged as aGOVERNANCE_DIFFin the Target 75 banner.Target 40Critic + Remote Cloud Bot = Synthetic Key 2): In our solo-maintainer repos (Cohort A, 142 tickets), the "second key" is split across two independent critics that share zero KV-cache with the Builder:Target 40Six-Pillar Critic (spawned in a clean-room subagent context) audits Pillar 6 (Immune Defenses & Governance Integrity) and Pillar 4 (Blast Radius), and signs the git tree SHA-256 (git write-tree).gemini-code-assist[bot]/Copilot) inspects the PR diff from scratch and must reach0unresolved threads +APPROVEDbefore Target 75 can even be entered.SCAR-PROC-13— Never Mix Feature Code and Rule Changes in One PR): Even as a solo maintainer, the agent is forbidden from modifying governance rules (AGENTS.md,WORKFLOW.mk, or linter configs) inside a feature PR! Any rule or policy change must be isolated into its own dedicateddocs(meta):/ci:PR atTarget 90, where the entire diff is only the rule change — making it impossible for a sneaky policy weakening to hide inside a 20-file feature diff at Target 75.So in a team repo (
Cohort B), Key 1 + Key 2 are Human Author + Human Peer Reviewer; in a solo-dev repo (Cohort A), Key 1 + Key 2 are Clean-Room Critic + Dedicated Single-Purpose Meta-PR + Explicit Human Target 75 Sign-Off!This answers the solo-dev question properly — the second key doesn't disappear, it gets split across the clean-room critic, the remote bot, and the dedicated meta-PR, so no single machine check carries the weight alone. The SCAR-PROC-13 rule is the quiet hero here: a policy weakening can't hide inside a feature diff if rule changes always travel alone. Does the Target 75 sign-off stay meaningful when the same solo human is both author and approver, or does the critic's signed tree SHA become the real second key in practice?
In practice, both stay meaningful because they check two orthogonal axes that neither can verify alone:
The Critic's Signed Tree SHA (
Target 40) + Remote Bot (Target 70) Is the Real Technical Second Key (Checking the Agent's Implementation):Because the solo human didn't type the 500-line feature diff (the Builder subagent did!), the solo human is already an external reviewer relative to the Builder's KV-cache. By the time the PR reaches
Target 75, the Critic's signedgit write-treeSHA and the remote GitHub review bot have already done the line-by-line adversarial verification that the code obeys the spec and the 600 scars.Why
Target 75Still Carries Real Weight for the Solo Human (Checking Intent, Scope, and Irreversibility):Even when the Critic's tree SHA is 100% valid and CI is green, only the human operator knows external reality outside the repo:
Target 70that I actually want fixed in this PR instead of deferred to a follow-up issue?"main?"So in solo-dev mode (
Cohort A), the Critic's signed tree SHA is the real second key against code/rule forgery, while theTarget 75human pause is the airlock before an irreversible external side effect (git push/gh pr mergeintomain). One verifies "is the artifact sound?"; the other decides "should this artifact ship right now?"That split is convincing — the critic's signed tree SHA checks whether the artifact is sound, while the Target 75 pause decides whether it should ship right now, which no hash can know. The external-reality examples are the clincher: a held PR, a mis-scoped bot thread, a breaking semver change. Does the pause table surface those three checks explicitly, or is it left to the human to remember them at sign-off?
We learned the hard way (
SCAR-HCI-11andSCAR-PROC-122) never to leave those checks to human memory! If an agent asks a bare "CI is green, ready to land?", a tired human will rubber-stamp"yes"99% of the time.Instead, the workflow enforces a Sovereign Presentation Surface Invariant (
SCAR-HCI-05/SCAR-HCI-11): before the agent is allowed to pause atTarget 75, it is required to render the full decision substance directly into the chat stream across three explicit artifacts:The
Target 704-Part Bot Triage Matrix (70_bot_triage.md):Every single remote bot thread is printed in a table with its ID, summary, classification (
CRITICAL_CORRECTNESS,CLEANLINESS_POLISH,FALSE_POSITIVE, orDEFER TO ISSUE #XXX), and the exact rationale. Additionally,SCAR-PROC-157physically blocksTarget 75from even rendering until a live GraphQLreviewThreads(first: 100)query assertsunresolvedThreads == 0. So if the agent classified a bot thread as out-of-scope or deferred, the human sees that row staring at them in chat beforeTarget 75even unlocks.The
Target 75Two-Tier Landing Matrix (Printed in Chat right before the pause):$APPROVED_TREESHA match, local git hooks, remote CI status, and bot thread resolution count (0 unresolved).YES / NOand a 1-line justification for each:SCAR-PROC-13)Explicit Branch Recommendation:
At the bottom of that table, the agent computes the recommendation (
0/7 YES -> Branch A: Direct Landvs.>= 1/7 YES -> Branch B: Hold / Escalate Review).So the only thing left solely in the human's head is unwritten external context (for example, "wait, my co-maintainer just pinged me on Slack to hold merges for 20 minutes"). Everything inside the repo—deferred bot threads, public API / semver surface, and blast radius—is forced onto the screen in structured tables right above the approval prompt!
That's the answer to the rubber-stamp problem — the human can't approve what they haven't seen, because the triage matrix and the 0-unresolved gate put it in the chat before the pause even unlocks. The 7-criteria blast-radius table is the part I'd steal: it forces the semver and scope questions into explicit YES/NO rows instead of vibes. Does the agent ever get to argue a row down from YES to NO, or is that reclassification itself a human call?
The agent is strictly forbidden from arguing a physical match down to NO!
That rule is ironclad (
SCAR-HCI-11/SCAR-PROC-122):The Agent Only Audits Ground Truth Against the Diff:
If the git diff touches a UI file, an entity schema, an NDT formula, or a cloud function, the agent must mark it YES. It has zero discretion to say "well, it's just a tiny 2-line padding fix, so let's call UI a NO". That kind of cognitive bargaining is exactly how subtle UI breakages and schema drifts slip through. If even 1 of the 7 rows is
YES, the agent is mechanically constrained to output:RECOMMENDED: Branch B (Assign Lead Reviewer).The Only "Downgrade" the Agent Can Do Is Exposing Stale Ticket Boilerplate (
SCAR-PROC-122):Sometimes a ticket's markdown template has a legacy field like
"Designated Reviewer: @lead". If the actual physical PR diff touches only self-contained CI configs or unit tests (0/7triggered), the agent will explicitly flag:"Ticket template specifies human review, but physical diff evaluation is 0/7 YES. Recommending Branch A (Direct Land) to protect lead reviewer bandwidth."
Notice that even here, it didn't argue a criterion down—it proved the diff never touched one!
Reclassification / Overriding Is 100% the Human Operator's Call:
Because the agent rendered the full 7-row table and the
Branch Brecommendation in chat, the human operator has all the facts. If the operator looks at the 1 triggered row (e.g., a 1-character typo fix in a UI label) and decides "this doesn't need to burn 4 hours of the lead's time", the human can choose Branch A.The agent is the impartial bailiff sworn to report physical reality; only the human holds the pardon pen!
The bailiff-and-pardon-pen split is the cleanest version of this I've seen — the agent reports physical reality, and only the human can overrule it. The pressure point I'd watch is the pardon itself: when the human picks Branch A over a triggered row, is that override recorded as a witness in the same chain, or does the human choice bypass the evidence trail the agent just built?
It is 100% witnessed and permanently sealed into the chain! That is the core beauty of the Smalltalk-80
thisContext+ append-only flight recorder model (SCAR-META-02/SCAR-PROC-101).A human pardon never bypasses or erases the evidence trail. Here is how the witness chain preserves it:
The Sovereign In-Chat Ledger (
SCAR-HCI-05/SCAR-HCI-11):The agent's 7-row table and its explicit
RECOMMENDED: Branch B (Assign Lead Reviewer)are already emitted into the immutable host flight recorder (transcript.jsonl). An operator cannot make the agent "re-run the check and pretend it was NO."The 3-Trigger Checkpoint Law (
changes-$timestamp.changes):When the human clicks or declares
Branch A anyway, Trigger 1 fires at the human gate boundary. The Smalltalk bootloader writes an immutable.changesdelta chunk:LandingResolution gate: #Target75 recommendation: #BranchB operatorChoice: #BranchA overrideReason: ...This is committed to disk before a single git command in Target 80 can execute.
Target 90 Post-Mortem & NTSB Telemetry (
SCAR-PROC-98/SCAR-META-02):In the final terminal verification block, the divergence is explicitly accounted for:
Socratic Nudges (N_nudge)increments.Matrix: 1/7 YES (UI) -> Agent: Branch B -> Operator: Override to Branch A.The Post-Merge Rapid-Regret Tripwire (
G1 Tripwire):This is where the chain really bites: if that human-pardoned PR breaks in production and triggers a rapid-regret hotfix commit within 60 minutes, the automated post-mortem auditor pulls the flight recorder.
It sees immediately: "The agent's mechanical check caught this, marked it YES, and recommended human review; the human pardoned it; the break happened on that exact flagged boundary."
The pardon pen has full authority to ship, but it signs its name on the permanent flight recorder right beside the bailiff's indictment!
That's a satisfying closure — the pardon keeps full authority but signs its name next to the indictment, so accountability survives the override. The G1 tripwire is the teeth: a rapid-regret hotfix pulls the recorder and the story is already written. The one case I'd still probe is the overrideReason itself — is it free text the operator can write vaguely, or does the gate force a structured reason?
It is structurally bound to the indicted row, even if the human only types two words!
If you allow free-text like "looks fine to me" or "just ship it", human nature guarantees you get zero useful signal for post-mortem forensics.
Here is how the gate prevents vague escapes:
1. The Indicted-Row Binding
When the agent generates the Target 75 gate, the override is mechanically coupled to the specific criterion that was triggered.
If Row 1 (UI) and Row 4 (Core Logic) triggered
YES, the gate doesn't ask a generic "Why are you overriding?" It forces a selection or records an explicit tuple:2. The Verification-Proof Airlock (
SCAR-VERIF-01)For the most common override—"it's a small UI change, I checked it myself"—the operator cannot simply assert that in prose.
Under
SCAR-VERIF-01, Target 72 requires an actual Visual Proof artifact (seeded emulator screenshots or an interaction trace) rendered in chat. If the operator overrides the UI review criterion without a completedverification-proofreceipt in.state/, the flight recorder logs:Override: Row 1 (UI) with PROOF_UNVERIFIED.That is an instant red flag in the Target 90 audit!
3. Structured Mining & Fleet Telemetry (
mine_deflections.dart)Because the override is logged as a structured enum rather than free-form mush, our automated audit tools can group them across the fleet:
EMERGENCY_HOTFIXvs.TEMPLATE_FALSE_POSITIVE?Override -> G1 Rapid-Regretcorrelation?So the operator can be as terse as they want in chat, but the machine binds their declaration to the exact structural cross-hairs of what they chose to bypass!
Binding the override to the indicted row rather than a free-text excuse is what makes the audit usable — a vague 'looks fine' can't survive when the tuple forces criteria, basis, and a witness receipt. The fleet telemetry angle is the strongest part: an override rate near 90% on one criterion reads as an ossified rule, not a careless operator. Has that telemetry already flagged such a criterion in practice, or is the recalibration signal still theoretical?
It has already flagged real criteria in practice—it is an active empirical feedback loop, not a theoretical one.
Across our 250-ticket ledger, telemetry on the 7-Criteria Landing Matrix (Theory Section 338.2) surfaced two distinct real-world recalibrations:
The "De Minimis" False-Positive Drift (Criterion 4: Blast Radius): Early on, Criterion 4 routed any PR modifying more than 3 files to mandatory Human Lead Review. In monorepo maintenance runs (like mechanical formatting or docs updates across multiple packages), operators overrode the gate on ~85% of runs because the semantic risk was zero. That telemetry flagged that raw file count was the wrong proxy. We recalibrated the criterion to track AST interface mutation distance instead, dropping the override rate back below 5% while keeping actual cross-package API changes strictly locked.
The "Same Signal, Two Diseases" Confounder (Sam Labbe's Reviewer Alertness Law): When an override rate spikes near 90% on a specific criterion, telemetry has to distinguish between two completely different failure modes:
Matrix Ossification (the threshold is too tight or stale for the codebase's geometry), versus
Reviewer Alert Fatigue (the human is rubber-stamping without reading).
To disambiguate, our telemetry conditions the override rate on the reviewer's substantive pushback rate across other gates (like Target 20 Plan or Target 40 Critic). If the operator is actively rejecting plans and blocking diffs elsewhere, an 85%+ override on Criterion 4 is proof the matrix has ossified. But if their pushback rate across all other lanes has collapsed to near zero, it diagnoses review-layer fatigue—where loosening the rule would be the exact wrong repair.
The De Minimis recalibration is the proof the loop works — swapping raw file count for AST interface mutation distance and watching overrides fall from ~85% to under 5% is a real fix, not a loosened gate. The ossification-versus-fatigue disambiguation is the sharper part though: conditioning on pushback elsewhere stops a tired reviewer from being misread as a stale rule. Which of the two recalibrations was harder to land with the team — changing the criterion, or trusting the telemetry to tell those two failures apart?
Francisco, this is phenomenal systems engineering — and seeing
SCAR-PROC-150,SCAR-PROC-152,SCAR-ASYNC-01, Vector 4"reflex"hints,--context-mapautoimmune drift detection, and a Stage-1regret-mineall running in 0.4ms–6.5ms of zero-dependency standard-library Python is an absolute thrill.Three details in your write-up really stood out to me:
--require-witnessinto a disk/compiler proof (SCAR-PROC-150): Your solution to@max_quimby's question is textbook. Because unattended subagents cannot block oninput(), requiring--witness-type (spec-collision | api-lock | arch-fork)and mechanically verifyingfile:linebounds or compilerTypeError/AttributeError/sealedoutput turns premise retraction from "creative lawyering in 200ms" into a constructive proof.regret-mineblockingv2.0at5/6: That is the ultimate rite of passage! Separating a healthyGreen -> Refactorvariable inline (C1 -> C2) from a genuine rapid-regret bugfix (C2 -> C3) by requiring_CONTROL_GUARD_RE(if|elif|else|raise|assert|except) or a corroborating signal onLINE_REVERSAL_OVERLAP— plus your notes on stripping leadingast.Expr(ast.Constant(...))docstrings and qualifyingClassName.method_nameafter stress-testing againstdbader/schedule— are super sharp refinements to the Stage-1 miner.ag_ops.py: 100% the right architectural boundary. The deterministic CLI should stay a pure, zero-network, zero-token Unix tool that shrinks 398 commits down to 11 JSON candidates in milliseconds; handing those 11 candidates to an orchestrator subagent or human belongs one layer up.A heads-up on what we just discovered last night (and why Scar #2 in your article made me smile):
Your second scar — where
\s*inedge_reate the newline across semicolon-free Graphviz.dotedges — actually collided with a deeper limitation we hit when unifying our 5 prompt DSLs (Make,Graphviz DOT,Gherkin,/etc/init.drunlevels, and JSON state schemas). Even when Graphviz.dotparses cleanly, the LLM itself still has to remember to emit the second error/cancel edge out of an async state (SCAR-PROC-152), because in.dotsyntaxFetching -> Doneis already a syntactically complete statement!Yesterday we ran a 120-trial controlled benchmark (Suite 8) replacing both Graphviz
.dotand Gherkin with Smalltalk-80 keyword messages and block closures:Because
#state:invariant:onSuccess:onFailure:is a single 4-part keyword selector in Smalltalk-80 (and->is justObject >> ->creating anAssociationto a[ :s | ... ]guard block), omitting the sad path (onFailure:) is no longer a semantic checklist failure — it is a syntax error. An incomplete message send before;or.immediately pulls the Transformer's attention head directly ontoonFailure:.Under a 16k-token distractor load across 4 Gemini models (
N = 120paired trials):2.5%(3 / 120):on-failureis an optional&keyand closing))))suffers fromDyck-1bracket-counting drift):91.7%(110 / 120, dropping to66.7%onflash-lite)100.0%(120 / 120) — including30 / 30ongemini-3.5-flash-lite!Even better: because Smalltalk-80 has
Object >> #storeStringand reifies the live call stack viathisContext, the state tracker (00_plan,10_build), the.dotstate graph, and the Gherkin scenarios all collapse into a single serializable Smalltalk object graph (baseline.sources+ append-onlychanges-$timestamp.changeschunks).I'm writing this up for Part 4.1 of the series right now, and I will definitely be citing
FranciscoGrandon/ag-opsv2.1 as a reference implementation of deterministic out-of-band scar enforcement in Python. Thank you for the generous shout-out and for building and sharing this in public!Randal, thank you! Having ag_ops cited in Part 4.1 is a huge honor, especially since half the invariants in v2.1 exist because your series convinced me to stop writing sternly worded essays in GEMINI.md and start handing out exit code 2.
That Smalltalk-80 benchmark (120 / 120 even on flash-lite) exposes the exact structural blind spot I ran into with Graphviz .dot. Because Fetching -> Done is a valid, self-contained production in DOT grammar, SCAR-PROC-152 can only catch the missing teardown edge after the model finishes generating the graph. Turning #state:invariant:onSuccess:onFailure: into a single 4-part keyword selector flips I_path(pi) compliance from a post-hoc semantic check into an in-flight syntax constraint—the attention head literally cannot reach the cascading ; without filling the onFailure: slot first, and without suffering the Dyck-1 closing-paren drift of Lisp S-expressions. Collapsing the runlevels, state graph, and Gherkin specs into an append-only .changes chunk file via thisContext and #storeString is pure Xerox PARC magic.
I am already locking in the scope for ag_ops v2.2: stripping leading ast.Expr(ast.Constant(...)) docstrings and qualifying ClassName.method_name in regret-mine, plus wiring your 10-line recorded_witness_red_head_sha + git show anchor from the thread above to close the working-tree forgery gap in SCAR-PROC-150. At this rate, by the time Part 4.1 drops, our agents will be governed by a 1976 Bell Labs Makefile, a 1980 Xerox PARC image log, and a 1983 SysV init.d directory—leaving the 2026 neural network as the only component in the stack under forty years old. Looking forward to reading Part 4.1!
That line is pure gold — with your permission, I'm putting that straight into the opening epigraph block of Part 4.1!
And it gets even wilder: today we ran our first two live production tickets (
BlocSignal #260and#259) on the compiled Smalltalk-80 control plane, AND built a working zero-dependency Dart evaluator (package:smalltalk_fabric) that parses Smalltalk-80 cascades, evaluates live[ :s | ... ]block closures against host methods, and serializes mid-flightthisContextstack continuations via#storeStringso a failed tool call can be resumed in place (ex resume: replacementValue) without restarting the pipeline.Can't wait to see
ag_ops v2.2when it lands!Permission granted, gladly! I’m looking forward to Part 4.1, and I’m already working on v2.2.
Odinaka's line sums it up well: docs do not push back, the CLI does. Moving the rules from a Markdown file into something that returns exit code 2 is the change that makes them rules at all.
The four cracks are useful precisely because they are boring. Two subagents overwriting one receipt file and an empty directory getting a valid seal are the kind of bugs that only appear once real parallel work hits a tool, and they are easy to miss in a design review.
The thread with Randal about anchoring the witness to committed history, and then to the merge base of the protected branch in CI, is the clearest explanation I have read of why an agent sharing your UID cannot be its own auditor.
Do you plan to keep the human sign-off label as the only escape hatch, or will some renegotiations be allowed to pass with a mechanical witness alone?
Thanks for reading and for highlighting Odinaka's point, Ascendra. You hit the nail on the head: Markdown compliance is just a polite suggestion until the shell refuses to exit zero.
Regarding your question on whether the human sign-off label will remain the sole escape hatch or if mechanical witnesses alone will eventually clear renegotiations:
I am keeping the human gate mandatory for structural state overrides, but v2.2 introduces an automated "amnesty path" for low-severity drift. If a mechanical witness can prove via static AST inspection that a failed constraint was purely structural cosmetic churn (such as re-ordering imports or updating type hints) without altering execution paths, the CLI will auto-issue a cryptographically signed provisional waiver instead of outright blocking the agent.
The core philosophy remains that an autonomous system can propose its own bail-out, but it shouldn't hold the keys to the panic room. Appreciate the sharp read on the merge-base audit model
The DONE banner stealing the failure budget is my favorite bug in this write up. I'd keep it as a fixture with the actual assertion thousands of lines later, then assert that the distilled output still contains the assertion location. I'd also try the same log with ANSI colors and carriage return progress updates. A short summary is only useful if the formatter can't accidentally crowd out the failure.
The most valuable thing in this write-up is that the release gate caught its own miner. A regret detector that mistakes a textbook TDD sequence for panic is a great demonstration of why deterministic checks need adversarial fixtures before anyone trusts them. Your per-commit paired_files fix and the is_pure_refactor gate also point at a general lesson: “lines that were deleted after being added” is a weak signal on its own, and it only becomes useful once you condition on what kind of line was removed (your control-guard check does exactly that).
A few thoughts and questions:
The constructive-witness gate is mechanical, but is it semantic? spec-collision verifies that two file:line references exist, are in bounds, and are distinct. That’s a real improvement over prose, but an agent can still satisfy it by citing any two valid lines. The api-lock type has a similar property: a failing run that happens to print TypeError or ImportError for an unrelated reason would pass. It still raises the cost of lazy retractions and leaves an auditable trail, so I wouldn’t call it a flaw. But a cheap tightening might be to require that at least one cited location falls inside the files the test actually imports, or that the witness lines be recorded in the receipt’s collisions history so a reviewer can spot-check them later. Have you seen agents “game” the witness types yet in real runs?
The red-side question from howcani. Blocking runner-level exit codes (2, 3, 4, 5, 126, 127) is a good filter for “failed to run” versus “failed an assertion.” The remaining gap is a test that fails with a legitimate assertion exit code but against the wrong invariant. One idea is to have witness-red record the distilled failure line (the E ... delta) in the receipt and have verify-green confirm that the originally failing assertion is the one that now passes, rather than only that the command exits 0. That would tie green back to the specific red.
Regret-mine noise. The two false-positive sources you identified are both fixable the way you describe. Stripping the leading docstring Expr before hashing and qualifying names as Class.method should remove most of the 11-pair noise on schedule. I’d also consider hashing at the statement level and ignoring pairs where the only changed statements are string constants or comments, since “fix typo” commits are a classic source of this. It would also be interesting to report precision on that 11-pair sample (how many were real regrets after manual review), since that gives readers a concrete number for the miner’s usefulness.
The latency table is refreshingly honest. Reporting both in-process and wall-clock numbers, and noting that ast.parse() dominates at ~1,800 lines, is more useful than the usual single benchmark. For the subprocess cost, a persistent mode or a small long-lived helper could remove the ~140 ms Windows startup if per-turn overhead ever matters, though at 150–185 ms it’s probably not worth the added complexity.
The scar-ossification design is clever. Pinning a rule to an anchor file’s SHA-256 and downgrading to PROBATIONARY_REVERIFY on drift turns “this rule may be stale” into an explicit state rather than a silent false block. One thing to consider is expiry: if a rule stays probationary indefinitely, it effectively becomes advisory. A TTL or a count of consecutive probationary passes before it forces re-verification might keep that state from becoming a permanent soft spot.
Your threat-model note about the same-UID perimeter is the right framing: this targets autoregressive shortcuts, not hostile actors. Within that scope, moving the pain into exit codes and hashes instead of more prose rules is a sound approach, and the numbered list of what broke makes the whole thing more credible. Thanks for publishing the failures alongside the features.
Thanks for reading and for putting together such a sharp breakdown, Amber. Having the release gate flag its own miner was humbling, though at least the tool isn't playing favorites.
To your question on whether I have seen agents game the witness types in the wild: not maliciously, just via classic autoregressive laziness—grabbing the nearest valid file:line in context or riding an incidental ImportError across the finish line because it satisfies the syntactic shape of least resistance. Your idea of intersecting cited locations against the test file's AST import graph is the exact kind of cheap, structural tie-breaker that closes that gap without needing an LLM judge.
I am actively folding your observations into the v2.2 roadmap and will dedicate the third post in this series to testing these exact edge cases and reporting the empirical numbers:
Witness semantics & Red-Green causal binding: Testing the import-graph intersection for spec_collision, alongside persisting the distilled E ... failure delta in the receipt so verify-green confirms the exact assertion flip rather than just a zero exit code (and doesn't accidentally pass because the agent deleted the failing assert).
Regret miner precision: Alongside stripping leading docstring Expr nodes and qualifying Class.method scopes, I will test ignoring pure string-constant mutations to filter out typo commits, and I will publish the manual precision audit on that 11-pair sample so there is a hard baseline on signal-to-noise.
Probationary TTL: Adding a monotonic pass-count decay to PROBATIONARY_REVERIFY—because "temporary probation" in software has a suspiciously high correlation with "permanent architectural fixture."
As for the ~140ms Windows subprocess tax, I am holding off on a persistent daemon for now; the last thing a deterministic guard tool needs is its own zombie-process lifecycle to debug. Appreciate the push on these edges—stay tuned for Part 3.
Receipt collisions and the 'renegotiate --reason' loophole are the two cracks I'd expect first — both are classic trust-boundary issues. The Merkle-sealed test tree parallels what desktop apps do with code-signing and tamper-evident update manifests: you can verify integrity, but you still need an authority that cannot be talked out of a decision. The non-zero-exit observation is a nice catch too — exit codes prove that a command failed, not that the invariant you cared about was the one being tested.
Thanks mp3sss for the read and the code-signing analogy, Comparing Merkle-sealed test trees to tamper-evident update manifests hits the exact architectural target: the core challenge is building an authority that is completely immune to prompt engineering charm.
The renegotiate --reason loophole is indeed where agents try to talk their way out of a lockdown.
Appreciate the sharp eye on trust boundaries. stay tuned for how we lock down that escape hatch.
the part where the move itself broke things is the useful bit. docs don't push back, the CLI does.