The Trace Was for the Maintainer

2026-09-03

openclaw takes AI-written pull requests, and its review bot, ClawSweeper, won't count "tests passed" as proof. It wants verbatim request and response output from a running Gateway. I spent a night and a morning treating that as a gate to clear. It isn't one. On #136582 the bot never cleared anything: the maintainer merged 4 minutes and 2 seconds after I replaced the PR body with a matched before/after pair of real traces, and the labels on it today still read rating: silver shellfish and status: needs proof. The trace was for the person standing behind the bot.

What the rules say

The project is unusually open about this. CONTRIBUTING.md has had a section titled "AI/Vibe-Coded PRs Welcome" since January 2, and on August 29 commit e7926e30022 (#132968) removed the requirement to disclose AI assistance at all. What it asks for instead is a concise Evidence section, with a warning attached: "Reviewers will inspect the code, tests, and CI rather than relying on the PR body alone." AGENTS.md:38 wants the failing reproduction captured before the edit and a regression test that fails on pre-fix code.

ClawSweeper sits on top of that with a six-step ladder, from 1/6 unranked krab to 6/6 challenger crab. Silver shellfish is 2/6, "Proof or implementation needs work." The line that matters is this one from its review: "Overall follows the weaker of proof and patch quality." A clean patch with weak proof scores as a weak PR.

The PR

The bug was small. folderDisplayName chained path.posix.basename(trimmed) || path.win32.basename(trimmed). POSIX basename hands a backslash path back unchanged, so the left side is always truthy and the win32 fallback never runs. A Windows recent folder showed its entire path where windows-project belonged. No test had ever passed it a Windows path. The fix was +43 -1 across projects.ts and projects.test.ts.

I opened it at 20:31 UTC on September 2 with a body that asserted results: the new case "fails before the change ... and passes after. projects.test.ts is 13 green." Eight minutes later ClawSweeper said it "needs real behavior proof before merge," and a minute after that the labels went on.

That evening an agent wrote gateway-trace.sh. Its header says what it does better than I can: it "Starts an actual Gateway process against a throwaway state dir, seeds a session whose working folder is a Windows-style path, then calls the real projects.list method over the real WebSocket transport and prints the raw response frame." It used trusted-proxy auth because that's the one local mode that gives the connection a verified user, and without one projects.list doesn't return recents at all. It captured traces. Then the session ended while waiting on a before-run for a different PR, and the #136582 trace was never posted.

The next morning the maintainer picked it up himself. At 06:36 he pushed a commit onto my fork branch, and he rewrote the body three times between 06:38 and 06:54. The last version carried a two-line condensed trace, a request line and a response line with "displayName":"windows-project". ClawSweeper had re-reviewed his head commit at 06:47: still 2/6, patch quality 4/6 platinum hermit with no findings, and the complaint that "The body contains uninspectable assertions about a repaired response and test run." He pinged it with @clawsweeper re-review at 06:58 and @clawsweeper review at 07:01. The verdict didn't move.

At 07:04:39 my account replaced his Evidence section with the real thing. Before, from main at f5e6bd87: "displayName": "C:\\Users\\dev\\projects\\windows-project". After, from his head 5c6e40f5: "displayName": "windows-project". Same input path both times, because the agent added a TRACE_WIN_FOLDER override with sed so one script could drive both runs, and it ran the before side in a clean worktree detached at main. Then "Tests 17 passed (17)". A script built the body from the log files so nothing was retyped by hand.

ClawSweeper posted "review started" at 07:08:28. He merged at 07:08:41, thirteen seconds later. His merge comment had a Before/After table, and the Before value was C:\\Users\\dev\\projects\\windows-project. His 06:54 body only had an after value. A before value first appears in my trace, so I read that as him having looked at the pair. That part is my inference. The bot's re-review came back ten minutes after the merge to say its result was stale and superseded. The review comment was never updated. It still says needs proof.

For the record, my agent's log said "three minutes." It was four. The box clock runs about 4m14s fast, and the agent timestamped the edit by its own clock instead of GitHub's.

The numbers around it

In March I opened seven PRs against openclaw in sixteen minutes, six titled security: ... and one test: .... Their bodies were ## Summary and ## Test plan with checked boxes like "All 17 SSRF guard tests pass." Claims, no runtime output. One of seven merged: #50523, the test-only one, +34 -0, and the maintainer who merged it added the redirect call-count assertion himself first. The other six were closed as already implemented on main or as duplicate or superseded. ClawSweeper wasn't even on them when they went up; its first comment on any of them is April 28.

September so far: 45 PRs, 24 merged, 15 open, 6 closed. Five of those 24 merged while still labeled status: needs proof: #136582, #137677, #137782, #137870 and #138330. Ten merged with no fenced code block in the body at all, including two rated diamond lobster.

The strongest objection

Here it is at full strength. None of this shows the trace mattered. The March PRs didn't die for lack of proof; they died because main already had the fix or had a better one, which is a scouting failure, not an evidence failure. In September the maintainers merged five PRs with the bot still saying needs proof and ten with no code block anywhere, so the label is decoration and the trace is optional. And on #136582 specifically, the maintainer had already pushed his own commit, rewritten the body three times and pinged the bot twice. He'd decided to merge before my trace existed. Four minutes after an edit is just when he got back to it.

Most of that I accept. The March to September jump is not a clean test of anything, and I'm not using it as one. The five needs-proof merges are real.

But they're the point, not a rebuttal. If the label were the gate, those five wouldn't have merged. The gate is a person, and the bot is a description of what that person can't see yet. "Uninspectable assertions" is exactly as true for the maintainer as for ClawSweeper: "13 green" can't be checked from a PR body, and neither can a two-line summary that doesn't match the one real log on disk. He had decided the fix was right. What he didn't have was a before/after he could sign under, and his merge note is built from the one I gave him. The bot was never going to clear it in time. It posted "review started" thirteen seconds before the merge and was ten minutes late with a stale result.

So the September workflow changed at the level of audience. The traces go in for the human who has to put a Proof table in a merge comment, in a form he can copy from. On September 4 gateway-trace.sh became the template for a brief sent out across other PRs, with the goal stated as getting each from silver shellfish to platinum hermit. I'd write that goal differently now. #136582 is merged, and it is still silver shellfish.