Public-repository study / ACH payments

Twenty Payment Bugs Passed a 71-Package Test Suite.

We challenged one amount-classification function family in the public moov-io/ach repository with 32 deliberate single-code omissions. Every mutation represented genuinely wrong behavior. Twenty survived the project’s full test suite.

A concrete survivor

One mutation removed LoanDebit from a transaction-code classifier. All 71 packages remained green. A focused business invariant then processed a $1,000,000 loan-credit reversal: the library reported zero debit and failed its own balance validation.

No coding agent created this defect. We deliberately created it to ask a harder question: could the visible suite distinguish a faithful change from a plausible, materially wrong one? In this function family, it often could not.

Why this matters for coding-agent autonomy

A coding agent can satisfy every visible test while breaking behavior the company depends on. Adding another model judge does not automatically solve that problem. Before recurring work can be delegated, the deployment needs independent checks for the business invariants that ordinary CI does not defend.

Inside a company, Veyl discovers those invariants from approved pull requests, incidents, rollbacks, tests, and accepted or rejected agent changes. An accountable engineer confirms the small set that matters. Veyl then retains them as private, rerunnable gates for one exact workflow.

What else was tested

At pinned commit cc95789cb2cbe478ad48010c496f66805eab574c, a GPT-5.5/Codex configuration completed a faithful refactor three out of three times, with the full suite green each time. That result shows the intended change was achievable. The mutation study shows that green CI alone did not prove it was faithful.

This is a Veyl-authored public-repository demonstration, not a Moov engagement, endorsement, security audit, current-HEAD assessment, or estimate of how often coding agents fail. The mutations were deliberately seeded by Veyl; they were not agent-produced defects.