Introducing Refactron: the verification layer for AI code change

The economics of a code change just flipped. Writing a framework migration or a sweeping refactor used to be the slow, expensive part, the work teams deferred for years. Now an agent hands you the diff in seconds. The cost did not vanish. It moved downstream, to the one question nobody automated: did this change actually preserve behavior, or did it quietly break something in a way that still compiles and still passes a glance?

By some estimates a quarter of AI-generated code changes are wrong in exactly that way. Not red builds, not stack traces. Confident, plausible, wrong code, merged. That is the defining failure mode of this era of software, and it is the gap Refactron was built to close.

Generating the change was never the hard part

Coding agents got good, fast. Cursor, Claude Code, Copilot, Codex: point any of them at a task and you get a diff that looks right. But a diff that looks right and a diff that is right are different claims, and the distance between them is where production incidents live. A reviewer cannot hold a forty-file refactor in their head. “LGTM” is a hope, not a proof. The bottleneck is no longer authoring the change. It is trusting it.

What Refactron is

Refactron is the verification layer for AI code change. Point it at any diff, your agent's, a codemod's, or your own, and it proves whether that diff preserved behavior, then returns a verdict and a signed, reproducible report. It is not another model with an opinion about your code. It is a gate that runs your real tests and refuses to let an unproven change reach your main branch.

bashrefactron verify --diff pr-1421.diff

  parse diff          142 hunks mapped
  blast radius        23 modules touched
  coverage preflight  changed lines exercised
  shadow-tree run     4,096 tests · all green

  VERDICT  SAFE   signed sha256:4f2a…c91d · reproducible

How the gate works

Every verdict runs through the same five stages, scoped tightly to the change so it stays fast enough to run on every diff, even inside an agent loop.

• Parse the diff. Map exactly which symbols changed, whoever authored them.
• Blast radius. Walk the import and call graph outward to everything the change can reach.
• Coverage preflight. Confirm the changed lines are actually exercised by a test, before a green run is allowed to mean anything.
• Shadow-tree run. Apply the diff in an isolated, throwaway copy of the repo and run the scoped suite there.
• Verdict and report. Return SAFE, UNSAFE, or UNPROVEN, with signed evidence for the call.

The shadow tree

Refactron never mutates your working copy to check a change. It materializes an isolated, content-addressed tree, applies the diff there, runs the scoped suite against it, and throws it away. Your disk only ever sees the change if the verdict comes back green, and when it does, the change lands atomically in a single operation. There is no window where a half-verified edit sits in your tree, and a failing run leaves nothing to revert.

Three verdicts, no hand-waving

SAFE means the changed code is covered and your tests pass under it. Land it automatically, or gate it behind review. UNSAFE means a test broke, and the exact failing case is in the report, before the change ever touched disk. UNPROVEN is the verdict nobody else will give you: your tests pass, but the changed lines were never exercised, so green proves nothing about your change. Instead of rubber-stamping it, we say so and list the tests you would need to make it provable.

Every team we talked to had the same question, and it was never “can you write this change?” It was “how do I know this change is safe?”

A report you can actually hand to someone

A verdict you cannot reproduce is just an opinion. Every Refactron report records the base tree, the diff, the scoped test set, and the shadow-tree hash the run happened in, so anyone can rebuild the exact tree, re-run the exact suite, and land on the exact verdict. It is signed with a sha256 over its contents, which turns “our CI said it was fine” into an audit-grade record of why it was fine.

Where it plugs in

• CLI, today. Pipe in any diff and get a verdict and a report, locally, against the test suite you already have.
• MCP server, early access. The tool your agent calls before it lands a change, so it can check its own work.
• CI gate, in the open. A check that blocks unsafe merges and flags the unproven ones with the missing-coverage list.
• Migration mode, live. We still author and verify the migrations teams fear to automate, like SQLAlchemy 1 to 2, because they are the hardest test of the gate.

It works today on Python and TypeScript, verified against the suite you already run, and nothing leaves your machine to produce the verdict.

That is the whole thesis. Generating the change is a solved problem. Trusting it is not. Refactron is the layer that closes the gap, and you can hold us to that the same way we hold every diff to it. Join early access to be a design partner and shape the trust layer your team will run.