Introducing Refactron: the verification layer for AI code change

The economics of a code change just flipped. Writing a framework migration or a sweeping refactor used to be the slow, expensive part, the work teams deferred for years. Now an agent hands you the diff in seconds. The cost did not vanish. It moved downstream, to the one question nobody automated: did this change actually preserve behavior, or did it quietly break something in a way that still compiles and still passes a glance?

By some estimates a quarter of AI-generated code changes are wrong in exactly that way. Not red builds, not stack traces. Confident, plausible, wrong code, merged. That is the defining failure mode of this era of software, and it is the gap Refactron was built to close.

Generating the change was never the hard part

Coding agents got good, fast. Cursor, Claude Code, Copilot, Codex: point any of them at a task and you get a diff that looks right. But a diff that looks right and a diff that is right are different claims, and the distance between them is where production incidents live. A reviewer cannot hold a forty-file refactor in their head. “LGTM” is a hope, not a proof. The bottleneck is no longer authoring the change. It is trusting it.

What Refactron is

Refactron is the verification layer for AI code change. Point it at any diff, your agent's, a codemod's, or your own, and it proves whether that diff preserved behavior, then returns a verdict and a reproducible report. It is not another model with an opinion about your code. It is a gate that runs your real tests and refuses to let an unproven change reach your main branch.

bashrefactron verify-diff . --diff pr-1421.diff

  parse diff      142 hunks mapped
  shadow tree     isolated copy, your tree untouched
  gates           syntax · imports · suite
  suite           4,096 tests · all green
  coverage        changed lines exercised

  VERDICT  SAFE   reproducible

How the gate works

Every verdict runs through the same stages in the same order, fast enough to run on every diff, even inside an agent loop.

• Parse the diff. Map exactly which symbols changed, whoever authored them.
• Shadow tree. Apply the diff to a throwaway copy of the repo. Your own tree is never touched.
• Three gates. Syntax, then imports, then your real test suite, run in that copy.
• Attribution. Check whether a test that ran actually executed the changed lines. You cannot know that until the suite has run, which is why this comes after the gates and not before them.
• Verdict and report. Return SAFE, UNSAFE, or UNPROVEN, with the evidence behind the call.

The shadow tree

Refactron never mutates your working copy to check a change. It materializes an isolated, content-addressed tree, applies the diff there, runs the scoped suite against it, and throws it away. Your disk never sees the change at all. Refactron does not write to your repo on any verdict, so landing stays yours: git commit, or let your agent do it. There is no window where a half-verified edit sits in your tree, and a failing run leaves nothing to revert.

Three verdicts, no hand-waving

SAFE means the changed code is covered and your tests pass under it. Land it automatically, or gate it behind review. UNSAFE means a test broke, and the exact failing case is in the report, before the change ever touched disk. UNPROVEN is the verdict nobody else will give you: your tests pass, but the changed lines were never exercised, so green proves nothing about your change. Instead of rubber-stamping it, we say so and list the tests you would need to make it provable.

Every team we talked to had the same question, and it was never “can you write this change?” It was “how do I know this change is safe?”

A report you can actually hand to someone

A verdict you cannot reproduce is just an opinion. Every Refactron report records the base tree, the diff, the test set, and the shadow-tree hash the run happened in, so anyone can rebuild the exact tree, re-run the exact suite, and land on the exact verdict. That record turns “our CI said it was fine” into an audit-grade account of why it was fine.

Where it plugs in

• CLI, today. Point it at any diff and get a verdict and a report, locally, against the test suite you already have.
• MCP server, early access. The tool your agent calls before it lands a change, so it can check its own work.
• CI gate, in the open. A check that blocks unsafe merges and flags the unproven ones with the missing-coverage list.

It runs today on Python and TypeScript, against the suite you already have, and nothing leaves your machine to produce the verdict. One limit worth naming: changed-line coverage is proved on Python only, so a TypeScript diff clears every gate and still caps at UNPROVEN rather than claiming a proof we cannot make.

That is the whole thesis. Generating the change is a solved problem. Trusting it is not. Refactron is the layer that closes the gap, and you can hold us to that the same way we hold every diff to it. Join early access to be a design partner and shape the trust layer your team will run.