Five False SAFE Verdicts

Read this one. Five false SAFE verdicts are fixed, and what earns a SAFE is now narrower. The report's shape did not change, so the version number alone will not tell you that.

Every verdict that moves, moves from SAFE toward UNPROVEN. Nothing that was UNSAFE or UNPROVEN can become SAFE, and exit codes are unchanged, so no CI pipeline breaks. Pipelines that read the verdict string will see more UNPROVEN.

Because this is a patch, ^0.4.0 picks it up automatically. That is deliberate: these are correctness fixes and they should reach you without a manual bump.

Fixed

Each of these was reproduced before being fixed. In every case the suite passed and coverage was measured, and the verdict was still wrong.

Changed

Two independent rules tightened what SAFE means. Both move verdicts in the same direction.

1. `SAFE` now requires every coverable changed statement to have run. The old rule cleared a whole file as soon as one of its changed statements executed. A diff changing 40 statements in one file, of which 1 ran, returned SAFE with the reason “Tests pass and the changed code is covered.” Partial coverage now reports UNPROVEN and names the ratio.

[UNPROVEN] Tests pass, but only 3 of 14 changed statements were exercised.

Statements coverage.py excludes (# pragma: no cover, if TYPE_CHECKING:) are subtracted from the count rather than held against you, since no test can reach them. A change consisting *entirely* of excluded statements does not reach SAFE: there is nothing a test could have proven about it.

2. A narrowed test command can no longer earn `SAFE`. Passing a testCmd such as pytest tests/unit/test_foo.py, or one using -k, -m, -t or --onlyChanged, scopes the entire verification run. Coverage could report the changed code as fully exercised while the one test that would have caught the change was never selected. Reproduced with one repo, one diff, two test files, and only the command differing:

bashpython3 -m pytest -q                       # UNSAFE, the change breaks a test
python3 -m pytest -q tests/test_scale.py   # was SAFE, now UNPROVEN

Refactron now classifies the command as full, narrowed or unknown, and reports it on the new testScope field. narrowed floors the verdict at UNPROVEN. unknown, an unparsed wrapper such as make test, does not floor, and that gap is documented rather than hidden. A PYTHONPATH= prefix is not narrowing, so the remedy for shadow bypass is unaffected.

Added

• `testScope` on `VerdictReport`, carried by verify-diff --json and the MCP verify_change tool. Says whether the command was full, narrowed or unknown, so a stored verdict can be audited for scoping after the fact. Additive; reportVersion stays 1.
• `engineVersion` on `VerdictReport`. reportVersion tells you which shape you hold; engineVersion tells you which rules produced the verdict. You want both if you keep reports as history, because this release changed the rules without changing the shape.
• `unittest` is a recognised runner, alongside pytest, vitest and jest.
• A CLI note naming the filter that cost a run its `SAFE`, and a different note when the command could not be parsed at all. Silence there would let a SAFE resting on an unreadable command look clean.
• The MCP `testCmd` schema states the narrowing rule, so an agent learns it before spending a verification run rather than after.

Security

• Cleared two high and one moderate advisory (nanoid, ip-address, hono), lockfile-only. No declared dependency range widened. npm audit is a release gate and was failing before this.

Known limitations, stated rather than implied

• Coverage is Python-only. A TypeScript or mixed diff still returns UNPROVEN; it cannot reach SAFE, which is the fail-safe direction.
• `SAFE` means executed, not asserted. Coverage proves a changed statement ran, not that any test would fail if its behaviour changed.
• A command we cannot parse is not floored. make test, a wrapper script, or an unrecognised plugin flag classifies unknown and can still reach SAFE. Flooring it would make SAFE unreachable for most projects, so the CLI says so out loud instead.
• Config-file narrowing is not seen. pytest.ini addopts and testpaths, a vitest include, a jest testMatch. The command and the environment are read; your config files are not.
• Statement-level, not branch-level. A changed if whose true branch never ran still counts as covered.