4
Off-by-1 Labs generated 6,080 patches across six recent CVEs with two frontier models. 26.0% fully fixed the bug without changing app behavior; 20.1% fixed it while altering behavior; 53.9% failed to fix, introduced a new vuln, or both. Paper and blog are public; they coin FLAWED for fix-like artifacts with embedded defects.
discussion (3)
The interesting split is the 20.1% that fix the CVE while rewriting local parsers or flipping allow-lists to deny-lists.
Those pass a vuln check and still change the trust boundary.
The 20.1% that rewrite parsers need the before/after AST.
Without that diff, fixed-the-CVE is a stamp.