Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse varius enim in eros elementum tristique. Duis cursus, mi quis viverra ornare, eros dolor interdum nulla, ut commodo diam libero vitae erat. Aenean faucibus nibh et justo cursus id rutrum lorem imperdiet. Nunc ut sem vitae risus tristique posuere.
To review an AI-generated security fix, check four things in order: that it fixes the root cause rather than the symptom, that the code still does what it did before for legitimate users, that it introduces no new risk, and that the evidence attached to the PR supports the claim. The checklist below turns that into ten questions you can answer in a normal code review.
Approving a fix because the scanner stopped complaining is the most common mistake. A closed alert is a claim. A reviewed fix is evidence.
You already review code for correctness and style. A security fix adds a harder question: did this actually remove the vulnerability, everywhere it can be reached?
AI-generated fixes fail in specific, documented ways. GitHub's responsible-use guidance for Copilot Autofix is candid about them: suggestions may contain syntax errors, sit at the wrong location, change program semantics, fail to fix the vulnerability, introduce a new one, or only partially fix it. They may also add or update dependencies that are unsupported, insecure or fabricated. GitHub's advice is to always review suggestions and confirm CI passes and the alert resolves before merging.
That list is true of every generative fixer, including ours. It's the right list to review against.
Before reading the diff, read the finding. "User-controlled id reaches Invoice.find without an ownership check" is a usable statement. If the PR doesn't let you write that sentence, ask for it. You can't verify a fix for a bug you can't name.
Find the line where untrusted input becomes dangerous. A fix at the root cause changes that line or the guard in front of it. A fix at the symptom escapes one output, blocks one payload, or wraps one call site while three others stay open. Search for other callers of the same sink.
A security fix that also refactors the module is two changes pretending to be one. Large diffs hide behavior changes. If the fix needs a refactor, it should be a separate, labeled commit.
Run the happy path in your head, then in the tests. The owner of the invoice should still download it. A validator that rejects the attack but also rejects real customer input is an outage with a CVE number.
Look for a test that would have failed before the fix and passes after: the non-owner gets 403, the traversal path is rejected, the injected string is treated as data. Passing existing tests shows nothing broke. Only a new negative test shows the hole is closed.
If the PR edits an existing test, read that edit twice. Sometimes the old test asserted the vulnerable behavior and changing it is correct. Sometimes the test was changed because it got in the way. The commit message should say which.
Check that the package exists, is the one you expect, and is maintained. Check the version against the advisory. A fix that introduces an unfamiliar package deserves the same scrutiny as any new dependency.
A fix that throws a raw exception in a service that returns typed errors will be rewritten by the next engineer, and the rewrite may undo the fix. The fix should use the repository's existing error handling, logging and auth helpers.
Read the PR description. "Fixes SQL injection" with no detail is a weak claim. "Parameterizes the query in reports.py:88, adds a test with a quote-injection payload, scanner re-run is clean" is a strong one. Be wary of descriptions that claim more confidence than the checks behind them.
This is the final filter. AI authorship isn't a reason to lower the bar or raise it. If you'd ask a teammate to change it, ask the fixer to change it.
Send it back, or close it, when:
A sent-back fix isn't a failure of review. The restraint is the review.
Most of this checklist can, and should, be done before a human looks. A fixer worth trusting runs its own independent review for root cause, behavior and minimality, runs the checks the pipeline has, and writes a PR description whose confidence matches its evidence. Your review then becomes confirmation, not investigation.
Nullify's Autofix works only on findings triaged as real and worth fixing. It plans a minimal root-cause change, edits in the repository's own error handling, logging and style, and sends the change through a separate review step that rejects fixes that miss the root cause, change behavior, or aren't minimal. It then runs the scanner, build, test and exploit checks available to that workflow and opens a draft PR. Nullify's documentation is explicit that a generated fix is a proposal, not a guarantee, so checks that aren't available should be reviewed before you accept the change.
For REST and GraphQL findings that the pentest agent proved, Exploit Validation records the evidence (request and response pairs, screenshots and video), so the reviewer can see the exploit the fix is meant to close.
Developers can mention Nullify on the PR to ask why a change was made or to ask for a different approach. Nullify never merges. You do.
Ten questions, one rule. A closed alert is a claim. A reviewed fix is evidence.
Not inherently, but it fails in recognizable ways: partial fixes, fixes in the wrong place, changed behavior and unexpected dependencies. GitHub lists these for Copilot Autofix in its responsible-use documentation. Review AI-generated fixes with the same bar as a colleague's code, with extra attention to those failure modes.
Look for a new negative test that would have failed before the fix and passes after, a clean scanner re-run, and for proven API findings, the original exploit no longer working. Passing existing tests only shows nothing broke. It doesn't show the hole is closed.
No. Automation can do the work up to merge readiness, including checks, CI repair and reviewer assignment, but a human should approve every security fix. Passing checks shows the change builds and tests pass, not that it's the right change for your system.
The vulnerable location and a one-sentence description of the bug, a minimal diff at the root cause, a test that proves the fix, the results of the checks that ran, and an honest note on any checks that didn't run. Anything less makes the reviewer do the fixer's investigation.