A validated finding is one a stranger can reproduce, counsel can rely on, and a retest can close. Five artifacts, no more and no less.
Frontier models now find security weaknesses faster than any human team can review them. Unit 42 published the operational numbers in September: no single model catches more than about 40 percent of the vulnerabilities in a complex environment, the leading cyber models overlap on less than 10 percent of what they report, and in third-party applications two of every three validated exposures had no CVE at all. That describes a discovery engine that produces more claims than anyone can act on. The scarce thing is not findings. It is findings someone can defend.
A finding is defensible when a person who was not in the room can reproduce it, when counsel can rely on it, and when the retest after the fix proves the path is closed. I have been building toward that standard in assay, an open-source multi-model validation harness, and in the evidence practice that preceded it. This note states the standard plainly. It is five artifacts. A finding that lacks any of them is a theory, and should be labeled one.
1. Reproduction steps a stranger can run
Not a narrative of what the model did. A numbered sequence of requests, commands or actions, with every parameter, header and payload that mattered, that produces the same observable result from a clean start. If the steps depend on state the model built up over forty tool calls, the steps are not reproducible yet; reduce them until they are. The test is simple: hand them to someone who did not run the assessment and watch whether they get the same result without asking a question.
2. An environment fingerprint
What was tested, exactly. Hostnames and addresses as scoped, software versions and build identifiers where they could be read, the time window, the identity the test ran as, and the scope document that authorized it. The fingerprint is what lets a customer say "that was the version we shipped on the 14th" and what lets a retest confirm it is looking at the same thing. Without it, a finding floats free of the system it describes and nobody can say later whether the fix applied to the thing that was broken.
3. Evidence with hashes, captured before interpretation
The raw request and response pairs, the tool output, the screenshots or recordings, and the control-plane log of what actually executed. Each file hashed at capture time, before anyone decided what it meant. An AI agent's own transcript is a claim about what it did; it is not evidence of what ran. The execution record is. When the two disagree, the disagreement is itself a finding. Keep the chain of custody short and written down: who captured it, when, and where it has been since.
4. A chain diagram when flaws combine
Most serious exposures are not one flaw. Unit 42's financial-sector example was three individually minor issues, a payment link that skipped identity re-verification, a bypassed one-time code, and a session routing flaw, that together gave full account takeover with no victim action. A validated finding for a chain shows each link, the evidence for each link, and the point at which breaking any one link stops the attack. That last part is what makes the diagram useful for remediation instead of just alarming.
5. A retest record
After the fix, the same reproduction steps against the same fingerprint, run by someone who did not write the fix, with the result recorded and hashed. The retest either reproduces the finding or it does not. "The developer says it's patched" is not a retest. "The scanner no longer flags it" is not a retest either, because the scanner never validated it in the first place. The retest is the only artifact that turns a finding into a closed finding, and it is the one most often skipped.
Why five and not fifteen
Every one of these answers a question that an executive, an auditor or opposing counsel will ask. Can you show me? On what system? Where is the proof? How do the pieces connect? Is it fixed? Anything beyond those five is detail that belongs inside one of them. Anything less than those five leaves a question open, and an open question on a security finding is how a remediation budget gets argued about instead of spent.
The models will keep getting better at finding things. The standard for what counts as found should not move at all.