Field Notes · Technical Notes August 21, 2026 · 6 min read

AI can accelerate analysis without owning the conclusion.

Language models are now genuinely useful in analytical work: summarizing large evidence sets, normalizing log formats, drafting narrative, surfacing patterns a human would take days to reach. They also produce fluent output whether or not it is grounded, and fluency carries no marker separating what came from the source material from what was generated. Handled without an explicit validation boundary, the acceleration quietly relocates authorship of the conclusion.

The provenance collapse

The specific mechanism is worth stating precisely, because the problem is not that models are unreliable in some general sense. It is that their output is undifferentiated.

Feed a model a set of logs and ask for a timeline, and what comes back contains several categories of statement at once: items read directly from the source, items inferred from the source, items that are conventional for this kind of scenario and were not in the source at all, and items that are simply wrong. All four arrive in the same typeface, the same register, and the same confident tone. Nothing about the artifact distinguishes them, and the model's own confidence is not a reliable signal of which category a given line belongs to.

This is the same structural failure described one layer down in Forensic Parsers Should Fail Closed: a tool that emits confident output carrying no marker of its own incompleteness. The parser case is arguably easier, because a parser at least fails on input it cannot handle. A language model does not fail. It produces something plausible, which is a harder failure mode to notice and a much harder one to audit after the fact.

Human review is not automatically a control

The standard mitigation is that a qualified person reviews the output. That mitigation is real only under a condition that frequently goes unstated: the reviewer has to know what to validate.

A reviewer who reads a fluent summary and finds nothing that contradicts their understanding has validated coherence. Coherence is exactly the property the model optimizes, so this check is weakest against the errors most likely to occur. A reviewer who instead selects the claims that will bear weight and traces each one to the source record is performing a different and much stronger operation, and it costs meaningfully more time.

The practical consequence is that "reviewed by a senior analyst" describes an activity, not an assurance level, in the same way that a control being present differs from a control being effective. What matters is which claims were traced, by whom, against what.

Setting the validation boundary

A validation boundary is a decision, made before the work starts, about which categories of claim must be traced to source regardless of how convincing the generated version looks. Making it explicit converts an unbounded review problem into a bounded one, which is what makes the discipline survivable in practice.

Categories that generally belong inside the boundary:

• Anything entering a conclusion. If a statement will appear in the findings, or a decision will rest on it, it gets traced. This is a small fraction of most documents and it carries nearly all of the risk.
• Timestamps and sequence. Ordering drives causal interpretation, timezone and precision errors are easy to introduce and hard to notice, and a plausible-looking sequence is exactly what a model is good at producing.
• Attribution of an action to an identity or system. The difference between "this account performed the action" and "this action is consistent with that account" is the difference between a finding and an inference, and generated text collapses it readily.
• Quantities. Counts of affected records, hosts, or accounts tend to acquire false precision in summarization.
• Any negative finding. Discussed separately below, because it deserves its own treatment.

Categories that can generally sit outside it: navigation and prioritization of what a human examines next, format normalization where the transformation is mechanical and checkable, first-draft prose where the human supplies the findings and the model supplies the sentences, and hypothesis generation where the output is explicitly labeled as candidate explanations to test rather than as results.

Negative findings are the sharpest case

A model should not be the source of a statement that something did not happen. "No evidence of exfiltration appears in these logs" is a claim about the completeness of a search across a defined corpus, and its strength depends on what the corpus covered, over what period, with what retention, and whether the relevant telemetry existed at all. A model asked to summarize a set of logs has access to the set it was given. It has no access to the question of what was missing from that set, which is precisely the reasoning that makes an absence claim meaningful, and which is the subject of Evidence Gaps Are Findings.

A generated absence claim therefore inherits none of the coverage analysis that would justify it while sounding identical to one that does. Of all the categories, this is the one where the distance between apparent and actual support is widest.

What preserving provenance looks like in practice

The tooling implication is modest and mostly structural. Keep generated interpretation in a separate field or document from source extracts rather than merging them into one narrative. Retain the input actually supplied to the model, since a summary of a summary loses the ability to check anything. Record which model and configuration produced a given artifact, because output behavior changes across versions and a conclusion produced months earlier may not be reproducible. And keep the reviewed-and-traced status attached to individual claims rather than to the document as a whole, for the same reason per-record status beats per-file status in evidence processing.

None of this requires unusual infrastructure. It requires deciding that generated content is a distinct class of material, and then not letting it merge into the record silently.

The strongest objection

The serious pushback is that this reasoning is calibrated to weaker models than the ones now available, that capable models are frequently more accurate than a tired analyst at hour eleven, and that humans introduce errors of exactly the same kinds.

Both halves of that are fair, and neither reaches the argument. The claim here is not that models are less accurate than humans. It is that model output lacks provenance markers, and accuracy without provenance is a worse combination than it appears, because a more accurate system earns more trust while remaining equally unauditable. When the rare error does occur inside work that has been reliable for months, the organization has neither the habit nor the record needed to find it. Human analysts are also fallible, and the difference is that a human's reasoning can be interrogated, their sources asked for, and their confidence challenged in a way that produces a revised answer rather than a regenerated one.

A boundary condition, stated plainly: most analytical work is not high-stakes, and applying this discipline to routine triage, internal summaries, or exploratory reading would waste the capability entirely. The boundary earns its cost when a conclusion will be relied upon by someone outside the team producing it.

Questions for a team or firm using AI in analysis

1. Which claims in this work product were generated, and which were read from source? Can you show me the difference in the artifact itself?
2. What was the validation boundary, decided when, and which specific claims were traced?
3. Was any negative finding produced or summarized by a model, and what coverage analysis supports it?
4. Is the material actually supplied to the model retained, along with the model and configuration used?
5. If we had to defend this conclusion to a regulator, an insurer, or opposing counsel in eighteen months, what would we be able to show about how it was reached?

A team that answers these easily is using the capability well. A team that has not considered them is not necessarily producing bad work, but it cannot currently demonstrate that it is not.

Conclusion

The useful framing is not whether to use these tools in analytical work. It is which part of the work they are permitted to own. Acceleration of search, normalization, prioritization, and drafting is a genuine gain and carries little evidentiary risk. Authorship of findings, attributions, quantities, and above all absence claims is a different matter, because generated text arrives without the markers that let anyone audit it later. Draw that line deliberately, keep source and generated material distinguishable in the record, and the capability makes analysis faster without making conclusions harder to defend.

← All Field Notes