Field Notes · Private AI August 21, 2026 · 6 min read

Private AI is a custody model, not a hosting model.

Organizations move AI workloads onto their own infrastructure to get privacy, and the move buys something real: custody. Data, weights, and telemetry sit where you can control and observe them. Custody is an opportunity, not an outcome. Privacy is a property of the whole deployed system, and the system includes identity, data flows, APIs, retrieval stores, prompt logs, administration, and dependencies that keep behaving the way they were built regardless of whose rack the server is in. This note separates what private deployment buys from what it obligates, and lists the questions to answer before approving one.

What "private" is actually deciding

Strip the adjective and the decision underneath is about custody: who holds the data and the model, who can access them, what leaves the boundary, and what evidence of access exists. A managed API gives custody to the provider and gives you contract and attestation in exchange: commitments about training use, retention, subprocessors, and access, verified as well as the paperwork allows. A private deployment gives custody to you: the ability to know and control what happens, bought with the obligation to actually exercise it. These are two custody models with different trust and verification properties. Neither is "private" by nature; each can be operated into or out of confidentiality.

The framing matters because the adjective smuggles in a conclusion. "We run it on our own hardware, therefore the data is protected" reasons from location to outcome, skipping the entire system in between. Location changes who is able to protect the data. It does not protect the data.

The system the adjective skips

A production inference deployment is a stack of components, and each carries part of the confidentiality outcome:

• The serving layer and its API. Inference servers exist to serve requests, and their reachability is a configuration fact, not a default virtue. A model bound to a broadly reachable interface without authentication is private in no meaningful sense, whatever building it sits in.
• The retrieval pipeline. RAG architectures copy the sensitive corpus into embedding stores and caches. The document management system's careful permissions do not follow the documents into the vector database unless someone builds that propagation. Retrieval stores are frequently the least governed copy of the most sensitive data in the stack.
• Prompt and output logs. An operating deployment accumulates a new dataset: what people asked, with what context attached, and what came back. This dataset concentrates exactly the information the deployment was meant to protect, and its retention, access, and inclusion in backups are usually decided by default rather than by anyone.
• Identity and administration. Who can query the model, who can read the logs, who can swap the weights, who administers the boxes. Administrative access to a private AI stack is access to everything the stack has seen, and it is commonly broader and less reviewed than the equivalent access at a managed provider.
• Dependencies and telemetry. The stack is assembled from open-source servers, frameworks, and containers, each with its own update path and some with their own phone-home behavior. What actually leaves the boundary is an empirical question about the deployed software, not a licensing question about the model.
• Model provenance. Weights downloaded from a public hub are an input to your custody chain. What was downloaded, from where, verified how, and under what license are questions with compliance and integrity consequences that arrive later, when they are expensive.

This decomposition is the reason my research work in this area, the deployment hardening checklist, the Tasia configuration reviewer, and the Exposure Index study, examines the surrounding system rather than the model: in practice, that is where the confidentiality outcome is decided.

The obligation transfer

The complication executives should sit with is that private deployment transfers obligations along with custody. A managed provider, whatever its other properties, employs people whose job is patching, hardening, monitoring, and incident response for the serving infrastructure. Bring the workload inside and those functions do not disappear; they land on your team, alongside everything else that team does. The honest comparison is therefore not "provider custody versus our custody" in the abstract. It is "their operated system versus our operated system," and a disciplined provider under strong contractual commitments can deliver better real-world confidentiality than an understaffed private stack running defaults. The decision deserves that comparison stated plainly, because the private option is frequently approved on the adjective and costed on the hardware, with the operating obligation discovered in production.

Questions to answer before approving the deployment

1. What data classes will this system touch, and what obligations attach to them? Contractual commitments, regulatory duties, and customer representations define what custody must actually achieve. "Sensitive" is not a requirement; a named obligation is.
2. Where is the boundary, exactly? Enumerate what crosses it: model downloads, updates, dependency telemetry, remote support access, backups. A boundary nobody has enumerated is a diagram, not a control.
3. Who can reach the serving layer, and as whom? Authentication, authorization, and network reachability for the API, stated as facts about the configuration rather than intentions about the architecture.
4. What happens to prompts and outputs? Retention, access, backup inclusion, and deletion for the logs, decided deliberately and matched to the sensitivity of what flows through them.
5. Does retrieval preserve authorization? If source permissions do not follow documents into the embedding store, the store is a permission bypass, and the question is who can query it.
6. Who operates this, with what time? Named ownership for patching, monitoring, model updates, and incident response. If the answer is a fraction of one engineer, that fraction is the real confidentiality budget.
7. What would evidence of misuse look like? If the system were queried inappropriately next quarter, what record would show it? A custody model without evidence of access fails exactly when it matters, a property this corpus has examined from the incident side in Evidence Gaps Are Findings.

Boundary conditions

Two limits on the thesis. First, custody sometimes is the requirement: data residency mandates, contractual prohibitions on third-party processing, and certain regulated or sovereign contexts make provider custody unavailable regardless of how well operated it is. In those cases the decision is not whether to hold custody but whether the organization will fund the operating capability custody requires. Second, the argument does not claim managed providers are safer in general; it claims the comparison is between operated systems, not between adjectives. An organization with strong infrastructure discipline can absolutely run a private stack to a higher standard than any external commitment reaches. The point is that this outcome is earned by the operating work, and the approval decision should price that work explicitly.

Conclusion

Private deployment is a custody decision: it relocates the opportunity to control and observe an AI system, and it relocates the obligations that make control real. Privacy is what a specific operated system achieves across its serving layer, retrieval stores, logs, identities, and dependencies. An executive approving a private AI deployment is not buying privacy; they are buying the ability to build it, plus the duty to staff it. Approve it on that basis, with the boundary enumerated, the logs governed, the retrieval authorization preserved, and an owner named, and the adjective becomes accurate.

Related: A Technical Claim Is Not Evidence, on decompressing "private" and its siblings when a vendor is the one saying them.

← All Field Notes