Private AI inference means your prompts and outputs can't be read, stored, or trained on by anyone but you. The catch is that this promise comes in two fundamentally different kinds: a retention policy you must trust, and technical enforcement you can verify. The industry uses the same reassuring words for both. Knowing which one you're getting is the entire skill — and the subject of the sibling explainer on open-weight models, the artifacts that make verifiable privacy possible in the first place.
Key facts
- Apple's Private Cloud Compute sets the reference design: personal data is never stored, and the software stack is published for independent verification with cryptographic attestation.
- Confidential computing on NVIDIA GPUs encrypts data in use, so the host operator — cloud or otherwise — cannot inspect model inputs even with physical access to the machine.
- TEE-based clouds like Phala extend the same idea to decentralized operators: remote attestation lets you check what code is running before you send a prompt.
- Fully local serving with tools like Ollama removes the remote operator entirely — no one else's server ever sees your data.
- Privacy governance frameworks, including NIST's AI Risk Management Framework, treat privacy as a measured risk function, not a checkbox — measure, don't assume.
How it works
Inference touches your data in three places: in transit, in memory during generation, and at rest in logs. TLS covers transit. The other two are where privacy is won or lost, and the three approaches below handle them very differently.
Tier 1 — Local inference. You download open weights and run them on hardware you control, with Ollama, vLLM, or a similar engine. No remote party ever receives your prompts, so there is nothing to retain, review, or train on. The trust surface shrinks to your own machine, the provenance of the model file, and whatever network calls the surrounding application makes. This is the strongest privacy posture available — and the one with the highest hardware and operations cost.
Tier 2 — Policy-based remote inference. A vendor promises, contractually, not to log your prompts, not to review them, and not to train on them — the "zero-data-retention" configuration. Enforcement is legal and procedural: data-processing addenda, audit rights, compliance certifications. These are real protections, and enterprises buy on them daily. But they are promises about behavior, and verifying a negative ("we did not store your prompt") from outside is structurally hard. Treat every absence-claim as unverified until something reproducible backs it.
Tier 3 — Mechanism-based remote inference. Confidential computing encrypts data while it is being processed, inside CPU and GPU enclaves the operator cannot peer into. Remote attestation produces a signed, checkable statement of exactly which code is running inside the enclave — you verify before you send anything sensitive. Apple's PCC demonstrated the pattern at consumer scale; TEE clouds bring it to rented and decentralized hardware. The guarantee moves from "trust us" to "check the attestation," which is a different epistemic category entirely.
The numbers
| Approach | Stops operator reads? | Stops logging? | Verifiable how? | Cost |
|---|---|---|---|---|
| Local weights (Ollama, vLLM) | Yes — no operator exists | Yes — nothing leaves | Inspect your own stack | Your GPUs + ops time |
| Zero-retention policy API | No — operator could, promises not to | Promised, not provable | Contracts, audits, certifications | Per-token pricing |
| TEE / confidential computing | Yes — memory encrypted | Yes, if logging code is outside the enclave | Remote attestation you can check | Premium over plain compute |
| Standard cloud API | No | No | Trust the terms of service | Cheapest, most convenient |
Common misconceptions
"We don't train on your data means we're private." Training exclusion is one clause of one policy. Logging, human review for abuse monitoring, and metadata retention are separate questions with frequently different answers. Ask all three, in writing, before assuming any of them.
"HTTPS means the operator can't see my prompts." TLS protects data from the network, not from the endpoint. The server terminates TLS and sees plaintext by design. Encryption in transit says nothing about what happens after arrival — that is what confidential computing and retention policies are for.
"Zero retention was announced, so it's settled." A press release is a claim, not a check. DeAI's standing rule applies: privacy absence-claims are never verified from a policy statement — only from a validatable mechanism or an independent reproducible test. Ask for the attestation flow or the audit report, and watch what happens next.
"Open weights are automatically private." Weights are inert files. Privacy is a property of where they execute: on your hardware, yes; inside someone else's hosted app wrapping the same weights, you inherit that app's logging. See decentralized inference for how the execution venue changes the trust picture.
Current state (September 2026)
The market is bifurcating cleanly: regulated and enterprise workloads are converging on attested confidential computing, while the zero-retention policy tier competes on convenience and price. Independent verification infrastructure — refusal probes, retention tests, attestation checkers — is still catching up to the claims. DeAI News maintains a tracker of providers and serving options, a zero-retention explainer for the policy tier in depth, and a confidential-computing guide for the mechanism tier. Bring skepticism; keep the checklist.
FAQ
What is private AI inference?
Inference where your prompts and outputs are protected from logging, human review, and training use. The protection is either a vendor policy (a promise, enforced by contracts and audits) or a technical mechanism such as confidential computing (enforced by hardware, checkable via attestation).
Is zero-data-retention the same as private inference?
No. Zero-data-retention is one policy configuration: the vendor promises not to store your data. Private inference is the stronger property — the operator cannot read your data even if it wanted to — and it requires technical enforcement, not just a promise.
How do you verify a privacy claim?
Ask for the enforcement mechanism, not the marketing page: confidential-computing attestation you can check yourself, a data-processing addendum with audit rights, or weights you run on hardware you control. A claim with no mechanism is unverified marketing.
Is running models locally fully private?
Local inference removes the remote operator from the trust picture — no one else's server ever sees your prompts. You still trust your own machine, the model file's provenance, and any network calls the surrounding app makes.
Related reading
For where private execution actually happens, read what decentralized inference is and what open-weight models are. For the two tiers in depth: zero-data-retention, explained and confidential AI inference with TEEs.
Questions
- What is private AI inference?
- Inference where your prompts and outputs are protected from logging, human review, and training use. The protection is either a vendor policy (a promise, enforced by contracts and audits) or a technical mechanism such as confidential computing (enforced by hardware, checkable via attestation).
- Is zero-data-retention the same as private inference?
- No. Zero-data-retention is one policy configuration: the vendor promises not to store your data. Private inference is the stronger property — the operator cannot read your data even if it wanted to — and it requires technical enforcement, not just a promise.
- How do you verify a privacy claim?
- Ask for the enforcement mechanism, not the marketing page: confidential-computing attestation you can check yourself, a data-processing addendum with audit rights, or weights you run on hardware you control. A claim with no mechanism is unverified marketing.
- Is running models locally fully private?
- Local inference removes the remote operator from the trust picture — no one else's server ever sees your prompts. You still trust your own machine, the model file's provenance, and any network calls the surrounding app makes.
Sources
- Private Cloud Compute: A new frontier for AI privacy in the cloud — Apple Security
- NVIDIA Confidential Computing Documentation — NVIDIA
- Phala Network Documentation — Phala Network
- Ollama — Ollama
- AI Risk Management Framework — NIST
- Akash Network Documentation — Akash Network
About DeAI
DeAI is an independent publication covering open-weight AI models, private inference, and decentralized infrastructure — the tools for running AI you actually control. We test providers on price, privacy, and refusal behavior and publish the numbers, not the vibes. DeAI is powered by Morpheus (mor.org), a decentralized inference marketplace, and covers it on the same terms as every other provider.
Powered by Morpheus and StrandCMS
Morpheus is a decentralized inference marketplace, covered on the same terms as every other provider — we rank it wherever the data lands. StrandCMS is the open-source, agent-first framework this site is built on.