Cohere Model Vault adds confidential computing for inference

Cohere has added confidential-computing support to Model Vault, its single-tenant inference product, so enterprise workloads run inside hardware-encrypted memory that Cohere says neither cloud providers nor Cohere itself can read. The company also says it will open-source the serving stack so auditors can check for logging or leakage — a pledge, not yet a delivered artifact.

Key facts

• Confidential-computing support in Model Vault is "newly available," Cohere's director of serving inference Manoj Govindassamy told VentureBeat on September 16, 2026, with beta access for limited customers.
• Workloads run inside hardware that encrypts and isolates its memory — Intel TDX and AMD SEV-SNP class CPU isolation plus Nvidia confidential computing on GPUs.
• Signed attestation tokens let customers independently verify the workload ran inside a genuine enclave — the mechanism that turns "trust us" into something checkable.
• Cohere plans to open-source the full serving stack so independent auditors can validate it does not log, export, or leak data; the code is not public yet.

What happened

Cohere's pitch, quoted in the announcement — "Your data is already encrypted at rest and in transit. But what about in use?" — names the exact gap confidential computing closes. Standard hosted inference decrypts prompts in memory on machines the provider and the cloud operator fully control; anyone with sufficient access at that layer can observe or copy them. Model Vault already isolated tenants from each other as a single-tenant product. The new support moves the isolation boundary down into the silicon: memory pages are encrypted with keys the host operating system, the hypervisor, and by extension the cloud provider cannot read.

The attestation piece matters as much as the encryption. Signed attestation tokens are hardware-signed statements about what code is running inside the enclave, which means a customer (or their auditor) can verify the workload's integrity without taking Cohere's word for it. That is the difference between a policy promise and a checkable property — and it is why this story belongs in the verifiability conversation rather than the marketing one. The launch signals GPU-side confidential computing becoming a product surface rather than a lab feature.

What did not happen: no prices changed, no retention policy was rewritten, and no auditor has yet validated anything. The open-sourcing pledge is forward-looking, the beta is limited, and the per-customer deployment matrix — which models, which clouds, which GPU types — is undisclosed. All of that is normal for a launch-day announcement. It is also why every privacy guarantee in this article is framed as Cohere's claim.

Why it matters

For a builder deciding where to run sensitive workloads, this adds a third option to a previously binary choice. Until now the menu was essentially self-host (full control, full operational burden) or hosted-with-promises (convenient, trust-based). A TEE-backed hosted endpoint with attestation is a middle path: outsource the GPUs, keep a cryptographic check on the handling. Regulated workloads — legal, health-adjacent, financial — are the obvious first beneficiaries, since their blockers are usually about demonstrability to compliance teams, not raw capability.

There is a strategic angle too. Cohere competes against both closed frontier labs and the rapidly improving open-weights field — Mozilla's new report puts the open-model lag at about 4.4 months. When models are increasingly interchangeable, the differentiator shifts to deployment guarantees: who can prove the strongest handling story. Cohere is betting that provable privacy is that differentiator. Whether buyers accept attestation-backed claims as sufficient — or demand the open-sourced stack first — is the adoption question to watch.

Background

Confidential computing is not new, but its application to LLM inference at product scale is. The primitives — Intel TDX, AMD SEV-SNP, Nvidia's confidential-computing mode — have existed for years in cloud infrastructure, mostly serving key-management and clean-room analytics workloads. Bringing them to token generation is harder: GPUs move enormous state at high bandwidth, and every percentage point of enclave overhead lands directly on latency and cost per token. Our confidential inference explainer covers the mechanism in depth, and the NEAR AI private-inference deployment shows the decentralized side of the same trend converging on TEEs from the opposite direction.

The policy backdrop matters here. A growing share of enterprise AI procurement now asks not just "do you retain prompts" but "prove your operators cannot see them" — the absence-claim problem our zero-retention guide documents. Absence claims are never verifiable from a policy page; they need either self-hosting or a validatable attestation. Cohere's announcement is best read as an attempt to supply exactly that missing proof layer for hosted inference.

What's next

Three things to watch, in order. First, the open-sourced serving stack: when the code lands, independent auditors can check the no-logging claim, and until then it stays a claim. Second, beta expansion details — which models and clouds are covered, and whether attestation verification tooling ships for customers or stays auditor-only. Third, competitive response: if TEE-backed inference becomes table stakes for enterprise deals, expect peer announcements within quarters, and expect the conversation to move from "do you have TEEs" to whose attestation story auditors actually accept.