Cohere encrypted inference so it cannot read your prompts
Ricardo Argüello, September 25, 2026
Cohere was worth 7 billion dollars a year ago and booked 240 million in 2025 revenue. On September 16 it signed a definitive merger with Aleph Alpha at a reported 20 billion.
That gap is not being paid for model quality. Nobody thinks Cohere is about to out-benchmark Anthropic.
It is being paid for a position: the AI vendor for buyers who cannot or will not standardize on a US lab. Berlin and Toronto headquarters, Aleph Alpha’s Heidelberg site as the research hub, 500 million euros from Schwarz Group, the owner of Lidl, and a possible 3 billion from a Canadian government-backed consortium, per SiliconANGLE.
And in the same week, Cohere shipped the product that makes that position defensible.
Encryption while the data is in use
Every AI privacy conversation for three years has bottomed out in a contract clause. The vendor promises not to train on your data and promises its staff will not look.
A paragraph. That was the control.
Encrypted Model Vault runs inference inside a hardware-based confidential VM with an attested NVIDIA GPU in confidential computing mode, on Intel TDX or AMD SEV-SNP on the CPU side.
The mechanism that matters is in the documentation rather than the announcement. The client encrypts its request straight to that environment over Oblivious HTTP. Keys are generated inside the enclave and never travel to Cohere. The load balancer and the network in between handle ciphertext and nothing else.
On top of that there is attestation. Golden values for the hardware and software are registered with Intel Trust Authority, so when the confidential VM boots, Intel confirms it matches the configuration Cohere, Intel and NVIDIA approved. A customer checks that before the first request.
So the vendor question changes shape. It stops being “do you promise not to look?” and becomes “can I verify that you are unable to look?”
The first question only ever had a legal answer.
Read the fine print before you get excited
Two things.
It is in beta, with early access for a limited group. This is not something you procure next week.
More importantly, attestation does not remove trust. It relocates it. You stop trusting Cohere’s internal policy and start trusting Intel, NVIDIA and the signing chain that certifies those images. That is a genuine improvement, because those parties have different incentives and their claims are checkable. It is not zero trust, and anyone selling it to you that way is overselling.
The rule we apply in AI vendor selection holds. What you cannot verify does not count as a control.
That rule earned its place the hard way. When the Claude Mythos leak happened, every affected company had a contract clause and none of them had a way to check anything.
You are not a German bank, and it still matters
Most readers here will never buy Model Vault. The bar it just raised is still useful.
Next time you evaluate an AI vendor for a workflow touching regulated or genuinely sensitive data, there is now a question with a real answer behind it. Is the data exposed in memory during processing, and is there any way for me to verify your answer without taking your word for it?
Most vendors will say no. That is fine, and it is information. You learn what you are buying and you adjust what you send through it.
What stopped being acceptable is treating the contract paragraph as if it were a technical control. We keep circling this distinction between what a vendor promises and what you actually govern, most recently in the kill switch you do not control.
If you are writing next year’s vendor criteria, add the attestation row to the matrix now. You will get blanks in that column across most of the market this year. The vendor that fills it will be charging a premium for it in 2027.
Let’s review your AI vendor matrixFrequently Asked Questions
What is confidential computing in Cohere's Model Vault?
It is a tier where inference executes inside a hardware-based confidential virtual machine paired with an attested NVIDIA GPU in confidential computing mode. Prompts and responses stay protected in transit, at rest and in use, with no path for Cohere to read them in plaintext.
How can you verify that an AI vendor is unable to read your prompts?
Through hardware attestation. In Cohere's implementation, golden values for the hardware and software stack are registered with Intel Trust Authority, and at boot Intel confirms the confidential VM matches the approved configuration. A customer can check that before sending any data.
What is Oblivious HTTP and why does Cohere use it in encrypted Model Vault?
Oblivious HTTP lets the client encrypt a request directly to the target execution environment. The encryption keys are generated inside the trusted execution environment and never shared with Cohere, so the load balancer and intervening network see ciphertext instead of readable content.
What does the Cohere and Aleph Alpha merger mean for regulated enterprises?
It creates a sovereign AI option headquartered in Berlin and Toronto at a reported valuation of 20 billion dollars, aimed at regulated and public sector buyers who do not want to standardize on OpenAI, Anthropic or Mistral. The agreement was signed September 16 and awaits regulatory clearance.
Newsletter
Get the best articles in your inbox
We respect your privacy. Unsubscribe anytime.