Inside Cohere Model Vault Building Verifiable Confidential Ai Inference

Cohere Model Vault: Verifiable Confidential AI Inference Skip to content Rerank 5 is here: sharper search, fewer tokens. Now with Context Focus. Learn more

Products

Solutions

Resources

Blog

Research

Company

Sign in

Request a demo

Platform North

Enterprise-ready AI for business

Compass

Intelligent search and discovery

Models Command

Generative language models

Transcribe

Speech recognition model

North Small Translate

Machine translation model

Parse New

Document parsing model

Embed New

Semantic representation model

Rerank New

Retrieval optimization model

Models Overview

Product Products Overview

Total Cost of AI Ownership

Pricing

Featured North 2: Cohere's latest platform update — with 15+ new features

Deploy Model Vault

Dedicated model inference platform

Private Deployments

On-prem or isolated VPCs

Security

Protect your data at every stage

See deployment options

By Industry Financial Services

Public Sector

Technology

Telecommunications

Energy and Utilities

Healthcare

Life Sciences

Manufacturing

Featured Model Vault provides fully-isolated, performant inference with Saas simplicity

Insights Customer Stories

For Developers Developers

Models Overview

Docs

Discord

LLM University

Connect Partners

Events

Webinars

Merch Store

Featured How CoreWeave used Cohere North to transform its customer support in 90 days

Blog

The latest news, launches, and insights

Read more

Cohere to acquire AISLE to advance cyber-resilient sovereign AI

North 2: Enterprise AI without compromises

Introducing Embed 5—A new family of frontier embedding models

Research Cohere Labs

Cohere’s ML research lab

Explorations Future(s) of Work

How will AI change the way we work?

Aya Models

Multilingual AI at scale

All Papers

Initiatives Research Scholars

Finding the new generation of ML talent

Open Science Community

Championing global, open science

Catalyst Grants

Supporting impactful ML endeavors

Resources Blog

Hugging Face

Events

Featured The future of work debate has an evidence problem

About

Careers

Newsroom

Oct 22, 2026

8 minute read

Inside Cohere Model Vault: Building a verifiable confidential AI inference How Model Vault combines confidential CPU and GPU execution, end-to-end encrypted request contents, and remote attestation to protect sensitive AI workloads while they are being processed.

In our product launch, we introduced Confidential Computing on Cohere Model Vault , now in public beta with NVIDIA GPUs on Microsoft Azure.

This article goes deeper into how the system works.

The goal is not simply to encrypt inference. It is to reduce the infrastructure that customers must trust with plaintext data and give them cryptographic evidence they can verify before sensitive information is released.

That requires more than a confidential virtual machine. A confidential inference system has to protect the computation itself, the path that carries sensitive requests to it, and the mechanism customers use to decide whether the destination can be trusted.

Model Vault brings those pieces together around a dedicated model deployment. From confidential hardware to confidential inference Confidential computing protects data in use , while an application or model is actively processing it.

At the foundation are two properties: Isolation , which uses hardware-enforced controls to protect workload memory from infrastructure outside the confidential environment. Attestation , which produces cryptographically signed evidence about the environment so another party can evaluate it before releasing sensitive data.

Together, these properties change the traditional infrastructure trust model. Privileged access to the underlying host no longer automatically implies access to the workload running inside the confidential environment.

For AI inference, however, the boundary cannot end at the CPU.

A substantial portion of model computation runs on accelerators. If the CPU were protected while the accelerator remained outside the confidential boundary, sensitive model state and intermediate computation could become exposed at the point where inference moves to the GPU.

Confidential Computing on NVIDIA GPUs protects GPU memory from infrastructure outside the protected environment and encrypts communication between the CPU and GPU. This allows the confidential boundary to span the confidential virtual machine with the accelerator performing inference.

Inside a confidential Model Vault No single component makes an inference request confidential.

Model Vault combines several controls around a dedicated model deployment, each addressing a different part of the trust problem.

Confidential CPU and GPU execution

The model-serving stack runs inside a confidential virtual machine.

The operating system, drivers, inference server, and their memory are protected from the host environment through hardware-backed isolation. The workload is also constrained by policy so that only approved software can run inside the environment and operators do not have an interactive path into the workload.

The NVIDIA Confidential Computing extends protection to the accelerator and encrypts traffic between the CPU and GPU.

This matters because the security boundary needs to cover the systems actually performing the inference, not only the CPU coordinating it.

Microsoft Azure provides the underlying confidential computing infrastructure. Model Vault builds the managed model-serving, encrypted request, attestation, and customer-verification layers around that foundation.

The result is that customers do not need to assemble, integrate, and operate these controls independently. Encrypting the request itself Protecting the compute environment solves only part of the problem.

Sensitive data also has to reach that environment.

TLS protects network connections, but connection-level encryption alone does not guarantee that every intermediary involved in operating a service remains outside the plaintext path.

Model Vault therefore protects the application payload itself .

It uses Oblivious HTTP, or OHTTP , to encrypt sensitive request contents to a public key associated with the attested confidential environment. The encrypted message can then pass through intermediary infrastructure as ciphertext.

Only the OHTTP gateway inside the confidential environment has access to the corresponding private key and can decrypt the protected request. The response is protected on the...