Confidential inference means the model, the prompt and the answer are all encrypted while the GPU is working on them. The host — the cloud provider, the hypervisor, and Modelyo — sees only ciphertext.
On H100 this is done in hardware: the GPU runs in confidential computing mode, its memory is encrypted, and the path between the CPU enclave and the GPU is authenticated end to end. Nothing about the workload is visible from outside the sealed boundary.
The part people expect to hurt is throughput. It does not: validated at near-native throughput on production GPU hardware, the cost of the seal is small enough that it is never the reason to leave data exposed.
