As demand for LLM inference grows, it is becoming increasingly important that providers and their customers can verify that inference processes are performed correctly, without errors or tampering. However, re-running the same inference process twice often leads to different results due to benign numerical noise, making it difficult to distinguish legitimate variation from actual problems. To address this problem, we introduce Token-DiFR (Token-Divergence-From-Reference), a method for verifying inference outputs by comparing generated tokens against predictions made by a trusted reference implementation conditioned on the same random seed. Sampling seed synchronization tightly constrains valid outputs, leaving providers minimal room to deviate from correct inference, which allows output tokens themselves to serve as auditable evidence of correctness at zero additional cost to the provider. Token-DiFR reliably identifies sampling errors, simulated bugs, and model quantization, detecting 4-bit quantization with AUC $>$ 0.999 within 300 output tokens. For applications requiring sample-efficient forward-pass verification, we additionally introduce Activation-DiFR, a scheme that uses random orthogonal projections to compress activations into compact fingerprints for subsequent verification. Activation-DiFR detects 4-bit quantization with AUC $>$ 0.999 using just 2 output tokens, while reducing communication overhead by 25-75% relative to existing methods. We release an open-source integration with vLLM to accelerate practical deployment of verifiable inference.

翻译：随着大语言模型（LLM）推理需求的增长，服务提供商及其客户能够验证推理过程是否正确执行、无错误或篡改变得日益重要。然而，由于良性的数值噪声，两次运行相同的推理过程常产生不同结果，使得难以区分合理变异与实际问题。为解决此问题，我们提出Token-DiFR（Token-Divergence-From-Reference），一种通过将生成的令牌与基于相同随机种子的可信参考实现所作预测进行比较的推理输出验证方法。采样种子同步严格约束有效输出，使服务提供商几乎无法偏离正确推理，从而允许输出令牌本身作为可审计的正确性证据，且对提供商零额外成本。Token-DiFR可可靠识别采样错误、模拟缺陷和模型量化，在300个输出令牌内检测4位量化的AUC $>$ 0.999。对于需要样本高效前向传递验证的应用，我们额外提出Activation-DiFR方案，其使用随机正交投影将激活值压缩为紧凑指纹以供后续验证。Activation-DiFR仅用2个输出令牌即可检测4位量化（AUC $>$ 0.999），同时相较于现有方法降低25-75%的通信开销。我们发布了与vLLM的开源集成，以加速可验证推理的实际部署。