Generative retrieval (GR) re-frames document retrieval as a sequence-based document identifier (DocID) generation task, memorizing documents with model parameters and enabling end-to-end retrieval without explicit indexing. Existing GR methods are based on auto-regressive generative models, i.e., the token generation is performed from left to right. However, such auto-regressive methods suffer from: (1) mismatch between DocID generation and natural language generation, e.g., an incorrect DocID token generated in early left steps would lead to totally erroneous retrieval; and (2) failure to balance the trade-off between retrieval efficiency and accuracy dynamically, which is crucial for practical applications. To address these limitations, we propose generative document retrieval with diffusion language models, dubbed DiffuGR. It models DocID generation as a discrete diffusion process: during training, DocIDs are corrupted through a stochastic masking process, and a diffusion language model is learned to recover them under a retrieval-aware objective. For inference, DiffuGR attempts to generate DocID tokens in parallel and refines them through a controllable number of denoising steps. In contrast to conventional left-to-right auto-regressive decoding, DiffuGR provides a novel mechanism to first generate more confident DocID tokens and refine the generation through diffusion-based denoising. Moreover, DiffuGR also offers explicit runtime control over the qualitylatency tradeoff. Extensive experiments on benchmark retrieval datasets show that DiffuGR is competitive with strong auto-regressive generative retrievers, while offering flexible speed and accuracy tradeoffs through variable denoising budgets. Overall, our results indicate that non-autoregressive diffusion models are a practical and effective alternative for generative document retrieval.

翻译：生成式检索（GR）将文档检索重新定义为基于序列的文档标识符（DocID）生成任务，通过模型参数记忆文档，实现无需显式索引的端到端检索。现有GR方法基于自回归生成模型，即从左到右进行标记生成。然而，此类自回归方法存在以下问题：（1）DocID生成与自然语言生成之间的不匹配，例如在早期左侧步骤中生成错误的DocID标记会导致完全错误的检索；（2）无法动态平衡检索效率与准确性之间的权衡，而这在实际应用中至关重要。为应对这些局限，我们提出基于扩散语言模型的生成式文档检索方法，命名为DiffuGR。它将DocID生成建模为离散扩散过程：在训练阶段，通过随机掩码过程对DocID进行破坏，并基于检索感知目标学习扩散语言模型以恢复DocID。在推理阶段，DiffuGR尝试并行生成DocID标记，并通过可控的去噪步骤进行优化。相较于传统的从左到右自回归解码，DiffuGR提供了一种新机制：先生成置信度更高的DocID标记，再通过基于扩散的去噪过程优化生成结果。此外，DiffuGR还支持对质量-延迟权衡进行显式运行时控制。在基准检索数据集上的大量实验表明，DiffuGR与强自回归生成式检索器性能相当，同时通过可变的去噪预算提供灵活的速度与精度权衡。总体而言，我们的结果表明非自回归扩散模型是生成式文档检索中实用且有效的替代方案。