We study the source of uncertainty in DeepSeek R1-32B by analyzing its self-reported verbal confidence on question answering (QA) tasks. In the default answer-then-confidence setting, the model is regularly over-confident, whereas semantic entropy - obtained by sampling many responses - remains reliable. We hypothesize that this is because of semantic entropy's larger test-time compute, which lets us explore the model's predictive distribution. We show that granting DeepSeek the budget to explore its distribution by forcing a long chain-of-thought before the final answer greatly improves its verbal score effectiveness, even on simple fact-retrieval questions that normally require no reasoning. Furthermore, a separate reader model that sees only the chain can reconstruct very similar confidences, indicating the verbal score might be merely a statistic of the alternatives surfaced during reasoning. Our analysis concludes that reliable uncertainty estimation requires explicit exploration of the generative space, and self-reported confidence is trustworthy only after such exploration.

翻译：本研究通过分析DeepSeek R1-32B模型在问答任务中自我报告的语言置信度，探究其不确定性的来源。在默认的“先回答后报告置信度”设置下，模型经常表现出过度自信，而通过多次响应采样获得的语义熵则保持可靠。我们假设这是因为语义熵在测试阶段消耗了更多计算资源，使我们能够探索模型的预测分布。实验表明，通过强制模型在给出最终答案前进行长链思维推理，赋予其探索自身分布的计算预算，能显著提升语言置信度的有效性——即使在通常无需推理的简单事实检索问题上也是如此。此外，仅观察思维链的独立读取器模型能重构出高度相似的置信度，这表明语言置信度可能仅是推理过程中浮现的替代选项的统计量。我们的分析得出结论：可靠的不确定性估计需要显式探索生成空间，而自我报告的置信度仅在此类探索后才值得信赖。