Online medical service provides patients convenient access to doctors, but effectively ranking doctors based on specific medical needs remains challenging. Current ranking approaches typically lack the interpretability crucial for patient trust and informed decision-making. Additionally, the scarcity of standardized benchmarks and labeled data for supervised learning impedes progress in expertise-aware doctor ranking. To address these challenges, we propose an explainable ranking framework for doctor ranking powered by large language models in a zero-shot setting. Our framework dynamically generates disease-specific ranking criteria to guide the large language model in assessing doctor relevance with transparency and consistency. It further enhances interpretability by generating step-by-step rationales for its ranking decisions, improving the overall explainability of the information retrieval process. To support rigorous evaluation, we built and released DrRank, a novel expertise-driven dataset comprising 38 disease-treatment pairs and 4,325 doctor profiles. On this benchmark, our framework significantly outperforms the strongest baseline by +6.45 NDCG@10. Comprehensive analyses also show our framework is fair across disease types, patient gender, and geographic regions. Furthermore, verification by medical experts confirms the reliability and interpretability of our approach, reinforcing its potential for trustworthy, real-world doctor recommendation. To demonstrate its broader applicability, we validate our framework on two datasets from BEIR benchmark, where it again achieves superior performance. The code and associated data are available at: https://github.com/YangLab-BUPT/DrRank.

翻译：在线医疗服务为患者提供了便捷的医生访问途径，但根据特定医疗需求对医生进行有效排序仍具挑战性。现有排序方法通常缺乏对患者信任和知情决策至关重要的可解释性。此外，用于监督学习的标准化基准和标注数据稀缺，阻碍了专业感知医生排序的进展。为解决这些挑战，我们提出了一种在零样本设置下由大语言模型驱动的可解释医生排序框架。该框架动态生成疾病特定的排序标准，以指导大语言模型以透明且一致的方式评估医生相关性。通过为排序决策生成逐步推理依据，进一步增强了可解释性，提升了信息检索过程的整体可解释性。为支持严谨评估，我们构建并发布了DrRank——一个包含38种疾病-治疗配对和4,325份医生档案的新型专业驱动数据集。在此基准测试中，我们的框架以+6.45 NDCG@10显著优于最强基线。综合分析还表明，我们的框架在疾病类型、患者性别和地理区域方面均保持公平性。此外，医学专家验证证实了我们方法的可靠性和可解释性，强化了其在可信赖的现实世界医生推荐中的潜力。为证明其更广泛的适用性，我们在BEIR基准的两个数据集上验证了该框架，其再次实现了卓越性能。代码及相关数据可在以下网址获取：https://github.com/YangLab-BUPT/DrRank。