Accurate road damage detection is crucial for timely infrastructure maintenance and public safety, but existing vision-only datasets and models lack the rich contextual understanding that textual information can provide. To address this limitation, we introduce RoadBench, the first multimodal benchmark for comprehensive road damage understanding. This dataset pairs high resolution images of road damages with detailed textual descriptions, providing a richer context for model training. We also present RoadCLIP, a novel vision language model that builds upon CLIP by integrating domain specific enhancements. It includes a disease aware positional encoding that captures spatial patterns of road defects and a mechanism for injecting road-condition priors to refine the model's understanding of road damages. We further employ a GPT driven data generation pipeline to expand the image to text pairs in RoadBench, greatly increasing data diversity without exhaustive manual annotation. Experiments demonstrate that RoadCLIP achieves state of the art performance on road damage recognition tasks, significantly outperforming existing vision-only models by 19.2%. These results highlight the advantages of integrating visual and textual information for enhanced road condition analysis, setting new benchmarks for the field and paving the way for more effective infrastructure monitoring through multimodal learning.

翻译：准确的道路损伤检测对于基础设施及时维护和公共安全至关重要，但现有的纯视觉数据集和模型缺乏文本信息所能提供的丰富上下文理解。为应对这一局限，我们提出了RoadBench，这是首个用于全面道路损伤理解的多模态基准。该数据集将高分辨率道路损伤图像与详细文本描述配对，为模型训练提供了更丰富的上下文。我们还提出了RoadCLIP，一种基于CLIP构建的新型视觉语言模型，通过集成领域特定增强功能，包括一种能够捕捉道路缺陷空间模式的病害感知位置编码，以及一种注入道路状况先验知识以优化模型对道路损伤理解的机制。此外，我们采用GPT驱动的数据生成流程来扩展RoadBench中的图像-文本对，在无需大量人工标注的情况下显著增加了数据多样性。实验表明，RoadCLIP在道路损伤识别任务上达到了最先进的性能，显著优于现有纯视觉模型19.2%。这些结果凸显了整合视觉与文本信息以增强道路状况分析的优势，为该领域设定了新的基准，并通过多模态学习为更有效的基础设施监测铺平了道路。