Stroke is a major public health problem, affecting millions worldwide. Deep learning has recently demonstrated promise for enhancing the diagnosis and risk prediction of stroke. However, existing methods rely on costly medical imaging modalities, such as computed tomography. Recent studies suggest that retinal imaging could offer a cost-effective alternative for cerebrovascular health assessment due to the shared clinical pathways between the retina and the brain. Hence, this study explores the impact of leveraging retinal images and clinical data for stroke detection and risk prediction. We propose a multimodal deep neural network that processes Optical Coherence Tomography (OCT) and infrared reflectance retinal scans, combined with clinical data, such as demographics, vital signs, and diagnosis codes. We pretrained our model using a self-supervised learning framework using a real-world dataset consisting of $37$ k scans, and then fine-tuned and evaluated the model using a smaller labeled subset. Our empirical findings establish the predictive ability of the considered modalities in detecting lasting effects in the retina associated with acute stroke and forecasting future risk within a specific time horizon. The experimental results demonstrate the effectiveness of our proposed framework by achieving $5$\% AUROC improvement as compared to the unimodal image-only baseline, and $8$\% improvement compared to an existing state-of-the-art foundation model. In conclusion, our study highlights the potential of retinal imaging in identifying high-risk patients and improving long-term outcomes.

翻译：卒中是一个重大的公共卫生问题，影响着全球数百万人。深度学习最近在增强卒中的诊断和风险预测方面展现出潜力。然而，现有方法依赖于昂贵的医学成像模态，如计算机断层扫描。近期研究表明，由于视网膜与大脑之间存在共同的临床通路，视网膜成像可能为脑血管健康评估提供一种经济有效的替代方案。因此，本研究探讨了利用视网膜图像和临床数据进行卒中检测与风险预测的影响。我们提出了一种多模态深度神经网络，该网络处理光学相干断层扫描（OCT）和红外反射视网膜扫描，并结合临床数据，如人口统计学、生命体征和诊断代码。我们使用一个包含$37$千次扫描的真实世界数据集，通过自监督学习框架对模型进行预训练，然后使用较小的标注子集对模型进行微调和评估。我们的实证研究结果证实了所考虑模态在检测与急性卒中相关的视网膜持久性效应以及预测特定时间范围内的未来风险方面的预测能力。实验结果表明，我们提出的框架有效实现了与单模态仅图像基线相比$5$%的AUROC提升，与现有最先进的基础模型相比提升了$8$%。总之，我们的研究强调了视网膜成像在识别高风险患者和改善长期预后方面的潜力。