Personalized diagnoses have not been possible due to sear amount of data pathologists have to bear during the day-to-day routine. This lead to the current generalized standards that are being continuously updated as new findings are reported. It is noticeable that these effective standards are developed based on a multi-source heterogeneous data, including whole-slide images and pathology and clinical reports. In this study, we propose a framework that combines pathological images and medical reports to generate a personalized diagnosis result for individual patient. We use nuclei-level image feature similarity and content-based deep learning method to search for a personalized group of population with similar pathological characteristics, extract structured prognostic information from descriptive pathology reports of the similar patient population, and assign importance of different prognostic factors to generate a personalized pathological diagnosis result. We use multi-source heterogeneous data from TCGA (The Cancer Genome Atlas) database. The result demonstrate that our framework matches the performance of pathologists in the diagnosis of renal cell carcinoma. This framework is designed to be generic, thus could be applied for other types of cancer. The weights could provide insights to the known prognostic factors and further guide more precise clinical treatment protocols.

翻译：由于数据病理学家在日常工作中不得不忍受大量数据病理学家的诊断,因此无法进行个人化诊断,这导致目前的普遍标准,随着新的调查结果的出现,正在不断更新;值得注意的是,这些有效标准是根据多种来源的多元数据制定的,包括全滑动图象和病理学以及临床报告;在这项研究中,我们提出了一个框架,将病理图象和医疗报告结合起来,为个人病人产生个性化诊断结果;我们使用核心层次图像特征相似和内容为基础的深层次学习方法,以寻找具有类似病理特征的个性化人口群体;从类似病人的描述病理报告中提取结构化的预测信息;重视不同的预测因素,以产生个性化病理诊断结果;我们使用TCGA(癌症基因组图集)数据库的多种来源的混合数据;结果表明,我们的框架与肾细胞癌诊断病理学家的绩效相匹配。这个框架的设计是通用的,因此可以适用于其他类型的癌症。重量可以提供更精确的临床治疗方法。