This study evaluated generative methods to potentially mitigate AI bias when diagnosing diabetic retinopathy (DR) resulting from training data imbalance, or domain generalization which occurs when deep learning systems (DLS) face concepts at test/inference time they were not initially trained on. The public domain Kaggle-EyePACS dataset (88,692 fundi and 44,346 individuals, originally diverse for ethnicity) was modified by adding clinician-annotated labels and constructing an artificial scenario of data imbalance and domain generalization by disallowing training (but not testing) exemplars for images of retinas with DR warranting referral (DR-referable) and from darker-skin individuals, who presumably have greater concentration of melanin within uveal melanocytes, on average, contributing to retinal image pigmentation. A traditional/baseline diagnostic DLS was compared against new DLSs that would use training data augmented via generative models for debiasing. Accuracy (95% confidence intervals [CI]) of the baseline diagnostics DLS for fundus images of lighter-skin individuals was 73.0% (66.9%, 79.2%) vs. darker-skin of 60.5% (53.5%, 67.3%), demonstrating bias/disparity (delta=12.5%) (Welch t-test t=2.670, P=.008) in AI performance across protected subpopulations. Using novel generative methods for addressing missing subpopulation training data (DR-referable darker-skin) achieved instead accuracy, for lighter-skin, of 72.0% (65.8%, 78.2%), and for darker-skin, of 71.5% (65.2%,77.8%), demonstrating closer parity (delta=0.5%) in accuracy across subpopulations (Welch t-test t=0.111, P=.912). Findings illustrate how data imbalance and domain generalization can lead to disparity of accuracy across subpopulations, and show that novel generative methods of synthetic fundus images may play a role for debiasing AI.


翻译:这项研究评估了基因方法, 以在以下情况下可能减轻AI 偏差: 诊断65个诊所加注的标签, 并构建一个人为的数据偏差和域性直观化假象, 其原因是培训数据不平衡, 或者当深学习系统(DLS)在测试/推断时, 最初没有接受过培训时, 出现域内的概念。 Kagggle- EyePACS 数据集( 88, 692 Fundi和44, 346个人, 最初因族裔而异) 。 公共域域内 Kaggle- EyePASCS 数据集( 88, 692 Fundi和44, 346个人), 与新的 DLS 相比, 将数据变异异性模型( 但不测试), 暗性直径直径直径直径直径( 95) 显示(CI) DLS 基线诊断( DLS) 直径直径60% = 直径个人( AI)。

0
下载
关闭预览

相关内容

机器学习系统设计系统评估标准
数据科学导论,54页ppt,Introduction to Data Science
专知会员服务
42+阅读 · 2020年7月27日
【哈佛大学商学院课程Fall 2019】机器学习可解释性
专知会员服务
105+阅读 · 2019年10月9日
Transferring Knowledge across Learning Processes
CreateAMind
29+阅读 · 2019年5月18日
人工智能 | SCI期刊专刊信息3条
Call4Papers
5+阅读 · 2019年1月10日
A Technical Overview of AI & ML in 2018 & Trends for 2019
待字闺中
18+阅读 · 2018年12月24日
人工智能 | AAAI 2019等国际会议信息7条
Call4Papers
5+阅读 · 2018年9月3日
Arxiv
10+阅读 · 2020年11月26日
Arxiv
45+阅读 · 2019年12月20日
VIP会员
相关资讯
Transferring Knowledge across Learning Processes
CreateAMind
29+阅读 · 2019年5月18日
人工智能 | SCI期刊专刊信息3条
Call4Papers
5+阅读 · 2019年1月10日
A Technical Overview of AI & ML in 2018 & Trends for 2019
待字闺中
18+阅读 · 2018年12月24日
人工智能 | AAAI 2019等国际会议信息7条
Call4Papers
5+阅读 · 2018年9月3日
Top
微信扫码咨询专知VIP会员