When a deep learning model is sequentially trained on different datasets, it forgets the knowledge acquired from previous data, a phenomenon known as catastrophic forgetting. It deteriorates performance of the deep learning model on diverse datasets, which is critical in privacy-preserving deep learning (PPDL) applications based on transfer learning (TL). To overcome this, we propose review learning (RL), a generative-replay-based continual learning technique that does not require a separate generator. Data samples are generated from the memory stored within the synaptic weights of the deep learning model which are used to review knowledge acquired from previous datasets. The performance of RL was validated through PPDL experiments. Simulations and real-world medical multi-institutional experiments were conducted using three types of binary classification electronic health record data. In the real-world experiments, the global area under the receiver operating curve was 0.710 for RL and 0.655 for TL. Thus, RL was highly effective in retaining previously learned knowledge.

翻译：当一个深层次学习模式在不同的数据集方面连续接受训练时,它忘记了从先前数据中获得的知识,这是一种被称为灾难性的遗忘现象;它使关于不同数据集的深层次学习模式的性能恶化,而这种深层次学习模式对于基于转移学习(TL)的隐私保护深层次学习(PPDL)应用至关重要。为了克服这一点,我们提议审查学习(RL),这是一种基于基因回放的不断学习技术,不需要单独的生成器;数据样本来自深层次学习模式的记忆中储存的记忆,该记忆中储存着从以前的数据集中获得的知识;通过PPDL实验验证了RL的性能;模拟和现实世界医学多机构实验使用了三种类型的双向分类电子健康记录数据。在现实世界实验中,接收器操作曲线下的全球区域为0.710(RL)和0.655(TL),因此,RL在保留以前学到的知识方面非常有效。