Humans continually expand their learned knowledge to new domains and learn new concepts without any interference with past learned experiences. In contrast, machine learning models perform poorly in a continual learning setting, where input data distribution changes over time. Inspired by the nervous system learning mechanisms, we develop a computational model that enables a deep neural network to learn new concepts and expand its learned knowledge to new domains incrementally in a continual learning setting. We rely on the Parallel Distributed Processing theory to encode abstract concepts in an embedding space in terms of a multimodal distribution. This embedding space is modeled by internal data representations in a hidden network layer. We also leverage the Complementary Learning Systems theory to equip the model with a memory mechanism to overcome catastrophic forgetting through implementing pseudo-rehearsal. Our model can generate pseudo-data points for experience replay and accumulate new experiences to past learned experiences without causing cross-task interference.

翻译：人类不断将其所学知识扩展到新的领域,并在不干扰以往所学经验的情况下学习新概念。 相反,机器学习模式在不断学习的环境中表现不佳,输入数据的分配随着时间推移而变化。在神经系统学习机制的启发下,我们开发了一个计算模型,使深神经网络能够在不断学习的环境中学习新概念并将其所学知识逐步扩展到新的领域。我们依靠平行分配处理理论,将抽象概念编码在嵌入空间的多式分布中。这种嵌入空间由隐藏的网络层的内部数据代表制成模型。我们还利用补充学习系统理论为模型配备一个记忆机制,通过执行假排练来克服灾难性的遗忘。我们的模型可以产生假数据点,用于经验重现和积累新经验,而不会引起交叉任务干扰。