In recent years we have witnessed a renewed interest in machine learning methodologies, especially for deep representation learning, that could overcome basic i.i.d. assumptions and tackle non-stationary environments subject to various distributional shifts or sample selection biases. Within this context, several computational approaches based on architectural priors, regularizers and replay policies have been proposed with different degrees of success depending on the specific scenario in which they were developed and assessed. However, designing comprehensive hybrid solutions that can flexibly and generally be applied with tunable efficiency-effectiveness trade-offs still seems a distant goal. In this paper, we propose "Architect, Regularize and Replay" (ARR), an hybrid generalization of the renowned AR1 algorithm and its variants, that can achieve state-of-the-art results in classic scenarios (e.g. class-incremental learning) but also generalize to arbitrary data streams generated from real-world datasets such as CIFAR-100, CORe50 and ImageNet-1000.

翻译：近年来,我们看到人们重新关注机器学习方法,特别是深层代表性学习方法,这些方法可以克服基本的一.d.假设,并处理非静止环境,这些环境受到各种分布变化或抽样选择偏差的影响,在这方面,根据建筑前程、正规化和重现政策提出了几种计算方法,其成功程度不同,取决于其发展和评估的具体情景,然而,设计全面混合解决办法,这些解决办法可以灵活和一般地用于可节省金枪鱼效率的权衡,这似乎仍是一个遥远的目标。我们在本文件中提议“建筑、正规化和重现”(ARR),这是著名的AR1算法及其变式的混合统称,可以在经典情景中(例如,等级认知学习)取得最新结果,但也概括地反映了从诸如CIFAR-100、CORe50和图像网络1000等真实世界数据集中产生的任意数据流。