Many deployed learning systems must update models on streaming data under memory constraints. The default strategy, sequential fine-tuning on each new phase, is architecture-agnostic but often suffers catastrophic forgetting when later phases correspond to different sub-populations or tasks. Replay with a finite buffer is a simple alternative, yet its behaviour across generative and predictive objectives is not well understood. We present a unified study of stateful replay for streaming autoencoding, time series forecasting, and classification. We view both sequential fine-tuning and replay as stochastic gradient methods for an ideal joint objective, and use a gradient alignment analysis to show when mixing current and historical samples should reduce forgetting. We then evaluate a single replay mechanism on six streaming scenarios built from Rotated MNIST, ElectricityLoadDiagrams 2011-2014, and Airlines delay data, using matched training budgets and three seeds. On heterogeneous multi task streams, replay reduces average forgetting by a factor of two to three, while on benign time based streams both methods perform similarly. These results position stateful replay as a strong and simple baseline for continual learning in streaming environments.

翻译：许多已部署的学习系统必须在内存受限条件下基于流式数据更新模型。默认策略——在每个新阶段进行顺序微调——与架构无关，但当后续阶段对应不同子群体或任务时，常遭受灾难性遗忘。使用有限缓冲区的回放是一种简单替代方案，但其在生成与预测目标上的行为尚未得到充分理解。本文对面向流式自编码、时间序列预测和分类的状态回放方法进行了统一研究。我们将顺序微调和回放均视为理想联合目标的随机梯度方法，并通过梯度对齐分析阐明混合当前与历史样本何时应能减少遗忘。随后，我们在由旋转MNIST、2011-2014年电力负荷图及航空公司延误数据构建的六种流式场景中，以匹配的训练预算和三个随机种子评估了单一回放机制。在异构多任务流上，回放将平均遗忘程度降低至原来的二分之一到三分之一；而在良性时间序列流上，两种方法表现相近。这些结果表明，状态回放可作为流式环境中持续学习的一个强效且简洁的基线方法。