Detecting anomalies in multivariate time series is essential for monitoring complex industrial systems, where high dimensionality, limited labeled data, and subtle dependencies between sensors cause significant challenges. This paper presents a deep reinforcement learning framework that combines a Variational Autoencoder (VAE), an LSTM-based Deep Q-Network (DQN), dynamic reward shaping, and an active learning module to address these issues in a unified learning framework. The main contribution is the implementation of Dynamic Reward Scaling for Multivariate Time Series Anomaly Detection (DRSMT), which demonstrates how each component enhances the detection process. The VAE captures compact latent representations and reduces noise. The DQN enables adaptive, sequential anomaly classification, and the dynamic reward shaping balances exploration and exploitation during training by adjusting the importance of reconstruction and classification signals. In addition, active learning identifies the most uncertain samples for labeling, reducing the need for extensive manual supervision. Experiments on two multivariate benchmarks, namely Server Machine Dataset (SMD) and Water Distribution Testbed (WADI), show that the proposed method outperforms existing baselines in F1-score and AU-PR. These results highlight the effectiveness of combining generative modeling, reinforcement learning, and selective supervision for accurate and scalable anomaly detection in real-world multivariate systems.

翻译：多元时间序列中的异常检测对于监控复杂工业系统至关重要，其中高维度、有限的标记数据以及传感器间微妙的依赖关系带来了显著挑战。本文提出了一种深度强化学习框架，该框架结合了变分自编码器（VAE）、基于LSTM的深度Q网络（DQN）、动态奖励塑形以及主动学习模块，在一个统一的学习框架中解决这些问题。主要贡献在于实现了用于多元时间序列异常检测的动态奖励缩放（DRSMT），展示了每个组件如何增强检测过程。VAE捕获紧凑的潜在表示并降低噪声；DQN实现自适应的序列异常分类；动态奖励塑形通过调整重构和分类信号的重要性，在训练期间平衡探索与利用。此外，主动学习识别最不确定的样本进行标记，减少了对大量人工监督的需求。在两个多元基准数据集——服务器机器数据集（SMD）和水分配测试平台（WADI）上的实验表明，所提方法在F1分数和AU-PR方面优于现有基线。这些结果突显了结合生成建模、强化学习和选择性监督，对于现实世界多元系统中准确且可扩展的异常检测的有效性。