SHallow REcurrent Decoders (SHRED) are effective for system identification and forecasting from sparse sensor measurements. Such models are light-weight and computationally efficient, allowing them to be trained on consumer laptops. SHRED-based models rely on Recurrent Neural Networks (RNNs) and a simple Multi-Layer Perceptron (MLP) for the temporal encoding and spatial decoding respectively. Despite the relatively simple structure of SHRED, they are able to predict chaotic dynamical systems on different physical, spatial, and temporal scales directly from a sparse set of sensor measurements. In this work, we modify SHRED by leveraging transformers (T-SHRED) embedded with symbolic regression for the temporal encoding, circumventing auto-regressive long-term forecasting for physical data. This is achieved through a new sparse identification of nonlinear dynamics (SINDy) attention mechanism into T-SHRED to impose sparsity regularization on the latent space, which also allows for immediate symbolic interpretation. Symbolic regression improves model interpretability by learning and regularizing the dynamics of the latent space during training. We analyze the performance of T-SHRED on three different dynamical systems ranging from low-data to high-data regimes.

翻译：浅层循环解码器（SHRED）在稀疏传感器测量下的系统辨识与预测中表现优异。此类模型轻量且计算高效，可在消费级笔记本电脑上完成训练。基于SHRED的模型分别采用循环神经网络（RNN）和简单多层感知机（MLP）进行时间编码与空间解码。尽管SHRED结构相对简单，其能直接从稀疏传感器测量数据中预测不同物理、空间和时间尺度下的混沌动力系统。本研究通过引入嵌入符号回归的Transformer模块（T-SHRED）改进SHRED的时间编码机制，规避物理数据自回归长期预测的局限。该改进通过在T-SHRED中集成新型非线性动力学稀疏辨识（SINDy）注意力机制，对潜在空间施加稀疏正则化约束，同时实现即时符号化解释。符号回归通过训练期间学习并正则化潜在空间动力学，显著提升模型可解释性。本文在低数据量至高数据量区间的三种不同动力系统上评估了T-SHRED的性能表现。