The standard Universal Approximation Theorem for operator neural networks (NNs) holds for arbitrary width and bounded depth. Here, we prove that operator NNs of bounded width and arbitrary depth are universal approximators for continuous nonlinear operators. In our main result, we prove that for non-polynomial activation functions that are continuously differentiable at a point with a nonzero derivative, one can construct an operator NN of width five, whose inputs are real numbers with finite decimal representations, that is arbitrarily close to any given continuous nonlinear operator. We derive an analogous result for non-affine polynomial activation functions. We also show that depth has theoretical advantages by constructing operator ReLU NNs of depth $2k^3+8$ and constant width that cannot be well-approximated by any operator ReLU NN of depth $k$, unless its width is exponential in $k$.

翻译：操作器神经网络的标准通用近似理论(NNs)具有任意宽度和宽度深度。 在这里, 我们证明, 约束宽度和任意深度的操作器NN是连续非线性操作器的普遍近似物。 我们的主要结果是, 我们证明, 对于在非零衍生物的某个点上可以持续差异的非球状激活功能, 可以建造宽度为5的操作器NN, 其输入为实际数字, 小数表示有限, 任意地接近任何给定的非线性操作器。 我们得出一个类似的结果, 用于非硬性多线性激活功能。 我们还表明, 深度具有理论上的优势, 建造操作器ReLU NN, 深度为2k+3美元+8美元, 且常年宽度不能被任何操作器RLU NN美元深度相近, 除非其宽度以美元指数指数表示。