We study the approximation properties of shallow neural networks (NN) with activation function which is a power of the rectified linear unit. Specifically, we consider the dependence of the approximation rate on the dimension and the smoothness of the underlying function to be approximated. Like the finite element method, such networks represent piecewise polynomial functions. However, we show that for sufficiently smooth functions the approximation properties of shallow ReLU$^k$ networks are much better than finite elements or wavelets, and they even overcome the curse of dimensionality more effectively than the sparse grid method. Specifically, for a sufficiently smooth function $f$, there exists a ReLU$^k$-NN with $n$ neurons which approximates $f$ in $L^2([0,1]^d)$ with $O(n^{-(k+1)}\log(n))$ error. Finally, we prove lower bounds showing that the approximation rates attained are optimal under the given assumptions.

翻译：我们研究了具有激活功能的浅神经网络的近似特性,这是纠正线性单位的功率。 具体地说, 我们考虑近近率对维度和基本功能的顺畅性的依赖性, 与有限元素法一样, 这种网络代表了零碎的多元功能。 然而, 我们显示, 如果功能足够顺畅, 浅ReLU$k$网络的近似特性比有限元素或波子要好得多, 它们甚至比稀薄的网性方法更有效地克服了维度的诅咒。 具体地说, 对于足够顺畅的功能, 美元, 存在一个RLU$k$-nNN, 其神经元约合1美元($2, [0,1,1美元) 美元, $(n) 错误。 最后, 我们证明, 较低的界限表明, 在给定的假设下, 所达到的近似率是最佳的。