The rise of geometric problems in machine learning has necessitated the development of equivariant methods, which preserve their output under the action of rotation or some other transformation. At the same time, the Adam optimization algorithm has proven remarkably effective across machine learning and even traditional tasks in geometric optimization. In this work, we observe that naively applying Adam to optimize vector-valued data is not rotation equivariant, due to per-coordinate moment updates, and in fact this leads to significant artifacts and biases in practice. We propose to resolve this deficiency with VectorAdam, a simple modification which makes Adam rotation-equivariant by accounting for the vector structure of optimization variables. We demonstrate this approach on problems in machine learning and traditional geometric optimization, showing that equivariant VectorAdam resolves the artifacts and biases of traditional Adam when applied to vector-valued data, with equivalent or even improved rates of convergence.

翻译：机器学习中几何问题的上升使得有必要开发等式方法,这些方法通过轮换或某些其他转换来保持其输出。与此同时,亚当优化算法在机器学习甚至传统的几何优化任务中证明非常有效。在这项工作中,我们观察到,天真地运用亚当优化矢量估值数据并非是旋转等式的,因为每时每时每刻更新,这实际上导致大量艺术品和实践中的偏见。我们提议与矢量亚当(VictorAdam)一起解决这一缺陷,这是一种简单的修改,使亚当通过计算优化变量的矢量结构而实现旋转-等式。我们在机器学习和传统几何优化方面的问题上展示了这一方法,表明等量矢量-矢量-亚当应用到矢量估值数据时,等量-矢量-亚当解决传统亚当(Adam)的文物和偏向,其趋同率甚至提高。