In the framework of convolutional neural networks, downsampling is often performed with an average-pooling, where all the activations are treated equally, or with a max-pooling operation that only retains an element with maximum activation while discarding the others. Both of these operations are restrictive and have previously been shown to be sub-optimal. To address this issue, a novel pooling scheme, named\emph{ ordinal pooling}, is introduced in this work. Ordinal pooling rearranges all the elements of a pooling region in a sequence and assigns a different weight to each element based upon its order in the sequence. These weights are used to compute the pooling operation as a weighted sum of the rearranged elements of the pooling region. They are learned via a standard gradient-based training, allowing to learn a behavior anywhere in the spectrum of average-pooling to max-pooling in a differentiable manner. Our experiments suggest that it is advantageous for the networks to perform different types of pooling operations within a pooling layer and that a hybrid behavior between average- and max-pooling is often beneficial. More importantly, they also demonstrate that ordinal pooling leads to consistent improvements in the accuracy over average- or max-pooling operations while speeding up the training and alleviating the issue of the choice of the pooling operations and activation functions to be used in the networks. In particular, ordinal pooling mainly helps on lightweight or quantized deep learning architectures, as typically considered e.g. for embedded applications.

翻译：在进化神经网络的框架内,下游取样通常以平均集合方式进行,所有激活都得到同等处理,或采用最大集合操作,仅保留最大激活元素,而抛弃其它元素。这两种操作都是限制性的,以前都显示是亚最佳的。为了解决这一问题,在这项工作中引入了一个名为emph{或dinal集合的新组合计划。奥丁集合将集合区域的所有要素按顺序排列,根据顺序顺序排列每个元素的重量不同。这些重量被用来计算集合操作,作为集合区域重新组合的元素的加权总和。这两种操作都是通过标准的梯度培训学习的,可以以不同的方式学习平均集合到最大集合的任何地方的行为。我们的实验表明,在集合层内进行不同种类的集合作业是有利的,根据顺序排列每个元素的顺序对每个元素分配不同的重量。这些重量用来计算集合操作,作为集合区域重新组合的元素的加权总和。这些重量通常用于计算集合区域重新组合区域的重新组合要素。它们通过标准的梯度培训学习,可以学习各种平均集合到最大集合的组合,同时,或者以最稳定的方式,在最小化的组合中,它们通常会显示,或者集中到最接近的集化的组合作业中,用来进行。