Transformer-based sequential recommenders are very powerful for capturing both short-term and long-term sequential item dependencies. This is mainly attributed to their unique self-attention networks to exploit pairwise item-item interactions within the sequence. However, real-world item sequences are often noisy, which is particularly true for implicit feedback. For example, a large portion of clicks do not align well with user preferences, and many products end up with negative reviews or being returned. As such, the current user action only depends on a subset of items, not on the entire sequences. Many existing Transformer-based models use full attention distributions, which inevitably assign certain credits to irrelevant items. This may lead to sub-optimal performance if Transformers are not regularized properly.

翻译：以变换器为基础的相继推荐人对于捕捉短期和长期相继项目依赖性都非常强大,这主要归功于他们独特的自我关注网络,以在序列中开发对称的项目项目互动。然而,现实世界的项目序列往往很吵,对于隐含的反馈来说尤其如此。例如,大量点击与用户的偏好不吻合,许多产品最终会受到负面审查或被退回。因此,目前的用户行动只取决于一组项目,而不是整个序列。许多基于变换器的现有模型都使用充分关注分布,这不可避免地给无关的项目分配了一定的信用。如果变换器不适当正规化,这可能导致不最优的业绩。