Tabular data is a common format for storing information in rows and columns to represent data entries and their features. Although deep neural networks have become the main approach for modeling a wide range of domains including computer vision and NLP, many of them are not well-suited for tabular data. Recently, a few deep learning models have been proposed for deep tabular learning, featuring an internal feature selection mechanism with end-to-end gradient-based optimization. However, their feature selection mechanisms are unidimensional, and hence fail to account for the contextual dependence of feature importance, potentially overlooking crucial interactions that govern complex tasks. In addition, they overlook the bias of high-impact features and the risk associated with the limitations of attention generalization. To address this limitation, this study proposes a novel iterative feature exclusion module that enhances the feature importance ranking in tabular data. The proposed module iteratively excludes each feature from the input data and computes the attention scores, which represent the impact of the features on the prediction. By aggregating the attention scores from each iteration, the proposed module generates a refined representation of feature importance that captures both global and local interactions between features. The effectiveness of the proposed module is evaluated on four public datasets. The results demonstrate that the proposed module consistently outperforms state-of-the-art methods and baseline models in feature ranking and classification tasks. The code is publicly available at https://github.com/abaraka2020/Iterative-Feature-Exclusion-Ranking-Module and https://github.com/mohalim/IFENet

翻译：表格数据是一种以行和列存储信息的常见格式，用于表示数据条目及其特征。尽管深度神经网络已成为计算机视觉和自然语言处理等多个领域建模的主要方法，但其中许多模型并不适用于表格数据。近年来，已有一些深度学习模型被提出用于深度表格学习，其特点是采用基于端到端梯度优化的内部特征选择机制。然而，这些特征选择机制是单维度的，因此未能考虑特征重要性的上下文依赖性，可能忽略支配复杂任务的关键交互作用。此外，它们忽视了高影响力特征的偏差以及注意力泛化局限性带来的风险。为应对这一局限，本研究提出了一种新颖的迭代特征排除模块，以增强表格数据中的特征重要性排序。该模块通过迭代排除输入数据中的每个特征并计算注意力分数（代表特征对预测的影响），在聚合每次迭代的注意力分数后，生成能同时捕获特征间全局与局部交互作用的精细化特征重要性表征。该模块的有效性在四个公开数据集上得到验证。实验结果表明，在特征排序和分类任务中，所提模块始终优于现有先进方法和基线模型。代码已公开于 https://github.com/abaraka2020/Iterative-Feature-Exclusion-Ranking-Module 和 https://github.com/mohalim/IFENet。