Characterizing the patterns of errors that a system makes helps researchers focus future development on increasing its accuracy and robustness. We propose a novel form of "meta learning" that automatically learns interpretable rules that characterize the types of errors that a system makes, and demonstrate these rules' ability to help understand and improve two NLP systems. Our approach works by collecting error cases on validation data, extracting meta-features describing these samples, and finally learning rules that characterize errors using these features. We apply our approach to VilBERT, for Visual Question Answering, and RoBERTa, for Common Sense Question Answering. Our system learns interpretable rules that provide insights into systemic errors these systems make on the given tasks. Using these insights, we are also able to "close the loop" and modestly improve performance of these systems.

翻译：描述一个系统帮助研究人员将未来发展重点放在提高其准确性和稳健性上的错误模式。 我们提议一种新型的“ 元学习”, 自动学习可解释的规则, 以描述一个系统的错误类型, 并展示这些规则帮助理解和改进两个 NLP 系统的能力。 我们的方法是收集验证数据的错误案例, 提取描述这些样本的元特征, 最后学习使用这些特征的错误特征。 我们运用了我们的方法, 用于 VilBERT, 用于视觉问答, 和 RoBERTA, 用于共同的 Sense 问题解答。 我们的系统学习了可解释的规则, 以洞察这些系统在给定任务上造成的系统错误。 我们利用这些洞察力, 我们还能够“ 关闭循环” 并适度改进这些系统的性能 。