Relation tuple extraction from text is an important task for building knowledge bases. Recently, joint entity and relation extraction models have achieved very high F1 scores in this task. However, the experimental settings used by these models are restrictive and the datasets used in the experiments are not realistic. They do not include sentences with zero tuples (zero-cardinality). In this paper, we evaluate the state-of-the-art joint entity and relation extraction models in a more realistic setting. We include sentences that do not contain any tuples in our experiments. Our experiments show that there is significant drop ($\sim 10-15\%$ in one dataset and $\sim 6-14\%$ in another dataset) in their F1 score in this setting. We also propose a two-step modeling using a simple BERT-based classifier that leads to improvement in the overall performance of these models in this realistic experimental setup.

翻译：从文本中提取关系图例是建立知识基础的一项重要任务。 最近, 联合实体和关系提取模型在这一任务中取得了非常高的F1分数。 但是, 这些模型所使用的实验设置是限制性的, 实验中使用的数据集是不现实的。 它们不包括无图例( 零心胸) 的句子 。 在本文中, 我们在一个更现实的环境下评估最先进的联合实体和关系提取模型 。 我们的实验中包括了不包含任何图例的句子。 我们的实验显示, 在其中的F1分中有显著的下降( 在一个数据集中为 10-15 美分, 在另一个数据集中为 $ 6-14 美分 ) 。 我们还建议使用一个简单的 BERT 分类器进行两步模型, 从而在现实的实验设置中改进这些模型的总体性能 。