Previous literature has proved that Pretrained Language Models (PLMs) can store factual knowledge. However, we find that facts stored in the PLMs are not always correct. It motivates us to explore a fundamental question: How do we calibrate factual knowledge in PLMs without re-training from scratch? In this work, we propose a simple and lightweight method CaliNet to achieve this goal. To be specific, we first detect whether PLMs can learn the right facts via a contrastive score between right and fake facts. If not, we then use a lightweight method to add and adapt new parameters to specific factual texts. Experiments on the knowledge probing task show the calibration effectiveness and efficiency. In addition, through closed-book question answering, we find that the calibrated PLM possesses knowledge generalization ability after fine-tuning. Beyond the calibration performance, we further investigate and visualize the knowledge calibration mechanism.

翻译：先前的文献已经证明,预先培训的语言模型(PLM)可以存储事实知识。然而,我们发现,PLM中储存的事实并不总是正确。它激励我们探索一个根本问题:我们如何在不从头再培训的情况下校准PLM中的事实知识?在这个工作中,我们提出一个简单和轻量级的方法CaliNet来实现这一目标。具体地说,我们首先检测PLM能否通过对准和假事实的对比分来了解正确的事实。如果不是,我们然后使用轻量级的方法来添加和调整具体事实文本中的新参数。关于知识检验任务的实验显示了校准的有效性和效率。此外,我们通过密读问题解答发现,经过校准的PLM在微调后拥有一般知识能力。除了校准功能之外,我们还进一步调查和直视知识校准机制。