Knowledge editing has emerged as an efficient approach for updating factual knowledge in large language models (LLMs). It typically locates knowledge storage modules and then modifies their parameters. However, most existing methods focus on the weights of multilayer perceptron (MLP) modules, which are often identified as the main repositories of factual information. Other components, such as attention (Attn) modules, are often ignored during editing. This imbalance can leave residual outdated knowledge and limit editing effectiveness. We perform comprehensive knowledge localization experiments on advanced LLMs and find that Attn modules play a substantial role in factual knowledge storage and retrieval, especially in earlier layers. Based on these insights, we propose IntAttn-Edit, a method that extends the associative memory paradigm to jointly update both MLP and Attn modules. Our approach uses a knowledge balancing strategy that allocates update magnitudes in proportion to each module's measured contribution to knowledge storage. Experiments on standard benchmarks show that IntAttn-Edit achieves higher edit success, better generalization, and stronger knowledge preservation than prior methods. Further analysis shows that the balancing strategy keeps editing performance within an optimal range across diverse settings.

翻译：知识编辑已成为更新大语言模型（LLMs）中事实知识的一种高效方法。该方法通常先定位知识存储模块，然后修改其参数。然而，现有方法大多聚焦于多层感知机（MLP）模块的权重，这些模块通常被认为是事实信息的主要存储库。其他组件，如注意力（Attn）模块，在编辑过程中往往被忽略。这种不平衡可能导致残留的过时知识，并限制编辑效果。我们在先进的大语言模型上进行了全面的知识定位实验，发现注意力模块在事实知识存储与检索中发挥着重要作用，尤其是在模型的较浅层。基于这些发现，我们提出了IntAttn-Edit方法，该方法将关联记忆范式扩展至联合更新MLP和Attn模块。我们的方法采用知识平衡策略，根据各模块对知识存储的实测贡献按比例分配更新幅度。在标准基准测试上的实验表明，与先前方法相比，IntAttn-Edit实现了更高的编辑成功率、更好的泛化能力以及更强的知识保留能力。进一步分析表明，该平衡策略能在不同设置下将编辑性能保持在最优范围内。