Measuring empathy in conversation can be challenging, as empathy is a complex and multifaceted psychological construct that involves both cognitive and emotional components. Human evaluations can be subjective, leading to inconsistent results. Therefore, there is a need for an automatic method for measuring empathy that reduces the need for human evaluations. In this paper, we proposed a novel approach EMP-EVAL, a simple yet effective automatic empathy evaluation method. The proposed technique takes the influence of Emotion, Cognitive and Emotional empathy. To the best knowledge, our work is the first to systematically measure empathy without the human-annotated provided scores. Experimental results demonstrate that our metrics can correlate with human preference, achieving comparable results with human judgments.

翻译：在对话中衡量共鸣可能具有挑战性,因为共鸣是一个复杂和多方面的心理结构,涉及认知和情感两个组成部分。人类评价可以是主观的,导致不一致的结果。因此,有必要采用自动方法衡量共鸣,以减少对人类评价的需求。在本文中,我们提出了一个新颖的方法EMP-EVAL,这是一个简单而有效的自动共鸣评价方法。拟议的技术具有情感、认知和情感共鸣的影响。根据最佳知识,我们的工作是首先系统衡量共鸣,而没有人类附加说明的分数。实验结果表明,我们的衡量标准可以与人类偏好相关,并取得与人类判断相似的结果。