Automatically generated emotion arcs -- that capture how an individual or a population feels over time -- are widely used in industry and research. However, there is little work on evaluating the generated arcs. This is in part due to the difficulty of establishing the true (gold) emotion arc. Our work, for the first time, systematically and quantitatively evaluates automatically generated emotion arcs. We also compare two common ways of generating emotion arcs: Machine-Learning (ML) models and Lexicon-Only (LexO) methods. Using a number of diverse datasets, we systematically study the relationship between the quality of an emotion lexicon and the quality of the emotion arc that can be generated with it. We also study the relationship between the quality of an instance-level emotion detection system (say from an ML model) and the quality of emotion arcs that can be generated with it. We show that despite being markedly poor at instance level, LexO methods are highly accurate at generating emotion arcs by aggregating information from hundreds of instances. This has wide-spread implications for commercial development, as well as research in psychology, public health, digital humanities, etc. that values simple interpretable methods and disprefers the need for domain-specific training data, programming expertise, and high-carbon-footprint models.

翻译：自动生成的情感电弧 -- -- 记录一个人或一个人口随时间而感觉如何 -- -- 被广泛用于工业和研究。然而,在评估产生的电弧质量和随之产生的情感电弧质量之间的关系方面,我们没有做多少工作。这部分是由于难以建立真实(金)情感电弧。我们的工作首次系统化和定量地评估了自动生成情感电弧的质量。我们还比较了两种常见的产生情感电弧的方法:机器-学习(ML)模型和Lexicon-Only(LexO)方法。我们使用许多不同的数据集,系统地研究情感词词典质量与由此产生的情感电弧质量之间的关系。我们还研究了实例级情感电弧质量(从ML模型中测定 ) 与由此生成的情感电弧质量之间的关系。我们发现,尽管在实例层面明显贫困,LexO方法在生成情感电弧(Lexocon-Lexon-Lexon-Lexon-Lexo)方法方面是非常精确的。我们利用从数百个实例中收集信息,这对商业发展有着广泛的影响,作为简单的心理学、公共卫生、公众健康、数字模型的研究,也需要。