This paper explores targeted distillation methods for sentiment analysis, aiming to build compact and practical models that preserve strong and generalizable sentiment analysis capabilities. To this end, we conceptually decouple the distillation target into knowledge and alignment and accordingly propose a two-stage distillation framework. Moreover, we introduce SentiBench, a comprehensive and systematic sentiment analysis benchmark that covers a diverse set of tasks across 12 datasets. We evaluate a wide range of models on this benchmark. Experimental results show that our approach substantially enhances the performance of compact models across diverse sentiment analysis tasks, and the resulting models demonstrate strong generalization to unseen tasks, showcasing robust competitiveness against existing small-scale models.

翻译：本文探讨了面向情感分析的目标蒸馏方法，旨在构建紧凑且实用的模型，同时保留强大且可泛化的情感分析能力。为此，我们从概念上将蒸馏目标解耦为知识与对齐，并相应提出了一个两阶段蒸馏框架。此外，我们引入了SentiBench，一个全面且系统的情感分析基准，涵盖了12个数据集中的多样化任务。我们在该基准上评估了广泛的模型。实验结果表明，我们的方法显著提升了紧凑模型在多种情感分析任务上的性能，所得模型对未见任务展现出强大的泛化能力，在与现有小规模模型的对比中表现出稳健的竞争力。