Prosody -- the melody of speech -- conveys critical information often not captured by the words or text of a message. In this paper, we propose an information-theoretic approach to quantify how much information is expressed by prosody alone and not by text, and crucially, what that information is about. Our approach applies large speech and language models to estimate the mutual information between a particular dimension of an utterance's meaning (e.g., its emotion) and any of its communication channels (e.g., audio or text). We then use this approach to quantify how much information is conveyed by audio and text about sarcasm, emotion, and questionhood, using speech from television and podcasts. We find that for sarcasm and emotion the audio channel -- and by implication the prosodic channel -- transmits over an order of magnitude more information about these features than the text channel alone, at least when long-term context beyond the current sentence is unavailable. For questionhood, prosody provides comparatively less additional information. We conclude by outlining a program applying our approach to more dimensions of meaning, communication channels, and languages.

翻译：韵律——即语音的旋律——传递着话语或文本信息中常被忽略的关键信息。本文提出一种信息论方法，用于量化仅通过韵律（而非文本）所表达的信息量，并重点探究这些信息的具体内涵。该方法通过大规模语音与语言模型，估算话语特定语义维度（如情感）与其任一通信通道（如音频或文本）之间的互信息。基于电视节目与播客中的语音数据，我们运用此方法量化了音频与文本在传达讽刺、情感及疑问属性时的信息量。研究发现，在缺乏当前语句之外的长上下文语境时，对于讽刺与情感特征，音频通道（进而引申至韵律通道）所传递的信息量比纯文本通道高出超过一个数量级；而对于疑问属性，韵律提供的附加信息相对较少。最后，我们规划了将本方法拓展至更多语义维度、通信通道及语言的研究路径。