标题： 重新审视（人类）标签变化下的主动学习 显示英文标题

标题： Revisiting Active Learning under (Human) Label Variation

摘要： 访问高质量标注数据仍然是应用监督学习中的一个限制因素。 尽管标签差异（LV），即同一实例的不同标签，在自然语言处理中很常见，但注释框架通常仍然基于单一真实标签的假设。 这忽略了人类标签差异（HLV），即注释中可能出现的合理差异，作为一种信息性信号。 同样，主动学习（AL），一种在训练机器学习模型时优化有限注释预算的流行方法，通常依赖于几种简化的假设之一，而当承认HLV时，这些假设很少在实践中成立。 在本文中，我们考察了关于真理和标签本质的基本假设，强调需要将观察到的LV分解为信号（例如，HLV）和噪声（例如，注释错误）。 我们调查了AL和（H）LV社区如何处理——或忽视——这些区别，并提出了一个概念框架，以在整个AL循环中包含HLV，包括实例选择、注释者选择和标签表示。 我们进一步讨论了将大型语言模型（LLM）作为注释者的整合。 我们的工作旨在为HLV感知的主动学习奠定概念基础，更好地反映现实注释的复杂性。 显示英文摘要

摘要： Access to high-quality labeled data remains a limiting factor in applied supervised learning. While label variation (LV), i.e., differing labels for the same instance, is common, especially in natural language processing, annotation frameworks often still rest on the assumption of a single ground truth. This overlooks human label variation (HLV), the occurrence of plausible differences in annotations, as an informative signal. Similarly, active learning (AL), a popular approach to optimizing the use of limited annotation budgets in training ML models, often relies on at least one of several simplifying assumptions, which rarely hold in practice when acknowledging HLV. In this paper, we examine foundational assumptions about truth and label nature, highlighting the need to decompose observed LV into signal (e.g., HLV) and noise (e.g., annotation error). We survey how the AL and (H)LV communities have addressed -- or neglected -- these distinctions and propose a conceptual framework for incorporating HLV throughout the AL loop, including instance selection, annotator choice, and label representation. We further discuss the integration of large language models (LLM) as annotators. Our work aims to lay a conceptual foundation for HLV-aware active learning, better reflecting the complexities of real-world annotation.