Title: Human-Aligned Calibration for AI-Assisted Decision Making Show Chinese title

Title: 人类对齐校准用于人工智能辅助决策

Abstract: Whenever a binary classifier is used to provide decision support, it typically provides both a label prediction and a confidence value. Then, the decision maker is supposed to use the confidence value to calibrate how much to trust the prediction. In this context, it has been often argued that the confidence value should correspond to a well calibrated estimate of the probability that the predicted label matches the ground truth label. However, multiple lines of empirical evidence suggest that decision makers have difficulties at developing a good sense on when to trust a prediction using these confidence values. In this paper, our goal is first to understand why and then investigate how to construct more useful confidence values. We first argue that, for a broad class of utility functions, there exist data distributions for which a rational decision maker is, in general, unlikely to discover the optimal decision policy using the above confidence values -- an optimal decision maker would need to sometimes place more (less) trust on predictions with lower (higher) confidence values. However, we then show that, if the confidence values satisfy a natural alignment property with respect to the decision maker's confidence on her own predictions, there always exists an optimal decision policy under which the level of trust the decision maker would need to place on predictions is monotone on the confidence values, facilitating its discoverability. Further, we show that multicalibration with respect to the decision maker's confidence on her own predictions is a sufficient condition for alignment. Experiments on four different AI-assisted decision making tasks where a classifier provides decision support to real human experts validate our theoretical results and suggest that alignment may lead to better decisions. Show Chinese abstract

Abstract: 每当使用二元分类器提供决策支持时，它通常会提供一个标签预测和一个置信度值。 然后，决策者应使用置信度值来校准对预测的信任程度。 在这一背景下，人们经常认为，置信度值应对应于预测标签与真实标签匹配的概率的校准良好的估计。 然而，多项实证证据表明，决策者在利用这些置信度值判断何时信任预测方面存在困难。 本文的目标是首先理解原因，然后研究如何构建更有用的置信度值。 我们首先认为，对于一大类效用函数，存在数据分布，在这种情况下，理性决策者通常不太可能通过上述置信度值发现最优决策策略——一个最优的决策者需要有时对低（高）置信度值的预测给予更多（更少）的信任。 然而，我们随后证明，如果置信度值相对于决策者对自己预测的置信度具有自然的一致性属性，那么总存在一个最优决策策略，在该策略下，决策者对预测所需信任的程度与置信度值单调相关，从而便于其发现。 此外，我们证明了相对于决策者对自己预测的置信度进行多校准是保持一致性的充分条件。 在四个不同的AI辅助决策任务中的实验，其中分类器为真实的人类专家提供决策支持，验证了我们的理论结果，并表明一致性可能导致更好的决策。