标题： 通过无监督域适应的协变量偏移校准预测 显示英文标题

标题： Calibrated Prediction with Covariate Shift via Unsupervised Domain Adaptation

摘要： 可靠的风险估计是帮助自主代理或人类决策者理解和利用预测模型的重要工具。 然而，现有的风险估计方法大多忽略了协变量偏移的可能性——即现实世界的数据分布可能与训练分布不同。 因此，现有的算法可能会高估确定性，可能在预测模型上产生一种虚假的自信感。 我们提出了一种校准预测的算法，该算法考虑到协变量偏移的可能性，给定来自训练分布的标记示例和来自现实世界分布的未标记示例。 我们的算法使用重要性加权来纠正从训练分布到现实世界分布的偏移。 然而，重要性加权依赖于训练分布和现实世界分布足够接近。 基于领域自适应的思想，我们还学习一个特征映射，试图使这两个分布相等。 在经验评估中，我们展示了当存在协变量偏移时，我们提出的方法优于现有的校准预测方法。 显示英文摘要

摘要： Reliable uncertainty estimates are an important tool for helping autonomous agents or human decision makers understand and leverage predictive models. However, existing approaches to estimating uncertainty largely ignore the possibility of covariate shift--i.e., where the real-world data distribution may differ from the training distribution. As a consequence, existing algorithms can overestimate certainty, possibly yielding a false sense of confidence in the predictive model. We propose an algorithm for calibrating predictions that accounts for the possibility of covariate shift, given labeled examples from the training distribution and unlabeled examples from the real-world distribution. Our algorithm uses importance weighting to correct for the shift from the training to the real-world distribution. However, importance weighting relies on the training and real-world distributions to be sufficiently close. Building on ideas from domain adaptation, we additionally learn a feature map that tries to equalize these two distributions. In an empirical evaluation, we show that our proposed approach outperforms existing approaches to calibrated prediction when there is covariate shift.