标题： 分布外泛化通过风险外推（REx） 显示英文标题

标题： Out-of-Distribution Generalization via Risk Extrapolation (REx)

摘要： 分布偏移是将机器学习预测系统从实验室转移到现实世界时的主要障碍之一。 为了解决这个问题，我们假设训练领域之间的变化代表了测试时可能遇到的变化，但也认为测试时的变化可能在幅度上更加极端。 特别是，我们表明减少训练领域之间的风险差异可以降低模型对各种极端分布偏移的敏感性，包括输入同时包含因果和反因果元素的挑战性情况。 我们提出一种称为风险外推（REx）的方法，作为对扩展领域扰动集的鲁棒优化形式（MM-REx），并提出一种对训练风险方差的惩罚（V-REx）作为更简单的变体。 我们证明，REx的各种变体可以恢复目标的因果机制，同时对输入分布的变化（“协变量偏移”）提供一定的鲁棒性。 通过适当权衡对因果引起的分布偏移和协变量偏移的鲁棒性，REx在这些类型的偏移同时发生的情况下能够优于其他方法，如不变风险最小化。 显示英文摘要

摘要： Distributional shift is one of the major obstacles when transferring machine learning prediction systems from the lab to the real world. To tackle this problem, we assume that variation across training domains is representative of the variation we might encounter at test time, but also that shifts at test time may be more extreme in magnitude. In particular, we show that reducing differences in risk across training domains can reduce a model's sensitivity to a wide range of extreme distributional shifts, including the challenging setting where the input contains both causal and anti-causal elements. We motivate this approach, Risk Extrapolation (REx), as a form of robust optimization over a perturbation set of extrapolated domains (MM-REx), and propose a penalty on the variance of training risks (V-REx) as a simpler variant. We prove that variants of REx can recover the causal mechanisms of the targets, while also providing some robustness to changes in the input distribution ("covariate shift"). By appropriately trading-off robustness to causally induced distributional shifts and covariate shift, REx is able to outperform alternative methods such as Invariant Risk Minimization in situations where these types of shift co-occur.