标题： 高维合成数据选择分析 显示英文标题

标题： High-dimensional Analysis of Synthetic Data Selection

摘要： 尽管在生成模型的发展上取得了进展，但它们在创建能够提高分类器预测性能的合成数据方面的实用性受到了质疑。 除了诸如“合成数据应接近真实数据分布”的启发式原则外，实际上并不清楚哪些特定属性会影响泛化误差。 我们的论文通过高维回归的视角来解决这个问题。 理论上，我们证明了对于线性模型，目标分布与合成数据分布之间的协方差偏移会影响泛化误差，但令人惊讶的是，均值偏移不会。 此外，我们证明在某些情况下，匹配目标分布的协方差是最优的。 值得注意的是，从线性模型中获得的理论见解可以推广到深度神经网络和生成模型中。 我们通过实证表明，协方差匹配过程（将合成数据的协方差与来自目标分布的数据的协方差进行匹配）在多种训练范式、架构、数据集以及用于增强的生成模型中，相对于几种最近的合成数据选择方法表现良好。 显示英文摘要

摘要： Despite the progress in the development of generative models, their usefulness in creating synthetic data that improve prediction performance of classifiers has been put into question. Besides heuristic principles such as "synthetic data should be close to the real data distribution", it is actually not clear which specific properties affect the generalization error. Our paper addresses this question through the lens of high-dimensional regression. Theoretically, we show that, for linear models, the covariance shift between the target distribution and the distribution of the synthetic data affects the generalization error but, surprisingly, the mean shift does not. Furthermore we prove that, in some settings, matching the covariance of the target distribution is optimal. Remarkably, the theoretical insights from linear models carry over to deep neural networks and generative models. We empirically demonstrate that the covariance matching procedure (matching the covariance of the synthetic data with that of the data coming from the target distribution) performs well against several recent approaches for synthetic data selection, across training paradigms, architectures, datasets and generative models used for augmentation.