标题： 为什么启发式加权有效：去噪得分匹配的理论分析 显示英文标题

标题： Why Heuristic Weighting Works: A Theoretical Analysis of Denoising Score Matching

摘要： 得分匹配使得能够估计数据分布的梯度，这是去噪扩散模型中的关键组成部分，用于从损坏的输入中恢复干净的数据。 在以前的工作中，使用了一种启发式的加权函数来处理去噪得分匹配损失，但没有正式的证明。 在这项工作中，我们证明了异方差性是去噪得分匹配目标的固有属性。 这一见解导致了广义的、任意阶的去噪得分匹配损失的最优加权函数的合理推导，而无需对噪声分布做出假设。 其中，一阶公式对于扩散模型尤其相关。 我们表明，广泛使用的启发式加权函数是期望最优加权迹的的一阶泰勒近似。 我们进一步提供了理论和实证比较，揭示了尽管启发式加权函数简单，但相对于参数梯度，它可以实现比最优加权更低的方差，这可以促进更稳定和高效的训练。 显示英文摘要

摘要： Score matching enables the estimation of the gradient of a data distribution, a key component in denoising diffusion models used to recover clean data from corrupted inputs. In prior work, a heuristic weighting function has been used for the denoising score matching loss without formal justification. In this work, we demonstrate that heteroskedasticity is an inherent property of the denoising score matching objective. This insight leads to a principled derivation of optimal weighting functions for generalized, arbitrary-order denoising score matching losses, without requiring assumptions about the noise distribution. Among these, the first-order formulation is especially relevant to diffusion models. We show that the widely used heuristical weighting function arises as a first-order Taylor approximation to the trace of the expected optimal weighting. We further provide theoretical and empirical comparisons, revealing that the heuristical weighting, despite its simplicity, can achieve lower variance than the optimal weighting with respect to parameter gradients, which can facilitate more stable and efficient training.