Title: Faster Diffusion Models via Higher-Order Approximation Show Chinese title

Title: 通过高阶近似加速扩散模型

Abstract: In this paper, we explore provable acceleration of diffusion models without any additional retraining. Focusing on the task of approximating a target data distribution in $\mathbb{R}^d$ to within $\varepsilon$ total-variation distance, we propose a principled, training-free sampling algorithm that requires only the order of $$ d^{1+2/K} \varepsilon^{-1/K} $$ score function evaluations (up to log factor) in the presence of accurate scores, where $K>0$ is an arbitrary fixed integer. This result applies to a broad class of target data distributions, without the need for assumptions such as smoothness or log-concavity. Our theory is robust vis-a-vis inexact score estimation, degrading gracefully as the score estimation error increases -- without demanding higher-order smoothness on the score estimates as assumed in previous work. The proposed algorithm draws insight from high-order ODE solvers, leveraging high-order Lagrange interpolation and successive refinement to approximate the integral derived from the probability flow ODE. More broadly, our work develops a theoretical framework towards understanding the efficacy of high-order methods for accelerated sampling. Show Chinese abstract

Abstract: 在本文中，我们探索了在无需任何额外训练的情况下对扩散模型进行可证明的加速。 专注于在$\mathbb{R}^d$中将目标数据分布近似到$\varepsilon$的总变分距离以内，我们提出了一种有原则的、无需训练的采样算法，该算法在存在精确分数函数的情况下，仅需要$$ d^{1+2/K} \varepsilon^{-1/K} $$阶的分数函数评估次数（忽略对数因子），其中$K>0$是一个任意的固定整数。 这一结果适用于广泛的目标数据分布，而无需诸如平滑性或对数凹性等假设。 我们的理论对于不精确的分数估计具有鲁棒性，随着分数估计误差的增加，性能会逐渐下降——而无需像以前的工作那样要求分数估计的高阶平滑性。 所提出的算法从高阶常微分方程求解器中获得启发，利用高阶拉格朗日插值和逐步细化来近似从概率流常微分方程中导出的积分。 更广泛地说，我们的工作建立了一个理论框架，以理解高阶方法在加速采样中的有效性。