标题： 常微分方程$_t$(常微分方程$_l$)：在扩散和流模型中缩短时间和长度以实现更快的采样 显示英文标题

标题： ODE$_t$(ODE$_l$): Shortcutting the Time and Length in Diffusion and Flow Models for Faster Sampling

摘要： 最近，连续归一化流（CNFs）和扩散模型（DMs）已被纳入统一的理论框架中进行研究。 尽管这些模型可以从噪声分布生成高质量的数据点，但采样需要多次迭代以解决具有高计算复杂度的常微分方程（ODE）。 大多数现有方法专注于减少采样过程中的时间步数以提高效率。 在本工作中，我们探索了一种补充方向，其中质量-复杂度权衡可以在时间步数和神经网络长度方面动态控制。 我们通过重新连接基于Transformer的架构中的块来求解关于其长度的内部离散化ODE来实现这一点。 然后，在流匹配训练过程中，我们采用时间和长度一致性的项，结果是采样可以使用任意数量的时间步和Transformer块。 与其他方法不同，我们的 ODE$_t$(ODE$_l$) 方法在时间维度上与求解器无关，并减少了延迟和内存使用。 与之前的最先进方法相比，CelebA-HQ和ImageNet上的图像生成实验显示，在最有效的采样模式下延迟减少了多达3$\times$，在高质量采样中FID分数提高了多达3.5分。 我们发布了代码和模型权重，并进行了完全可复现的实验。 显示英文摘要

摘要： Recently, continuous normalizing flows (CNFs) and diffusion models (DMs) have been studied using the unified theoretical framework. Although such models can generate high-quality data points from a noise distribution, the sampling demands multiple iterations to solve an ordinary differential equation (ODE) with high computational complexity. Most existing methods focus on reducing the number of time steps during the sampling process to improve efficiency. In this work, we explore a complementary direction in which the quality-complexity tradeoff can be dynamically controlled in terms of time steps and in the length of the neural network. We achieve this by rewiring the blocks in the transformer-based architecture to solve an inner discretized ODE w.r.t. its length. Then, we employ time- and length-wise consistency terms during flow matching training, and as a result, the sampling can be performed with an arbitrary number of time steps and transformer blocks. Unlike others, our ODE$_t$(ODE$_l$) approach is solver-agnostic in time dimension and decreases both latency and memory usage. Compared to the previous state of the art, image generation experiments on CelebA-HQ and ImageNet show a latency reduction of up to 3$\times$ in the most efficient sampling mode, and a FID score improvement of up to 3.5 points for high-quality sampling. We release our code and model weights with fully reproducible experiments.