Title: Particle Semi-Implicit Variational Inference Show Chinese title

Title: 粒子半隐式变分推理

Abstract: Semi-implicit variational inference (SIVI) enriches the expressiveness of variational families by utilizing a kernel and a mixing distribution to hierarchically define the variational distribution. Existing SIVI methods parameterize the mixing distribution using implicit distributions, leading to intractable variational densities. As a result, directly maximizing the evidence lower bound (ELBO) is not possible, so they resort to one of the following: optimizing bounds on the ELBO, employing costly inner-loop Markov chain Monte Carlo runs, or solving minimax objectives. In this paper, we propose a novel method for SIVI called Particle Variational Inference (PVI) which employs empirical measures to approximate the optimal mixing distributions characterized as the minimizer of a free energy functional. PVI arises naturally as a particle approximation of a Euclidean--Wasserstein gradient flow and, unlike prior works, it directly optimizes the ELBO whilst making no parametric assumption about the mixing distribution. Our empirical results demonstrate that PVI performs favourably compared to other SIVI methods across various tasks. Moreover, we provide a theoretical analysis of the behaviour of the gradient flow of a related free energy functional: establishing the existence and uniqueness of solutions as well as propagation of chaos results. Show Chinese abstract

Abstract: 半隐式变分推断（SIVI）通过利用核函数和混合分布来分级定义变分分布，从而增强了变分族的表达能力。现有的 SIVI 方法使用隐式分布来参数化混合分布，导致变分密度难以计算。因此，直接最大化证据下界（ELBO）是不可能的，于是它们转而采用以下方法之一：优化 ELBO 的边界、使用昂贵的内循环马尔可夫链蒙特卡洛运行，或者解决极小极大目标。本文提出了一种新的 SIVI 方法，称为粒子变分推断（PVI），它使用经验测度来近似由自由能泛函最小值刻画的最佳混合分布。PVI 自然地作为欧几里得–Wasserstein 梯度流的粒子近似出现，并且与先前的工作不同，它直接优化 ELBO，同时不对混合分布做出参数假设。我们的实证结果显示，与其他 SIVI 方法相比，PVI 在各种任务中的表现更优。此外，我们还对该相关自由能泛函梯度流的行为进行了理论分析：证明了解的存在性和唯一性以及混沌传播的结果。