Title: Mean-field variational inference with the TAP free energy: Geometric and statistical properties in linear models Show Chinese title

Title: 基于TAP自由能的平均场变分推断：线性模型中的几何与统计性质

Abstract: We study mean-field variational inference in a Bayesian linear model when the sample size n is comparable to the dimension p. In high dimensions, the common approach of minimizing a Kullback-Leibler divergence from the posterior distribution, or maximizing an evidence lower bound, may deviate from the true posterior mean and underestimate posterior uncertainty. We study instead minimization of the TAP free energy, showing in a high-dimensional asymptotic framework that it has a local minimizer which provides a consistent estimate of the posterior marginals and may be used for correctly calibrated posterior inference. Geometrically, we show that the landscape of the TAP free energy is strongly convex in an extensive neighborhood of this local minimizer, which under certain general conditions can be found by an Approximate Message Passing (AMP) algorithm. We then exhibit an efficient algorithm that linearly converges to the minimizer within this local neighborhood. In settings where it is conjectured that no efficient algorithm can find this local neighborhood, we prove analogous geometric properties for a local minimizer of the TAP free energy reachable by AMP, and show that posterior inference based on this minimizer remains correctly calibrated. Show Chinese abstract

Abstract: 我们研究了贝叶斯线性模型中的平均场变分推断，当样本量 \(n\) 与维度 \(p\) 可比较时。在高维情况下，通常通过最小化后验分布的 Kullback-Leibler 散度或最大化证据下界的方法可能会偏离真实的后验均值，并低估后验不确定性。 相反，我们研究了 TAP 自由能最小化问题，在高维渐近框架下证明它存在一个局部极小值点，该点能够一致估计后验边缘分布，并可用于正确校准的后验推理。 从几何角度看，我们证明了 TAP 自由能在该局部极小值点的一个大范围邻域内具有强凸性，且在某些一般条件下，可以通过近似消息传递（AMP）算法找到该极小值点。 然后，我们提出了一种高效算法，能够在该局部邻域内线性收敛到极小值点。 在被认为没有高效算法能够找到该局部邻域的情况下，我们证明了 AMP 可达的 TAP 自由能局部极小值点具有类似的几何性质，并表明基于此极小值点的后验推理仍然能够正确校准。