Title: Dynamical mean-field analysis of adaptive Langevin diffusions: Replica-symmetric fixed point and empirical Bayes Show Chinese title

Title: 自适应Langevin扩散的动力学平均场分析：副本对称不动点和经验贝叶斯

Abstract: In many applications of statistical estimation via sampling, one may wish to sample from a high-dimensional target distribution that is adaptively evolving to the samples already seen. We study an example of such dynamics, given by a Langevin diffusion for posterior sampling in a Bayesian linear regression model with i.i.d. regression design, whose prior continuously adapts to the Langevin trajectory via a maximum marginal-likelihood scheme. Results of dynamical mean-field theory (DMFT) developed in our companion paper establish a precise high-dimensional asymptotic limit for the joint evolution of the prior parameter and law of the Langevin sample. In this work, we carry out an analysis of the equations that describe this DMFT limit, under conditions of approximate time-translation-invariance which include, in particular, settings where the posterior law satisfies a log-Sobolev inequality. In such settings, we show that this adaptive Langevin trajectory converges on a dimension-independent time horizon to an equilibrium state that is characterized by a system of scalar fixed-point equations, and the associated prior parameter converges to a critical point of a replica-symmetric limit for the model free energy. As a by-product of our analyses, we obtain a new dynamical proof that this replica-symmetric limit for the free energy is exact, in models having a possibly misspecified prior and where a log-Sobolev inequality holds for the posterior law. Show Chinese abstract

Abstract: 在统计估计的许多采样应用中，人们可能希望从一个高维的目标分布采样，该目标分布会根据已经观察到的样本自适应地演化。 我们研究了这种动态的一个例子，由贝叶斯线性回归模型中的后验采样给出的朗之万扩散组成，该模型具有独立同分布的回归设计，并且其先验通过最大边缘似然方案连续适应朗之万轨迹。 我们在另一篇论文中开发的动力学平均场理论（DMFT）结果建立了先验参数和朗之万样本律联合演化的精确高维渐近极限。 在这项工作中，我们分析了描述这一DMFT极限的方程，在近似的时移不变条件下，其中包括后验律满足对数Sobolev不等式的设置。 在这种情况下，我们证明了这种自适应的朗之万轨迹在与维度无关的时间范围内收敛到一个平衡态，该平衡态由一组标量不动点方程表征，并且相关的先验参数收敛到模型自由能的复制对称极限的一个临界点。 作为我们分析的结果，我们在具有可能错误指定的先验并且后验律满足对数Sobolev不等式的模型中得到了自由能的复制对称极限精确的新动力学证明。