标题： 对数几率模型中切线变换算法的统计最优性和稳定性 显示英文标题

标题： Statistical optimality and stability of tangent transform algorithms in logit models

摘要： 一种在难以处理的非共轭模型中寻找变分近似的系统方法是利用凸对偶性的普遍原理，通过最小化边缘似然使问题变得可处理。 尽管这类方法在非共轭贝叶斯模型的变分推理背景下很受欢迎，但关于统计最优性和算法收敛性的理论保证却缺乏。 聚焦于逻辑回归模型，我们提供了数据生成过程的一些温和条件，以推导出变分最优解所导致的风险的非渐近上界。 我们证明，如果考虑将似然提升到分数幂的算法轻微变化，这些假设可以完全放松。 接下来，我们利用动力系统理论为逻辑回归和多项逻辑回归中的此类算法提供收敛保证。 特别是，我们在没有任何数据生成过程假设的情况下建立了算法的局部渐近稳定性。 我们探讨了一个涉及半正交设计的特殊情况，在此情况下获得了全局收敛性。 该理论通过几个数值研究进一步得到说明。 显示英文摘要

摘要： A systematic approach to finding variational approximation in an otherwise intractable non-conjugate model is to exploit the general principle of convex duality by minorizing the marginal likelihood that renders the problem tractable. While such approaches are popular in the context of variational inference in non-conjugate Bayesian models, theoretical guarantees on statistical optimality and algorithmic convergence are lacking. Focusing on logistic regression models, we provide mild conditions on the data generating process to derive non-asymptotic upper bounds to the risk incurred by the variational optima. We demonstrate that these assumptions can be completely relaxed if one considers a slight variation of the algorithm by raising the likelihood to a fractional power. Next, we utilize the theory of dynamical systems to provide convergence guarantees for such algorithms in logistic and multinomial logit regression. In particular, we establish local asymptotic stability of the algorithm without any assumptions on the data-generating process. We explore a special case involving a semi-orthogonal design under which a global convergence is obtained. The theory is further illustrated using several numerical studies.