标题： 离散分布相对熵的估计及缺失质量偏差 显示英文标题

标题： Estimation of discrete distributions in relative entropy, and the deviations of the missing mass

摘要： 我们研究了从独立同分布样本中估计有限字母表上分布的问题，精度通过相对熵（Kullback-Leibler散度）来衡量。虽然预期风险的最优界已经知道，但概率较高的保证仍然理解得不够充分。首先，我们分析了经典的拉普拉斯（加-$1$）估计器，得到了关于其性能的匹配的上下界，并证明了它在无信心独立估计器中的最优性。然后，我们刻画了任何估计器能达到的最小最大概率较高风险，这可以通过简单的置信依赖平滑技术实现。有趣的是，非渐近最优风险比理想的渐近风险多了一个对数因子。接下来，受字母表超过样本量场景的启发，我们研究了适应于手头分布稀疏性的方法。我们引入了一种使用数据依赖平滑的估计器，并建立了与两个有效稀疏参数相关的概率较高风险界。作为分析的一部分，我们还推导出了缺失质量的概率较高上界。 显示英文摘要

摘要： We study the problem of estimating a distribution over a finite alphabet from an i.i.d. sample, with accuracy measured in relative entropy (Kullback-Leibler divergence). While optimal expected risk bounds are known, high-probability guarantees remain less well-understood. First, we analyze the classical Laplace (add-$1$) estimator, obtaining matching upper and lower bounds on its performance and showing its optimality among confidence-independent estimators. We then characterize the minimax-optimal high-probability risk achievable by any estimator, which is attained via a simple confidence-dependent smoothing technique. Interestingly, the optimal non-asymptotic risk contains an additional logarithmic factor over the ideal asymptotic risk. Next, motivated by scenarios where the alphabet exceeds the sample size, we investigate methods that adapt to the sparsity of the distribution at hand. We introduce an estimator using data-dependent smoothing, for which we establish a high-probability risk bound depending on two effective sparsity parameters. As part of the analysis, we also derive a sharp high-probability upper bound on the missing mass.