标题： 渐近正态性对于广义香农熵的插入估计量 显示英文标题

标题： Asymptotic Normality for Plug-in Estimators of Generalized Shannon's Entropy

摘要： 香农熵是信息论的基石之一，也是机器学习方法（例如随机森林）的重要方面。然而，它仅对于在可数字母表上具有快速衰减尾部的分布有限定义。香农熵在字母表上所有分布的一般类中的无界性阻碍了其潜在效用的充分发挥。为填补信息论基础中的这一空白，张（2020）提出了广义香农熵，该熵在任何地方都有限定义。插件估计器被几乎所有基于熵的机器学习方法包采用，是估计香农熵的最流行的方法之一。现有文献已很好地研究了香农熵插件估计器的渐近分布。本文研究了可数字母表上广义香农熵插件估计器的渐近性质。所发展的渐近性质对原始分布无需假设。所提出的渐近性质允许对广义香农熵进行区间估计和统计检验。 显示英文摘要

摘要： Shannon's entropy is one of the building blocks of information theory and an essential aspect of Machine Learning methods (e.g., Random Forests). Yet, it is only finitely defined for distributions with fast decaying tails on a countable alphabet. The unboundedness of Shannon's entropy over the general class of all distributions on an alphabet prevents its potential utility from being fully realized. To fill the void in the foundation of information theory, Zhang (2020) proposed generalized Shannon's entropy, which is finitely defined everywhere. The plug-in estimator, adopted in almost all entropy-based ML method packages, is one of the most popular approaches to estimating Shannon's entropy. The asymptotic distribution for Shannon's entropy's plug-in estimator was well studied in the existing literature. This paper studies the asymptotic properties for the plug-in estimator of generalized Shannon's entropy on countable alphabets. The developed asymptotic properties require no assumptions on the original distribution. The proposed asymptotic properties allow interval estimation and statistical tests with generalized Shannon's entropy.