On the convergence of Shannon differential entropy, and its connections with density and entropy estimation

详细信息    查看全文

• 作者：Jorge F. Silva ; ^{josilva@ing.uchile.cl} ; ^{jorgesil.edu@gmail.com} ; ; Patricio Parada ^{pparada@ing.uchile.cl}
• 关键词：Convergence of probability measures ; Shannon information measures ; Strong consistency ; Density estimation ; Differential entropy estimation ; Consistency in information divergence ; Histogram-based estimators
• 刊名：Journal of Statistical Planning and Inference
• 出版年：2012
• 出版时间：July, 2012
• 年：2012
• 卷：142
• 期：7
• 页码：1716-1732
• 全文大小：321 K

文摘

This work extends the study of convergence properties of the Shannon differential entropy, and its connections with the convergence of probability measures in the sense of total variation and direct and reverse information divergence. The results relate the topics of distribution (density) estimation, and Shannon information measures estimation, with special focus on the case of differential entropy. On the application side, this work presents an explicit analysis of the density estimation, and differential entropy estimation, for distributions defined on a finite-dimension Euclidean space . New consistency results are derived for several histogram-based estimators: the classical product scheme, the Barron's estimator, one of the approaches proposed by Gy?rfi and Van der Meulen, and the data-driven partition scheme of Lugosi and Nobel.