IEEE Transactions on Information Theory vol:47 issue:5 pages:1867-1883
We investigate a nonparametric estimator of probability density introduced by A. R, Barren. Earlier papers established its consistency in a strong sense, e.g,, in the expected information divergence or expected chi-square divergence. This paper pays main attention to the expected chi-square divergence criterion. We give a new motivation of the Barren estimator by showing that a maximum-likelihood estimator (MLE) of a density from a family important in practice is consistent in expected information divergence but not in expected chi-square divergence. We also present new and practically applicable conditions of consistency in the expected chi-square divergence. Main attention is paid to optimization tin the sense of the mentioned criterion) of the two objects specifying the Barren estimator: the dominating probability density and the decomposition of the observation space into finitely many bins. Both problems are explicitly solved under certain regularity assumptions about the estimated density. A simulation study illustrates the results in exponential, Rayleigh, and Weibull families.