umu.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Statistical estimation of quadratic Rényi entropy for a stationary m-dependent sequence
Umeå University, Faculty of Science and Technology, Department of Mathematics and Mathematical Statistics.
Cardiff University, School of Mathematics.
Umeå University, Faculty of Science and Technology, Department of Mathematics and Mathematical Statistics.
2014 (English)In: Journal of nonparametric statistics (Print), ISSN 1048-5252, E-ISSN 1029-0311, Vol. 26, no 2, 385-411 p.Article in journal (Refereed) Published
Abstract [en]

The Rényi entropy is a generalization of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of ε-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g., consistency, asymptotic normality, Poisson convergence). The results can be used in diverse statistical and computer science problems whenever the conventional independence assumption is too strong (e.g., ε-keys in time series databases, distribution identication problems for dependent samples).

Place, publisher, year, edition, pages
Taylor & Francis, 2014. Vol. 26, no 2, 385-411 p.
Keyword [en]
entropy estimation, quadratic Rényi entropy, stationary m-dependent sequence, U-statistics, inter-point distances
National Category
Probability Theory and Statistics
Research subject
Mathematical Statistics
Identifiers
URN: urn:nbn:se:umu:diva-79958DOI: 10.1080/10485252.2013.854438ISI: 000334160600011OAI: oai:DiVA.org:umu-79958DiVA: diva2:645449
Note

Included in thesis 2013 in submitted form.

Available from: 2013-09-04 Created: 2013-09-04 Last updated: 2017-12-06Bibliographically approved
In thesis
1. Nonparametric Statistical Inference for Entropy-type Functionals
Open this publication in new window or tab >>Nonparametric Statistical Inference for Entropy-type Functionals
2013 (English)Doctoral thesis, comprehensive summary (Other academic)
Alternative title[sv]
Icke-parametrisk statistisk inferens för entropirelaterade funktionaler
Abstract [en]

In this thesis, we study statistical inference for entropy, divergence, and related functionals of one or two probability distributions. Asymptotic properties of particular nonparametric estimators of such functionals are investigated. We consider estimation from both independent and dependent observations. The thesis consists of an introductory survey of the subject and some related theory and four papers (A-D).

In Paper A, we consider a general class of entropy-type functionals which includes, for example, integer order Rényi entropy and certain Bregman divergences. We propose U-statistic estimators of these functionals based on the coincident or epsilon-close vector observations in the corresponding independent and identically distributed samples. We prove some asymptotic properties of the estimators such as consistency and asymptotic normality. Applications of the obtained results related to entropy maximizing distributions, stochastic databases, and image matching are discussed.

In Paper B, we provide some important generalizations of the results for continuous distributions in Paper A. The consistency of the estimators is obtained under weaker density assumptions. Moreover, we introduce a class of functionals of quadratic order, including both entropy and divergence, and prove normal limit results for the corresponding estimators which are valid even for densities of low smoothness. The asymptotic properties of a divergence-based two-sample test are also derived.

In Paper C, we consider estimation of the quadratic Rényi entropy and some related functionals for the marginal distribution of a stationary m-dependent sequence. We investigate asymptotic properties of the U-statistic estimators for these functionals introduced in Papers A and B when they are based on a sample from such a sequence. We prove consistency, asymptotic normality, and Poisson convergence under mild assumptions for the stationary m-dependent sequence. Applications of the results to time-series databases and entropy-based testing for dependent samples are discussed.

In Paper D, we further develop the approach for estimation of quadratic functionals with m-dependent observations introduced in Paper C. We consider quadratic functionals for one or two distributions. The consistency and rate of convergence of the corresponding U-statistic estimators are obtained under weak conditions on the stationary m-dependent sequences. Additionally, we propose estimators based on incomplete U-statistics and show their consistency properties under more general assumptions.

Place, publisher, year, edition, pages
Umeå: Umeå universitet, 2013. 21 p.
Keyword
entropy estimation, Rényi entropy, divergence estimation, quadratic density functional, U-statistics, consistency, asymptotic normality, Poisson convergence, stationary m-dependent sequence, inter-point distances, entropy maximizing distribution, two-sample problem, approximate matching
National Category
Probability Theory and Statistics
Research subject
Mathematical Statistics
Identifiers
urn:nbn:se:umu:diva-79976 (URN)978-91-7459-701-1 (ISBN)
Public defence
2013-09-27, MIT-huset, MA121, Umeå universitet, Umeå, 10:00 (English)
Opponent
Supervisors
Available from: 2013-09-06 Created: 2013-09-04 Last updated: 2013-09-05Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textURL

Search in DiVA

By author/editor
Källberg, DavidSeleznjev, Oleg
By organisation
Department of Mathematics and Mathematical Statistics
In the same journal
Journal of nonparametric statistics (Print)
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 123 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf