Nonparametric entropy estimation for stationary processes and random fields, with applications to English texts

We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesaro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem d...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information theory 1998-05, Vol.44 (3), p.1319
Main Authors: Kontoyiannis, I, Algoet, P H, Suhov, Yu M, Wyner, A J
Format: Article
Language:English
Subjects:
Algorithms
Data compression
English language
Entropy
Information processing
Statistics
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We discuss a family of estimators for the entropy rate of a stationary ergodic process and prove their pointwise and mean consistency under a Doeblin-type mixing condition. The estimators are Cesaro averages of longest match-lengths, and their consistency follows from a generalized ergodic theorem due to Maker.
ISSN:0018-9448
1557-9654