Smoothed Histograms for Frequency Data on Irregular Intervals

• Published in: The American statistician 2008-08, Vol.62 (3), p.256-261
• Main Authors: Scott, David W, Scott, Warren R
• Format: Article
• Language: English
• Subjects: Approximation; Computational statistics; Density; Density approximation; Density estimation; Estimators; Exact sciences and technology; Frequency distribution; General topics; Histograms; Household income; Interval arithmetic; Mathematics; Maximum likelihood estimation; Musical intervals; Penalized roughness criterion; Penalty function; Probability and statistics; Sample size; Sciences and techniques of general use; Statistical analysis; Statistical Computing and Graphics; Statistics; Unequal bin intervals
• ISSN: 0003-1305; 1537-2731
• DOI: 10.1198/000313008X335581

Frequency tables are often constructed on intervals of irregular width. When plotted as bar charts, the underlying true density information may be quite distorted. The majority of introductory statistics texts recommend tabulating data into intervals of equal width, but seldom caution the consequences of failing to do so. An occasional introductory text correctly emphasizes that area rather than frequency should be plotted. Nevertheless, the correctly scaled density figure is often visually less informative than one might expect, with wide bins at constant height. In many cases, the right most bin interval has no well-defined end point, making its depiction some what arbitrary. In this note, we introduce a regular histogram approximation that matches the frequencies and also minimizes a roughness criterion for visual and exploratory appeal. The resulting estimate can reveal the density structure much more clearly. We also formulate an alternative criterion that explicitly takes account of the uncertainty in the bin frequencies.