An Intrinsic Dimensionality Estimator from Near-Neighbor Information

The intrinsic dimensionality of a set of patterns is important in determining an appropriate number of features for representing the data and whether a reasonable two- or three-dimensional representation of the data exists. We propose an intuitively appealing, noniterative estimator for intrinsic di...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 1979-01, Vol.PAMI-1 (1), p.25-37
Main Authors: Pettis, Karl W., Bailey, Thomas A., Jain, Anil K., Dubes, Richard C.
Format: Article
Language:English
Subjects:
Algorithm design and analysis
Computer science
Covariance matrix
Data mining
Eigenvalues
Eigenvalues and eigenfunctions
interpoint distances
intrinsic dimensionality
Multidimensional systems
near-neighbor information
outliers
Pattern recognition
State estimation
Stress
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The intrinsic dimensionality of a set of patterns is important in determining an appropriate number of features for representing the data and whether a reasonable two- or three-dimensional representation of the data exists. We propose an intuitively appealing, noniterative estimator for intrinsic dimensionality which is based on nearneighbor information. We give plausible arguments supporting the consistency of this estimator. The method works well in identifying the true dimensionality for a variety of artificial data sets and is fairly insensitive to the number of samples and to the algorithmic parameters. Comparisons between this new method and the global eigenvalue approach demonstrate the utility of our estimator.
ISSN:0162-8828
1939-3539
DOI:10.1109/TPAMI.1979.4766873