Feature extraction using information-theoretic learning

A classification system typically consists of both a feature extractor (preprocessor) and a classifier. These two components can be trained either independently or simultaneously. The former option has an implementation advantage since the extractor need only be trained once for use with any classif...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2006-09, Vol.28 (9), p.1385-1392
Main Authors: Hild, K.E., Erdogmus, D., Torkkola, K., Principe, J.C.
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Artificial Intelligence
Classification
Classifiers
Computer science
control theory
systems
Criteria
Data mining
Degradation
Entropy
Errors
Exact sciences and technology
Feature extraction
Image Interpretation, Computer-Assisted - methods
Information Storage and Retrieval - methods
Information Theory
Minimization methods
Mutual information
nonparametric statistics
Pattern Recognition, Automated - methods
Pattern recognition. Digital image processing. Computational geometry
Performance evaluation
Statistics
Studies
Training
Trains
Visualization
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A classification system typically consists of both a feature extractor (preprocessor) and a classifier. These two components can be trained either independently or simultaneously. The former option has an implementation advantage since the extractor need only be trained once for use with any classifier, whereas the latter has an advantage since it can be used to minimize classification error directly. Certain criteria, such as minimum classification error, are better suited for simultaneous training, whereas other criteria, such as mutual information, are amenable for training the feature extractor either independently or simultaneously. Herein, an information-theoretic criterion is introduced and is evaluated for training the extractor independently of the classifier. The proposed method uses nonparametric estimation of Renyi's entropy to train the extractor by maximizing an approximation of the mutual information between the class labels and the output of the feature extractor. The evaluations show that the proposed method, even though it uses independent training, performs at least as well as three feature extraction methods that train the extractor and classifier simultaneously
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2006.186