An Empirical Study of Greedy Kernel Fisher Discriminants

A sparse version of Kernel Fisher Discriminant Analysis using an approach based on Matching Pursuit (MPKFDA) has been shown to be competitive with Kernel Fisher Discriminant Analysis and the Support Vector Machines on publicly available datasets, with additional experiments showing that MPKFDA on av...

Full description

Saved in:
Bibliographic Details
Published in:Mathematical problems in engineering 2015-01, Vol.2015 (2015), p.1-12
Main Author: Diethe, Tom
Format: Article
Language:English
Subjects:
Algorithms
Classification
Criteria
Datasets
Dimensional analysis
Discriminant analysis
Empirical analysis
Engineering
Greedy algorithms
Hilbert space
Kernels
Lagrange multiplier
Matched pursuit
Mathematical models
Optimization
Performance degradation
Probability
Support vector machines
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:A sparse version of Kernel Fisher Discriminant Analysis using an approach based on Matching Pursuit (MPKFDA) has been shown to be competitive with Kernel Fisher Discriminant Analysis and the Support Vector Machines on publicly available datasets, with additional experiments showing that MPKFDA on average outperforms these algorithms in extremely high dimensional settings. In (nearly) all cases, the resulting classifier was sparser than the Support Vector Machine. Natural questions that arise are what is the relative importance of the use of the Fisher criterion for selecting bases and the deflation step? Can we speed the algorithm up without degrading performance? Here we analyse the algorithm in more detail, providing alternatives to the optimisation criterion and the deflation procedure of the algorithm, and also propose a stagewise version. We demonstrate empirically that these alternatives can provide considerable improvements in the computational complexity, whilst maintaining the performance of the original algorithm (and in some cases improving it).
ISSN:1024-123X
1563-5147
DOI:10.1155/2015/793986