Using continuous features in the maximum entropy model

We investigate the problem of using continuous features in the maximum entropy (MaxEnt) model. We explain why the MaxEnt model with the moment constraint (MaxEnt-MC) works well with binary features but not with the continuous features. We describe how to enhance constraints on the continuous feature...

Full description

Saved in:
Bibliographic Details
Published in:Pattern recognition letters 2009-10, Vol.30 (14), p.1295-1300
Main Authors: Yu, Dong, Deng, Li, Acero, Alex
Format: Article
Language:English
Subjects:
Applied sciences
Continuous feature
Distribution constraint
Exact sciences and technology
Information, signal and communications theory
Maximum entropy model
Maximum entropy principle
Miscellaneous
Moment constraint
Signal processing
Spline interpolation
Telecommunications and information theory
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We investigate the problem of using continuous features in the maximum entropy (MaxEnt) model. We explain why the MaxEnt model with the moment constraint (MaxEnt-MC) works well with binary features but not with the continuous features. We describe how to enhance constraints on the continuous features and show that the weights associated with the continuous features should be continuous functions instead of single values. We propose a spline-based solution to the MaxEnt model with non-linear continuous weighting functions and illustrate that the optimization problem can be converted into a standard log-linear model at a higher-dimensional space. The empirical results on two classification tasks that contain continuous features are reported. The results confirm our insight and show that our proposed solution consistently outperforms the MaxEnt-MC model and the bucketing approach with significant margins.
ISSN:0167-8655
1872-7344
DOI:10.1016/j.patrec.2009.06.005