Domain Adaptation for Statistical Classifiers

The most basic assumption used in statistical learning theory is that training data and test data are drawn from the same underlying distribution. Unfortunately, in many applications, the "in-domain" test data is drawn from a distribution that is related, but not identical, to the "ou...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of artificial intelligence research 2006-01, Vol.26, p.101-126
Main Authors: Daume III, H., Marcu, D.
Format: Article
Language:English
Subjects:
Algorithms
Artificial intelligence
Classifiers
Domains
Learning theory
Maximum entropy
Natural language processing
Probabilistic models
Training
Citations: Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The most basic assumption used in statistical learning theory is that training data and test data are drawn from the same underlying distribution. Unfortunately, in many applications, the "in-domain" test data is drawn from a distribution that is related, but not identical, to the "out-of-domain" distribution of the training data. We consider the common case in which labeled out-of-domain data is plentiful, but labeled in-domain data is scarce. We introduce a statistical formulation of this problem in terms of a simple mixture model and present an instantiation of this framework to maximum entropy classifiers and their linear chain counterparts. We present efficient inference algorithms for this special case based on the technique of conditional expectation maximization. Our experimental results show that our approach leads to improved performance on three real world tasks on four different data sets from the natural language processing domain.
ISSN:1076-9757
1076-9757
1943-5037
DOI:10.1613/jair.1872