Domain Adaption via Feature Selection on Explicit Feature Map

• Published in: IEEE transaction on neural networks and learning systems 2019-04, Vol.30 (4), p.1180-1190
• Main Authors: Deng, Wan-Yu, Lendasse, Amaury, Ong, Yew-Soon, Tsang, Ivor Wai-Hung, Chen, Lin, Zheng, Qing-Hua
• Format: Article
• Language: English
• Subjects: Adaptation models; Algorithms; Benchmarks; Distribution distance; domain adaption; Domains; Feature extraction; Feature maps; feature selection; Invariants; Kernel; Learning systems; Optimization; Performance degradation; Taylor series; Telecommunications; transfer learning
• ISSN: 2162-237X; 2162-2388
• DOI: 10.1109/TNNLS.2018.2863240

In most domain adaption approaches, all features are used for domain adaption. However, often, not every feature is beneficial for domain adaption. In such cases, incorrectly involving all features might cause the performance to degrade. In other words, to make the model trained on the source domain work well on the target domain, it is desirable to find invariant features for domain adaption rather than using all features. However, invariant features across domains may lie in a higher order space, instead of in the original feature space. Moreover, the discriminative ability of some invariant features such as shared background information is weak, and needs to be further filtered. Therefore, in this paper, we propose a novel domain adaption algorithm based on an explicit feature map and feature selection. The data are first represented by a kernel-induced explicit feature map, such that high-order invariant features can be revealed. Then, by minimizing the marginal distribution difference, conditional distribution difference, and the model error, the invariant discriminative features are effectively selected. This problem is NP-hard to be solved, and we propose to relax it and solve it by a cutting plane algorithm. Experimental results on six real-world benchmarks have demonstrated the effectiveness and efficiency of the proposed algorithm, which outperforms many state-of-the-art domain adaption approaches.