Geometry preserving multi-task metric learning

• Published in: Machine learning 2013-07, Vol.92 (1), p.133-175
• Main Authors: Yang, Peipei, Huang, Kaizhu, Liu, Cheng-Lin
• Format: Article
• Language: English
• Subjects: Artificial Intelligence; Computer Science; Control; Divergence; Geometry; Learning; Mathematical analysis; Mechatronics; Natural Language Processing (NLP); Optimization; Preserves; Preserving; Regularization; Robotics; Simulation and Modeling; Tasks
• ISSN: 0885-6125; 1573-0565
• DOI: 10.1007/s10994-013-5379-y

In this paper, we consider the multi-task metric learning problem, i.e., the problem of learning multiple metrics from several correlated tasks simultaneously. Despite the importance, there are only a limited number of approaches in this field. While the existing methods often straightforwardly extend existing vector-based methods, we propose to couple multiple related metric learning tasks with the von Neumann divergence. On one hand, the novel regularized approach extends previous methods from the vector regularization to a general matrix regularization framework; on the other hand and more importantly, by exploiting von Neumann divergence as the regularization, the new multi-task metric learning method has the capability to well preserve the data geometry. This leads to more appropriate propagation of side-information among tasks and provides potential for further improving the performance. We propose the concept of geometry preserving probability and show that our framework encourages a higher geometry preserving probability in theory. In addition, our formulation proves to be jointly convex and the global optimal solution can be guaranteed. We have conducted extensive experiments on six data sets (across very different disciplines), and the results verify that our proposed approach can consistently outperform almost all the current methods.