Similarity Function Tracking Using Pairwise Comparisons

• Published in: IEEE transactions on signal processing 2017-11, Vol.65 (21), p.5635-5648
• Main Authors: Greenewald, Kristjan, Kelley, Stephen, Oselio, Brandon, Hero, Alfred O.
• Format: Article
• Language: English
• Subjects: clustering; Clustering algorithms; Distance metric learning; dynamic regret; Heuristic algorithms; Machine learning algorithms; Measurement; online learning; Principal component analysis; Robustness; semisupervised learning; Signal processing algorithms
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2017.2739100

Recent work in distance metric learning has focused on learning transformations of data that best align with specified pairwise similarity and dissimilarity constraints, often supplied by a human observer. The learned transformations lead to improved retrieval, classification, and clustering algorithms due to the better adapted distance or similarity measures. Here, we address the problem of learning these transformations when the underlying constraint generation process is nonstationary. This nonstationarity can, for example, be due to changes in the feature subspaces in which class dichotomies are apparent. We propose online convex ensemble strongly adaptive dynamic learning (OCELAD), a general adaptive, online approach for learning and tracking optimal metrics as they change over time that is highly robust to a variety of nonstationary behaviors in the changing metric. We apply the OCELAD framework to an ensemble of online learners. Specifically, we create a retro-initialized composite objective mirror descent (COMID) ensemble (RICE) consisting of a set of parallel COMID learners with different learning rates, and demonstrate parameter-free RICE-OCELAD metric learning on both synthetic data and a highly nonstationary Twitter dataset. We show significant performance improvements and increased robustness to nonstationarities relative to previously proposed batch and online distance metric learning algorithms.