Multi-Modal Learning With Generalizable Nonlinear Dimensionality Reduction

In practical machine learning settings, there often exist relations or links between data from different modalities. The goal of multi-modal learning algorithms is to efficiently use the information available in different modalities to solve multi-modal classification or retrieval problems. In this...

Full description

Saved in:
Bibliographic Details
Main Authors: Kaya, Semih, Vural, Elif
Format: Conference Proceeding
Language:English
Subjects:
Cross-modal learning
cross-modal retrieval
Dimensionality reduction
Interpolation
Kernel
Laplace equations
Learning systems
multi-view learning
nonlinear embeddings
Optimization
RBF interpolators
Training
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In practical machine learning settings, there often exist relations or links between data from different modalities. The goal of multi-modal learning algorithms is to efficiently use the information available in different modalities to solve multi-modal classification or retrieval problems. In this study, we propose a multi-modal supervised representation learning algorithm based on nonlinear dimensionality reduction. Nonlinear embeddings often yield more flexible representations compared to linear counterparts especially in case of high dissimilarity between the data geometries in different modalities. Based on recent performance bounds on nonlinear dimensionality reduction, we propose an optimization objective aiming to improve the intra- and inter-modal within-class compactness and between-class separation, as well as the Lipschitz regularity of the interpolator that generalizes the embedding to the whole data space. Experiments in multi-view face recognition and image-text retrieval applications show that the proposed method yields promising performance in comparison with state-of-the-art multi-modal learning methods.
ISSN:2381-8549
DOI:10.1109/ICIP.2019.8803196