Multimodal Representation Learning With Text and Images

In recent years, multimodal AI has seen an upward trend as researchers are integrating data of different types such as text, images, speech into modelling to get the best results. This project leverages multimodal AI and matrix factorization techniques for representation learning, on text and image...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2022-04
Main Authors: Jayagopal, Aishwarya, Ankireddy, Monica Aiswarya, Garg, Ankita, Nandakumar, Srinivasan Kolumam
Format: Article
Language:English
Subjects:
Coders
Computer vision
Learning
Natural language processing
Representations
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In recent years, multimodal AI has seen an upward trend as researchers are integrating data of different types such as text, images, speech into modelling to get the best results. This project leverages multimodal AI and matrix factorization techniques for representation learning, on text and image data simultaneously, thereby employing the widely used techniques of Natural Language Processing (NLP) and Computer Vision. The learnt representations are evaluated using downstream classification and regression tasks. The methodology adopted can be extended beyond the scope of this project as it uses Auto-Encoders for unsupervised representation learning.
ISSN:2331-8422