SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images

In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and learning-based fusion network which jointly estimate...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-07
Main Authors: Khargonkar, Tuneer, Choudhary, Shwetank, Kumar, Sumit, Barath, Raj KR
Format: Article
Language:English
Subjects:
Emotion recognition
Emotions
Feature extraction
Lightweight
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and learning-based fusion network which jointly estimates discrete emotion and human sentiments tasks. On the EMOTIC dataset, the proposed approach achieves an Average Precision (AP) score of 27.17 in comparison to the baseline AP score of 27.38 while reducing the model size by >85%. In addition, we report an on-device AP score of 26.42 with reduction in model size by >93% when compared to the baseline.
ISSN:2331-8422