SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images

In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and multitask learning-based fusion network which jointl...

Full description

Saved in:
Bibliographic Details
Main Authors: Khargonkar, Tuneer, Choudhary, Shwetank, Kumar, Sumit, KR, Barath Raj
Format: Conference Proceeding
Language:English
Subjects:
Circuits and systems
Emotion recognition
Feature extraction
Object detection
Pipelines
Predictive models
Semantics
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and multitask learning-based fusion network which jointly estimates discrete emotion and human sentiments tasks. On the EMOTIC dataset, the proposed approach achieves an Average Precision (AP) score of 27.17 in comparison to the baseline AP score of 27.38 while reducing the model size by >85%. In addition, we report an on-device AP score of 26.42 with reduction in model size by >93% when compared to the baseline.
ISSN:2158-1525
DOI:10.1109/ISCAS46773.2023.10181844