SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images

In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and learning-based fusion network which jointly estimate...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2023-07
Main Authors: Khargonkar, Tuneer, Choudhary, Shwetank, Kumar, Sumit, Barath, Raj KR
Format: Article
Language:English
Subjects:
Emotion recognition
Emotions
Feature extraction
Lightweight
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page
container_issue
container_start_page
container_title arXiv.org
container_volume
creator Khargonkar, Tuneer
Choudhary, Shwetank
Kumar, Sumit
Barath, Raj KR
description In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and learning-based fusion network which jointly estimates discrete emotion and human sentiments tasks. On the EMOTIC dataset, the proposed approach achieves an Average Precision (AP) score of 27.17 in comparison to the baseline AP score of 27.38 while reducing the model size by >85%. In addition, we report an on-device AP score of 26.42 with reduction in model size by >93% when compared to the baseline.
format article
fullrecord <record><control><sourceid>proquest</sourceid><recordid>TN_cdi_proquest_journals_2834345278</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2834345278</sourcerecordid><originalsourceid>FETCH-proquest_journals_28343452783</originalsourceid><addsrcrecordid>eNqNisEKgkAURYcgSMp_eNBasBlNaRtGgQSlexF76rOcqZkRfz-LPqDNuRfOmTGHC7Hx4oDzBXON6Xzf59uIh6Fw2CXDlM5od5ChtNRPAJSaqhZvkFLT2hE_hKkZlb5DrTQkvbKkJFyxUo2k7ycJp75s0KzYvC4fBt3fLtn6kOT7o_fU6jWgsUWnBi0nVfBYBCIIeRSL_6o3nkA_Uw</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2834345278</pqid></control><display><type>article</type><title>SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images</title><source>Publicly Available Content Database</source><creator>Khargonkar, Tuneer ; Choudhary, Shwetank ; Kumar, Sumit ; Barath, Raj KR</creator><creatorcontrib>Khargonkar, Tuneer ; Choudhary, Shwetank ; Kumar, Sumit ; Barath, Raj KR</creatorcontrib><description>In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and learning-based fusion network which jointly estimates discrete emotion and human sentiments tasks. On the EMOTIC dataset, the proposed approach achieves an Average Precision (AP) score of 27.17 in comparison to the baseline AP score of 27.38 while reducing the model size by &gt;85%. In addition, we report an on-device AP score of 26.42 with reduction in model size by &gt;93% when compared to the baseline.</description><identifier>EISSN: 2331-8422</identifier><language>eng</language><publisher>Ithaca: Cornell University Library, arXiv.org</publisher><subject>Emotion recognition ; Emotions ; Feature extraction ; Lightweight</subject><ispartof>arXiv.org, 2023-07</ispartof><rights>2023. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.proquest.com/docview/2834345278?pq-origsite=primo$$EHTML$$P50$$Gproquest$$Hfree_for_read</linktohtml><link.rule.ids>780,784,25751,37010,44588</link.rule.ids></links><search><creatorcontrib>Khargonkar, Tuneer</creatorcontrib><creatorcontrib>Choudhary, Shwetank</creatorcontrib><creatorcontrib>Kumar, Sumit</creatorcontrib><creatorcontrib>Barath, Raj KR</creatorcontrib><title>SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images</title><title>arXiv.org</title><description>In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and learning-based fusion network which jointly estimates discrete emotion and human sentiments tasks. On the EMOTIC dataset, the proposed approach achieves an Average Precision (AP) score of 27.17 in comparison to the baseline AP score of 27.38 while reducing the model size by &gt;85%. In addition, we report an on-device AP score of 26.42 with reduction in model size by &gt;93% when compared to the baseline.</description><subject>Emotion recognition</subject><subject>Emotions</subject><subject>Feature extraction</subject><subject>Lightweight</subject><issn>2331-8422</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2023</creationdate><recordtype>article</recordtype><sourceid>PIMPY</sourceid><recordid>eNqNisEKgkAURYcgSMp_eNBasBlNaRtGgQSlexF76rOcqZkRfz-LPqDNuRfOmTGHC7Hx4oDzBXON6Xzf59uIh6Fw2CXDlM5od5ChtNRPAJSaqhZvkFLT2hE_hKkZlb5DrTQkvbKkJFyxUo2k7ycJp75s0KzYvC4fBt3fLtn6kOT7o_fU6jWgsUWnBi0nVfBYBCIIeRSL_6o3nkA_Uw</recordid><startdate>20230706</startdate><enddate>20230706</enddate><creator>Khargonkar, Tuneer</creator><creator>Choudhary, Shwetank</creator><creator>Kumar, Sumit</creator><creator>Barath, Raj KR</creator><general>Cornell University Library, arXiv.org</general><scope>8FE</scope><scope>8FG</scope><scope>ABJCF</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>AZQEC</scope><scope>BENPR</scope><scope>BGLVJ</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>HCIFZ</scope><scope>L6V</scope><scope>M7S</scope><scope>PIMPY</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>PTHSS</scope></search><sort><creationdate>20230706</creationdate><title>SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images</title><author>Khargonkar, Tuneer ; Choudhary, Shwetank ; Kumar, Sumit ; Barath, Raj KR</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-proquest_journals_28343452783</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2023</creationdate><topic>Emotion recognition</topic><topic>Emotions</topic><topic>Feature extraction</topic><topic>Lightweight</topic><toplevel>online_resources</toplevel><creatorcontrib>Khargonkar, Tuneer</creatorcontrib><creatorcontrib>Choudhary, Shwetank</creatorcontrib><creatorcontrib>Kumar, Sumit</creatorcontrib><creatorcontrib>Barath, Raj KR</creatorcontrib><collection>ProQuest SciTech Collection</collection><collection>ProQuest Technology Collection</collection><collection>Materials Science &amp; Engineering Collection</collection><collection>ProQuest Central (Alumni)</collection><collection>ProQuest Central</collection><collection>ProQuest Central Essentials</collection><collection>ProQuest Central</collection><collection>Technology Collection</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>SciTech Premium Collection</collection><collection>ProQuest Engineering Collection</collection><collection>Engineering Database</collection><collection>Publicly Available Content Database</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>Engineering Collection</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Khargonkar, Tuneer</au><au>Choudhary, Shwetank</au><au>Kumar, Sumit</au><au>Barath, Raj KR</au><format>book</format><genre>document</genre><ristype>GEN</ristype><atitle>SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images</atitle><jtitle>arXiv.org</jtitle><date>2023-07-06</date><risdate>2023</risdate><eissn>2331-8422</eissn><abstract>In this paper, we propose a sentiment-enriched lightweight network SeLiNet and an end-to-end on-device pipeline for contextual emotion recognition in images. SeLiNet model consists of body feature extractor, image aesthetics feature extractor, and learning-based fusion network which jointly estimates discrete emotion and human sentiments tasks. On the EMOTIC dataset, the proposed approach achieves an Average Precision (AP) score of 27.17 in comparison to the baseline AP score of 27.38 while reducing the model size by &gt;85%. In addition, we report an on-device AP score of 26.42 with reduction in model size by &gt;93% when compared to the baseline.</abstract><cop>Ithaca</cop><pub>Cornell University Library, arXiv.org</pub><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier EISSN: 2331-8422
ispartof arXiv.org, 2023-07
issn 2331-8422
language eng
recordid cdi_proquest_journals_2834345278
source Publicly Available Content Database
subjects Emotion recognition
Emotions
Feature extraction
Lightweight
title SeLiNet: Sentiment enriched Lightweight Network for Emotion Recognition in Images
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-13T23%3A35%3A10IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest&rft_val_fmt=info:ofi/fmt:kev:mtx:book&rft.genre=document&rft.atitle=SeLiNet:%20Sentiment%20enriched%20Lightweight%20Network%20for%20Emotion%20Recognition%20in%20Images&rft.jtitle=arXiv.org&rft.au=Khargonkar,%20Tuneer&rft.date=2023-07-06&rft.eissn=2331-8422&rft_id=info:doi/&rft_dat=%3Cproquest%3E2834345278%3C/proquest%3E%3Cgrp_id%3Ecdi_FETCH-proquest_journals_28343452783%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2834345278&rft_id=info:pmid/&rfr_iscdi=true