Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach

Haptic feedback can improve safety of teleoperated robots when situational awareness is limited or operators are inattentive. Standard potential field approaches increase haptic resistance as an obstacle is approached, which is desirable when the operator is unaware of the obstacle but undesirable w...

Full description

Saved in:
Bibliographic Details
Published in:IEEE robotics and automation letters 2024-03, Vol.9 (3), p.2152-2159
Main Authors: Zhong, Ninghan, Hauser, Kris
Format: Article
Language:English
Subjects:
Feedback
Haptics
Obstacle avoidance
Potential fields
Robots
Safety
Situational awareness
User experience
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites
container_end_page 2159
container_issue 3
container_start_page 2152
container_title IEEE robotics and automation letters
container_volume 9
creator Zhong, Ninghan
Hauser, Kris
description Haptic feedback can improve safety of teleoperated robots when situational awareness is limited or operators are inattentive. Standard potential field approaches increase haptic resistance as an obstacle is approached, which is desirable when the operator is unaware of the obstacle but undesirable when the movement is intentional, such as when the operator wishes to inspect or manipulate an object. This letter presents a novel haptic teleoperation framework that estimates the operator's attentiveness to obstacles and dampens haptic feedback for intentional movement. A biologically-inspired attention model is developed based on computational working memory theories to integrate visual saliency estimation with spatial mapping. The attentiveness map is generated in real-time, and our system renders lower haptic forces for obstacles that the operator is estimated to be aware of. Experimental results in simulation show that the proposed framework outperforms haptic teleoperation without attentiveness estimation in terms of task performance, robot safety, and user experience.
doi_str_mv 10.1109/LRA.2024.3354613
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2918647539</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2918647539</sourcerecordid><originalsourceid>FETCH-LOGICAL-c224t-2d8d7ebc1ccb6f6a573c02b9ca377411e57813d8246b5d9687c21a8617f304353</originalsourceid><addsrcrecordid>eNpNkM1rAjEQxUNpoWK99xjoeW2-s3tcxNaCIog9hyQ7S1fsZptEof99V_TQ08zwHvN4P4SeKZlTSqrX9a6eM8LEnHMpFOV3aMK41gXXSt3_2x_RLKUDIYRKpnklJwjqnKHP3Rl6SAlv7ICXKXffNnehx22IeGWH3Hm8hyOEAeJVCC3eBNcdAe-CCxlvXcrWj2d9Dl1jew_Y9g2uhyEG67-e0ENrjwlmtzlFn2_L_WJVrLfvH4t6XXjGRC5YUzYanKfeO9UqKzX3hLnK27GAoBSkLilvSiaUk02lSu0ZtaWiuuVEcMmn6OX6d4z9OUHK5hBOsR8jDatoqYSWvBpd5OryMaQUoTVDHBvHX0OJufA0I09z4WluPPkf3MRn1g</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2918647539</pqid></control><display><type>article</type><title>Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach</title><source>IEEE Xplore (Online service)</source><creator>Zhong, Ninghan ; Hauser, Kris</creator><creatorcontrib>Zhong, Ninghan ; Hauser, Kris</creatorcontrib><description>Haptic feedback can improve safety of teleoperated robots when situational awareness is limited or operators are inattentive. Standard potential field approaches increase haptic resistance as an obstacle is approached, which is desirable when the operator is unaware of the obstacle but undesirable when the movement is intentional, such as when the operator wishes to inspect or manipulate an object. This letter presents a novel haptic teleoperation framework that estimates the operator's attentiveness to obstacles and dampens haptic feedback for intentional movement. A biologically-inspired attention model is developed based on computational working memory theories to integrate visual saliency estimation with spatial mapping. The attentiveness map is generated in real-time, and our system renders lower haptic forces for obstacles that the operator is estimated to be aware of. Experimental results in simulation show that the proposed framework outperforms haptic teleoperation without attentiveness estimation in terms of task performance, robot safety, and user experience.</description><identifier>ISSN: 2377-3766</identifier><identifier>EISSN: 2377-3766</identifier><identifier>DOI: 10.1109/LRA.2024.3354613</identifier><language>eng</language><publisher>Piscataway: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</publisher><subject>Feedback ; Haptics ; Obstacle avoidance ; Potential fields ; Robots ; Safety ; Situational awareness ; User experience</subject><ispartof>IEEE robotics and automation letters, 2024-03, Vol.9 (3), p.2152-2159</ispartof><rights>Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><orcidid>0000-0002-5207-1527 ; 0000-0002-4262-2989</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27924,27925</link.rule.ids></links><search><creatorcontrib>Zhong, Ninghan</creatorcontrib><creatorcontrib>Hauser, Kris</creatorcontrib><title>Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach</title><title>IEEE robotics and automation letters</title><description>Haptic feedback can improve safety of teleoperated robots when situational awareness is limited or operators are inattentive. Standard potential field approaches increase haptic resistance as an obstacle is approached, which is desirable when the operator is unaware of the obstacle but undesirable when the movement is intentional, such as when the operator wishes to inspect or manipulate an object. This letter presents a novel haptic teleoperation framework that estimates the operator's attentiveness to obstacles and dampens haptic feedback for intentional movement. A biologically-inspired attention model is developed based on computational working memory theories to integrate visual saliency estimation with spatial mapping. The attentiveness map is generated in real-time, and our system renders lower haptic forces for obstacles that the operator is estimated to be aware of. Experimental results in simulation show that the proposed framework outperforms haptic teleoperation without attentiveness estimation in terms of task performance, robot safety, and user experience.</description><subject>Feedback</subject><subject>Haptics</subject><subject>Obstacle avoidance</subject><subject>Potential fields</subject><subject>Robots</subject><subject>Safety</subject><subject>Situational awareness</subject><subject>User experience</subject><issn>2377-3766</issn><issn>2377-3766</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNpNkM1rAjEQxUNpoWK99xjoeW2-s3tcxNaCIog9hyQ7S1fsZptEof99V_TQ08zwHvN4P4SeKZlTSqrX9a6eM8LEnHMpFOV3aMK41gXXSt3_2x_RLKUDIYRKpnklJwjqnKHP3Rl6SAlv7ICXKXffNnehx22IeGWH3Hm8hyOEAeJVCC3eBNcdAe-CCxlvXcrWj2d9Dl1jew_Y9g2uhyEG67-e0ENrjwlmtzlFn2_L_WJVrLfvH4t6XXjGRC5YUzYanKfeO9UqKzX3hLnK27GAoBSkLilvSiaUk02lSu0ZtaWiuuVEcMmn6OX6d4z9OUHK5hBOsR8jDatoqYSWvBpd5OryMaQUoTVDHBvHX0OJufA0I09z4WluPPkf3MRn1g</recordid><startdate>20240301</startdate><enddate>20240301</enddate><creator>Zhong, Ninghan</creator><creator>Hauser, Kris</creator><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><orcidid>https://orcid.org/0000-0002-5207-1527</orcidid><orcidid>https://orcid.org/0000-0002-4262-2989</orcidid></search><sort><creationdate>20240301</creationdate><title>Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach</title><author>Zhong, Ninghan ; Hauser, Kris</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c224t-2d8d7ebc1ccb6f6a573c02b9ca377411e57813d8246b5d9687c21a8617f304353</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Feedback</topic><topic>Haptics</topic><topic>Obstacle avoidance</topic><topic>Potential fields</topic><topic>Robots</topic><topic>Safety</topic><topic>Situational awareness</topic><topic>User experience</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Zhong, Ninghan</creatorcontrib><creatorcontrib>Hauser, Kris</creatorcontrib><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics &amp; Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts – Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE robotics and automation letters</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Zhong, Ninghan</au><au>Hauser, Kris</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach</atitle><jtitle>IEEE robotics and automation letters</jtitle><date>2024-03-01</date><risdate>2024</risdate><volume>9</volume><issue>3</issue><spage>2152</spage><epage>2159</epage><pages>2152-2159</pages><issn>2377-3766</issn><eissn>2377-3766</eissn><abstract>Haptic feedback can improve safety of teleoperated robots when situational awareness is limited or operators are inattentive. Standard potential field approaches increase haptic resistance as an obstacle is approached, which is desirable when the operator is unaware of the obstacle but undesirable when the movement is intentional, such as when the operator wishes to inspect or manipulate an object. This letter presents a novel haptic teleoperation framework that estimates the operator's attentiveness to obstacles and dampens haptic feedback for intentional movement. A biologically-inspired attention model is developed based on computational working memory theories to integrate visual saliency estimation with spatial mapping. The attentiveness map is generated in real-time, and our system renders lower haptic forces for obstacles that the operator is estimated to be aware of. Experimental results in simulation show that the proposed framework outperforms haptic teleoperation without attentiveness estimation in terms of task performance, robot safety, and user experience.</abstract><cop>Piscataway</cop><pub>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</pub><doi>10.1109/LRA.2024.3354613</doi><tpages>8</tpages><orcidid>https://orcid.org/0000-0002-5207-1527</orcidid><orcidid>https://orcid.org/0000-0002-4262-2989</orcidid></addata></record>
fulltext fulltext
identifier ISSN: 2377-3766
ispartof IEEE robotics and automation letters, 2024-03, Vol.9 (3), p.2152-2159
issn 2377-3766
2377-3766
language eng
recordid cdi_proquest_journals_2918647539
source IEEE Xplore (Online service)
subjects Feedback
Haptics
Obstacle avoidance
Potential fields
Robots
Safety
Situational awareness
User experience
title Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-29T00%3A04%3A27IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Attentiveness%20Map%20Estimation%20for%20Haptic%20Teleoperation%20of%20Mobile%20Robot%20Obstacle%20Avoidance%20and%20Approach&rft.jtitle=IEEE%20robotics%20and%20automation%20letters&rft.au=Zhong,%20Ninghan&rft.date=2024-03-01&rft.volume=9&rft.issue=3&rft.spage=2152&rft.epage=2159&rft.pages=2152-2159&rft.issn=2377-3766&rft.eissn=2377-3766&rft_id=info:doi/10.1109/LRA.2024.3354613&rft_dat=%3Cproquest_cross%3E2918647539%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c224t-2d8d7ebc1ccb6f6a573c02b9ca377411e57813d8246b5d9687c21a8617f304353%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2918647539&rft_id=info:pmid/&rfr_iscdi=true