Signed expressions spotting and recognition in an assistive technology system

An algorithm for processing the continuous stream of incoming video data to detect and recognize signed expressions is proposed. The aim is to improve an assistive technology system. The method is based on the dynamic flow of expression models between a repository of inactive models and pools of act...

Full description

Saved in:
Bibliographic Details
Published in:Procedia computer science 2024, Vol.239, p.1141-1148
Main Author: Kapuscinski, Tomasz
Format: Article
Language:English
Subjects:
Assistive Technology
Computer Vision
Human-Computer Interaction
Pattern Recognition
Sign Language Recognition
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:An algorithm for processing the continuous stream of incoming video data to detect and recognize signed expressions is proposed. The aim is to improve an assistive technology system. The method is based on the dynamic flow of expression models between a repository of inactive models and pools of active and completed models. When modeling signed expressions, so-called visemes are used. These are units smaller than words - the equivalent of phonemes known from speech recognition. Visemes are extracted by clustering descriptors of the holistic skeletons. The sequential-simultaneous nature of gestures is considered, and the modeling is carried out in four parallel channels related to the position and shape of both hands. The conditions for activation, deactivation, and recognition of the model as completed are formulated, assuming asynchronous changes in individual channels. The method is successfully tested on a demanding dataset recorded by potential users.
ISSN:1877-0509
1877-0509
DOI:10.1016/j.procs.2024.06.280