Processing Real-Life Recordings of Facial Expressions of Polish Sign Language Using Action Units

Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall co...

Full description

Saved in:
Bibliographic Details
Published in:Entropy (Basel, Switzerland) Switzerland), 2023-01, Vol.25 (1), p.120
Main Authors: Irasiak, Anna, Kozak, Jan, Piasecki, Adam, Stęclik, Tomasz
Format: Article
Language:English
Subjects:
action units
Animation
Annotations
automatic translation
Communication
Deafness
Dictionaries
entropy of real data
Facial expressions
Foreign languages
Gestures
Handicapped accessibility
Language policy
Linguistics
Machine learning
Machine translation
National languages
People with disabilities
Polish language
R&D
Research & development
Second languages
Sign language
Translation methods and strategies
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Automatic translation between the national language and sign language is a complex process similar to translation between two different foreign languages. A very important aspect is the precision of not only manual gestures but also facial expressions, which are extremely important in the overall context of a sentence. In this article, we present the problem of including facial expressions in the automation of Polish-to-Polish Sign Language (PJM) translation-this is part of an ongoing project related to a comprehensive solution allowing for the animation of manual gestures, body movements and facial expressions. Our approach explores the possibility of using action unit (AU) recognition in the automatic annotation of recordings, which in the subsequent steps will be used to train machine learning models. This paper aims to evaluate entropy in real-life translation recordings and analyze the data associated with the detected action units. Our approach has been subjected to evaluation by experts related to Polish Sign Language, and the results obtained allow for the development of further work related to automatic translation into Polish Sign Language.
ISSN:1099-4300
1099-4300
DOI:10.3390/e25010120