Sequential Classification of ASL Signs in the Context of Daily Living Using RF Sensing

RF sensing based human activity and hand gesture recognition (HGR) methods have gained enormous popularity with the development of small package, high frequency radar systems and powerful machine learning tools. However, most HGR experiments in the literature have been conducted on individual gestur...

Full description

Saved in:
Bibliographic Details
Main Authors: Kurtoglu, Emre, Gurbuz, Ali C., Malaia, Evie, Griffin, Darrin, Crawford, Chris, Gurbuz, Sevgi Z.
Format: Conference Proceeding
Language:English
Subjects:
ASL recognition
Assistive technology
Gesture recognition
Human computer interaction
Radio frequency
RF sensing
Sensors
Sequential classification
Target recognition
trigger detection
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:RF sensing based human activity and hand gesture recognition (HGR) methods have gained enormous popularity with the development of small package, high frequency radar systems and powerful machine learning tools. However, most HGR experiments in the literature have been conducted on individual gestures and in isolation from preceding and subsequent motions. This paper considers the problem of American sign language (ASL) recognition in the context of daily living, which involves sequential classification of a continuous stream of signing mixed with daily activities. In particular, this paper investigates the efficacy of different RF input representations and fusion techniques for ASL and trigger gesture recognition tasks in a daily living scenario, which can be potentially used for sign language sensitive human-computer interfaces (HCI). The proposed approach involves first detecting and segmenting periods of motion, followed by feature level fusion of the range-Doppler map, micro-Doppler spectrogram, and envelope for classification with a bi-directional long short-term memory (BiL-STM) recurrent neural network. Results show 93.3% accuracy in identification of 6 activities and 4 ASL signs, as well as a trigger sign detection rate of 0.93.
ISSN:2375-5318
DOI:10.1109/RadarConf2147009.2021.9455178