A Pattern Recognition-Based Multi-Modal HCI Using 3 Sensory Channels

This paper describes a PDA-based MMCR (Multi- Modal Command Recognizer for PDA control and handling) using double-touching with a finger by coupling embedded speech and KSSL recognizer, and suggests an improved synchronization method between multi-modalities for simultaneous multi-modality, for a pa...

Full description

Saved in:
Bibliographic Details
Main Authors: Jung-Hyun Kim, Kue-Bum Lee, Kwang-Seok Hong
Format: Conference Proceeding
Language:English
Subjects:
Application software
Communication system control
Computer interfaces
Context modeling
Fuses
Handheld computers
Human computer interaction
Pattern recognition
Personal digital assistants
Speech recognition
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper describes a PDA-based MMCR (Multi- Modal Command Recognizer for PDA control and handling) using double-touching with a finger by coupling embedded speech and KSSL recognizer, and suggests an improved synchronization method between multi-modalities for simultaneous multi-modality, for a pattern recognition-based neo multi-modal HCI. The MMCR fuses and recognizes 16 word-based command models that are represented by stylus, speech and KSSI (Korean Standard Sign language), and then translates the recognition result into synthetic speech and visual illustration, for multi-modal PDA handling and interaction.
DOI:10.1109/SERA.2007.23