Towards realizing gesture-to-speech conversion with a HMM-based bilingual speech synthesis system

This paper realizes a gesture-to-speech conversion system to solve the communication problem between healthy people and speech disorders. An improved speeded up robust features (SURF) algorithm is adopted for static gesture recognition by combining Kinect sensor. Meanwhile, a Hidden Markov Model (HM...

Full description

Saved in:
Bibliographic Details
Main Authors: Hongwu Yang, Xiaochun An, Dong Pei, Yitong Liu
Format: Conference Proceeding
Language:English
Subjects:
Assistive technology
context-dependent label
Gesture recognition
hidden Markov Model
Hidden Markov models
improved SURF algorithm
Kinect sensor
Mandarin-Tibetan bilingual speech synthe-sis
Semantics
Speech
Speech recognition
Speech synthesis
static gesture recognition
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper realizes a gesture-to-speech conversion system to solve the communication problem between healthy people and speech disorders. An improved speeded up robust features (SURF) algorithm is adopted for static gesture recognition by combining Kinect sensor. Meanwhile, a Hidden Markov Model (HMM) based Mandarin-Tibetan bilingual speech synthesis system is developed by using speaker adaptive training. A set of semantic rules is designed for the static gestures. Chinese or Tibetan context-dependent labels of recognized static gestures are generated according to the semantic rules. The recognized gestures are finally converted to the Mandarin or Tibetan by using the Mandarin-Tibetan bilingual speech synthesis system with the context-dependent labels. Tests show that the static gesture recognition rate of the designed system achieves 97.1%. Subjective evaluation demonstrates that synthesized speech can get 4.0 of the mean opinion score (MOS) on synthesized speech.
DOI:10.1109/ICOT.2014.6956608