Automatic Lipreading with Limited Training Data

Speech recognition solely based on visual information such as the lip shape and its movement is referred to as lipreading. This paper presents an automatic lipreading technique for speaker dependent (SD) and speaker independent (SI) speech recognition tasks. Since the visual features are derived acc...

Full description

Saved in:
Bibliographic Details
Main Authors: Wang, S.L., Lau, W.H., Liew, A.W.C., Leung, S.H.
Format: Conference Proceeding
Language:English
Subjects:
Automatic speech recognition
Data mining
Data security
Hidden Markov models
Information security
Shape
Speech recognition
Spline
Training data
Video sequences
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Speech recognition solely based on visual information such as the lip shape and its movement is referred to as lipreading. This paper presents an automatic lipreading technique for speaker dependent (SD) and speaker independent (SI) speech recognition tasks. Since the visual features are derived according to the frame rate of the video sequence, spline representation is then employed to translate the discrete-time sampled visual features into continuous domain. The spline coefficients in the same word class are constrained to have similar expression and can be estimated from the training data by the EM algorithm. In addition, an adaptive multi-model approach is proposed to overcome the variation caused by different speaking style in speaker-independent recognition task. The experiments are carried out to recognize the ten English digits and an accuracy of 96% for speaker dependent recognition and 88% for speaker independent recognition have been achieved, which shows the superiority of our approach compared with other classifiers investigated
ISSN:1051-4651
2831-7475
DOI:10.1109/ICPR.2006.301