Using Speech for Handwritten Mathematical Expression Recognition Disambiguation

The main goal of this work is to set up a multimodal system dedicated to mathematical expression recognition. In the proposed architecture, the transcription coming out from a speech recognition system is used to disambiguate the result of a handwriting recognition module. A set of keywords is built...

Full description

Saved in:
Bibliographic Details
Main Authors: Medjkoune, S., Mouchere, H., Petitrenaud, S., Viard-Gaudin, C.
Format: Conference Proceeding
Language:English
Subjects:
Data fusion
Data integration
Educational institutions
Handwriting recognition
Mathematical expression
Speech
Speech recognition
Vocabulary
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The main goal of this work is to set up a multimodal system dedicated to mathematical expression recognition. In the proposed architecture, the transcription coming out from a speech recognition system is used to disambiguate the result of a handwriting recognition module. A set of keywords is built from the transcription module and used to rescore the outputs of both the handwriting classifier and the structural analysis module. Performances evaluated on the HAMEX dataset show a significant improvement over a single modality system.
DOI:10.1109/ICFHR.2012.288