UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition

• Published in: arXiv.org 2024-09
• Main Authors: Wang, Bin, Gu, Zhuangcheng, Liang, Guang, Xu, Chao, Zhang, Bo, Shi, Botian, He, Conghui
• Format: Article
• Language: English
• Subjects: Datasets; Formulas (mathematics); Mathematical analysis; Model accuracy; Recognition; Robustness (mathematics)
• ISSN: 2331-8422

The paper introduces the UniMER dataset, marking the first study on Mathematical Expression Recognition (MER) targeting complex real-world scenarios. The UniMER dataset includes a large-scale training set, UniMER-1M, which offers unprecedented scale and diversity with one million training instances to train high-quality, robust models. Additionally, UniMER features a meticulously designed, diverse test set, UniMER-Test, which covers a variety of formula distributions found in real-world scenarios, providing a more comprehensive and fair evaluation. To better utilize the UniMER dataset, the paper proposes a Universal Mathematical Expression Recognition Network (UniMERNet), tailored to the characteristics of formula recognition. UniMERNet consists of a carefully designed encoder that incorporates detail-aware and local context features, and an optimized decoder for accelerated performance. Extensive experiments conducted using the UniMER-1M dataset and UniMERNet demonstrate that training on the large-scale UniMER-1M dataset can produce a more generalizable formula recognition model, significantly outperforming all previous datasets. Furthermore, the introduction of UniMERNet enhances the model's performance in formula recognition, achieving higher accuracy and speeds. All data, models, and code are available at https://github.com/opendatalab/UniMERNet.