Multi-scene ancient chinese text recognition

• Published in: Neurocomputing (Amsterdam) 2020-02, Vol.377, p.64-72
• Main Authors: Wang, Kaili, Yi, Yaohua, Liu, Junjie, Lu, Liqiong, Song, Ying
• Format: Article
• Language: English
• Subjects: Multi-model ensemble (MME); Multi-scene ancient chinese text recognition (MACR); Synthetic dataset
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2019.10.029

•With the lack of related research, the problem of multi-scene ancient Chinese text recognition (MACR) is first proposed.•An multi-scene ancient Chinese text (MACT) dataset is established, which includes a testing set collected from real scenes and a synthetic training set.•A baseline method is used for experiments with 66.94% top-1 accuracy, which far surpasses subjective human recognition performance, and an improved method based on multi-model ensemble (MME) is proposed, which achieves 73.36% top-1 accuracy. Multi-scene ancient Chinese text recognition (MACR) is a challenging task for ordinary people without relevant professional knowledge. Due to language barriers and the lack of related open datasets, there is little research on MACR. In this paper, a multi-scene ancient Chinese text (MACT) dataset, formed by handwritten text, calligraphy, natural scene text in ancient fonts, is established that includes synthetic samples generated for training and real scene samples collected for testing. We first perform experiments on some CNN structures as the baseline method, and the top-1 recognition result, 66.94%, is approximately 13.96% higher than subjective human recognition results. Furthermore, based on these models and confidence score from the baseline, a multi-model ensemble (MME) method is proposed, which adopts auxiliary datasets and a feature extraction method to augment data before training, utilizes different hyper-parameters to optimize in training, and integrates multiple models to improve recognition accuracy. The top-1 accuracy results of the MME method reach 73.36% and other top-n results also greatly surpass the baseline results. The MACT dataset is publicly available on the website11https://1drv.ms/u/s!AqAU14ep3HF7bhNy8KcOtjfEpbI. https://pan.baidu.com/s/1y-KdY-mzVfwDyKxNles6vw, Passward: 7dqu..