A novel machine learning approach for scene text extraction

• Published in: Future generation computer systems 2018-10, Vol.87, p.328-340
• Main Authors: Ansari, Ghulam Jillani, Shah, Jamal Hussain, Yasmin, Mussarat, Sharif, Muhammad, Fernandes, Steven Lawrence
• Format: Article
• Language: English
• Subjects: CNN; LBP; SVM; Text detection; Text recognition
• ISSN: 0167-739X; 1872-7115
• DOI: 10.1016/j.future.2018.04.074

Image based text extraction is a popular and challenging research field in computer vision in recent times. In this paper, an exigent aspect such as natural scene text identification and extraction has been investigated due to cluttered background, unstructured scenes, orientations, ambiguities and much more. For text identification, contrast enhancement is done by applying LUV channel on an input image to get perfect stable regions. Then L-Channel is selected for region segmentation using standard segmentation technique MSER. In order to differentiate among text/non-text regions, various geometrical properties are also considered in this work. Further, classification of connected components is performed to obtain segmented image by the fusion of two feature descriptors LBP and T-HOG. Firstly both features descriptors are separately classified using linear SVM(s). Secondly the results of both are combined by applying weighted sum fusion technique to classify into text/non-text portions. In text recognition, text regions are recognized and labeled with a novel CNN network. The CNN output is stored in a text file to make a text word. Finally, the text file is searched through lexicon for proper optimized scene text word incorporating hamming distance (error correction) technique if necessary. •A novel method is proposed for scene text extraction, recognition and correction.•MSER technique is used for segmenting text/non-text areas after preprocessing.•A feature fusion approach is used for CC classification using SVM and weighted sum.•A CNN model is proposed for character labeling and hamming distance for correction.•Conclusions and analysis are performed on datasets ICDAR2003, SVT and IIIT5k.