Design of a two-stage ASCII recognizer for the case-sensitive inputs in handwritten and gesticulation mode of the text-entry interface

• Published in: Multimedia tools and applications 2024-02, Vol.83 (30), p.75101-75145
• Main Authors: Kirupakaran, Anish Monsley, Yadav, Kuldeep Singh, Saidulu, Naragoni, Barlaskar, Saharul Alom, Laskar, Rabul Hussain
• Format: Article
• Language: English
• Subjects: Accuracy; Artificial neural networks; Character recognition; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Handwriting recognition; Multimedia Information Systems; Neural networks; Pattern recognition; Special Purpose and Application-Based Systems; Track 3: Biometrics and HCI
• ISSN: 1573-7721; 1380-7501; 1573-7721
• DOI: 10.1007/s11042-024-18261-5

Unlike real-world objects, which remain the same irrespective of the changes in size on a fixed/varying scale, few alphanumeric characters become identical because of their case-ambiguous nature. Recognizing alphabets becomes further complex when different characters are gesticulated/handwritten with the same pattern or become similar due to the gesture/handwriting style. The generalization ability of 2D deep convolutional neural networks (DCNN) results in the misclassification of these characters. To overcome this, we propose a two-stage recognition model that comprises DCNN, advisor unit (AU), and conflict-resolving post-decision support modules (PDSM-I and PDSM-II). PDSM-I is an artificial neural network trained on the proposed handcrafted features engineered to capture size-variant and size-invariant characteristics. PDSM-II is a two-stream 1D convolutional neural network (Ca1DNet) designed to capture the learned descriptors from the convolutions of 1D kernels. Using four state-of-the-art DCNN models, experimentation is carried out on two bases: (i) the unmerged; (ii) the merged data of EMNIST, NIST handwritten, and NITS hand gesture databases. These experiments help to show the fall/gap in recognition rate because of the case-sensitive characters in unmerged data conditions. PDSM-I with DCNN results in an average accuracy of EMNIST – 87.89%; NIST – 91.63%; NITS – 96.60% over the existing DCNN models’ performance in unmerged conditions. Similarly, when PDSM-II gets used with DCNN, an average accuracy of 87.96% (EMNIST), 91.79% (NIST), and 96.91% (NITS) is achieved. These results suggest that case-sensitive characters from handwritten/gestures can be recognized, provided the peripheral information of these characters is preserved.