A novel domain independent scene text localizer

• Published in: Pattern recognition 2025-02, Vol.158, p.111015, Article 111015
• Main Authors: Roy, Ayush, Palaiahnakote, Shivakumara, Pal, Umapada, Liu, Cheng-Lin
• Format: Article
• Language: English
• Subjects: Attention module; Drone images; Scene text detection; Transformer; Underwater images
• ISSN: 0031-3203
• DOI: 10.1016/j.patcog.2024.111015

•The proposed domain independent model for scene text localization is new.•Exploring partial convolution with Yolov5-transformer for feature extraction.•Integrating the swin transformer with the novel channel attention modules.•The result of the proposed method is superior to the existing methods. Text localization across multiple domains is crucial for applications like autonomous driving and tracking marathon runners. This work introduces DIPCYT, a novel model that utilizes Domain Independent Partial Convolution and a Yolov5-based Transformer for text localization in scene images from various domains, including natural scenes, underwater, and drone images. Each domain presents unique challenges: underwater images suffer from poor quality and degradation, drone images suffer from tiny text and loss of shapes, and scene images suffer from arbitrarily oriented, shaped text. Additionally, license plates in drone images may not provide rich semantic information compared to other text types due to loss of contextual information between characters. To tackle these challenges, DIPCYT employs new partial convolution layers within Yolov5 and integrates Transformer detection heads with a novel Fourier Positional Convolutional Block Attention Module (FPCBAM). This approach leverages common text properties across domains, such as contextual (global) and spatial (local) relationships. Experimental results demonstrate that DIPCYT outperforms existing methods, achieving F-scores of 0.90, 0.90, 0.77, 0.85, 0.85, and 0.88 on Total-Text, ICDAR 2015, ICDAR 2019 MLT, CTW1500, Drone, and Underwater datasets, respectively.