Forged document detection and writer identification through unsupervised deep learning approach

• Published in: Multimedia tools and applications 2024-02, Vol.83 (6), p.18459-18478
• Main Authors: Tyagi, Prachi, Agarwal, Khushboo, Jaiswal, Garima, Sharma, Arun, Rani, Ritu
• Format: Article
• Language: English
• Subjects: Art works; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Deep learning; Documents; Environmental monitoring; Feature extraction; Forensic sciences; Forgery; Handwriting; Hyperspectral imaging; Imaging techniques; Inks; Machine learning; Medical imaging; Multimedia Information Systems; Remote monitoring; Remote sensing; Special Purpose and Application-Based Systems; Support vector machines; Track 6: Computer Vision for Multimedia Applications
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-023-16146-7

In recent years, there has been a significant increase in document forgery, which includes the fraudulent replication of currency, diplomas, and works of art. This has become a major issue due to the widespread usage of paper-based documentation. Handwriting is closely linked to document forgery and forensics as it possesses unique characteristics, including variations in text characters, pen pressure, writing angle, and stroke patterns, which makes it impossible to replicate accurately. As a result, handwriting serves as a personalized biometric that can be used to determine the authenticity of documents. However, traditional methods of writer identification are both time-consuming and destructive, requiring substantial expertise. To overcome these limitations, the study explores the potential of hyperspectral imaging (HSI) as a non-destructive and advanced approach for detecting and preventing document forgery. HSI provides detailed spectral information from a scene, making it possible to capture subtle spectral differences in handwriting samples. This imaging technique has diverse applications in various fields such as agriculture, environmental monitoring, remote sensing, forensics, document analysis, and medical imaging. Our study proposes a novel unsupervised approach, CAE-SVM that uses Convolutional Autoencoder (CAE) for feature extraction and Support Vector Machine (SVM) for writer identification. It was tested on the UWA writing ink hyperspectral images dataset for blue and black inks which is available publicly and compared with state-of-the-art methods and CNN. The proposed approach achieved the highest accuracy of 92.78% for blue ink, surpassing existing methods. The study’s results emphasize the efficacy of HSI as a potent forensic analysis tool for detecting and preventing document forgery.