Continual learning for adaptive social network identification

• Published in: Pattern recognition letters 2024-04, Vol.180, p.82-89
• Main Authors: Magistri, Simone, Baracchi, Daniele, Shullani, Dasara, Bagdanov, Andrew D., Piva, Alessandro
• Format: Article
• Language: English
• Subjects: Continual learning; Multimedia forensics; Social network identification
• ISSN: 0167-8655; 1872-7344
• DOI: 10.1016/j.patrec.2024.02.020

The popularity of social networks as primary mediums for sharing visual content has made it crucial for forensic experts to identify the original platform of multimedia content. Various methods address this challenge, but the constant emergence of new platforms and updates to existing ones often render forensic tools ineffective shortly after release. This necessitates the regular updating of methods and models, which can be particularly cumbersome for techniques based on neural networks which cannot quickly adapt to new classes without sacrificing performance on previously learned ones – a phenomenon known as catastrophic forgetting. Recently, researchers aimed at mitigating this problem via a family of techniques known as continual learning. In this paper we study the applicability of continual learning techniques to the social network identification task by evaluating two relevant forensic scenarios: Incremental Social Platform Classification, for handling newly introduced social media platforms, and Incremental Social Version Classification, for addressing updated versions of a set of existing social networks. We perform an extensive experimental evaluation of a variety of continual learning approaches applied to these two scenarios. Experimental results demonstrate that, although Continual Social Network Identification remains a difficult problem, catastrophic forgetting can be significantly mitigated in both scenarios by retaining only a fraction of the image patches from past task training samples or by employing previous tasks prototypes. •We investigate continual learning methods for social network identification.•We exploit a state-of-the-art dual-branch neural network designed for this task.•We define two realistic experimental scenarios on multiple datasets.•Exemplar-based methods yield good performance with limited memory requirements.•Prototype-based methods are a viable solution when storing exemplars is not feasible.