Task-aware network: Mitigation of task-aware and task-free performance gap in online continual learning

• Published in: Neurocomputing (Amsterdam) 2023-10, Vol.552, p.126527, Article 126527
• Main Authors: Hong, Yongwon, Park, Sungho, Byun, Hyeran
• Format: Article
• Language: English
• Subjects: Online Continual Learning; Online Incremental Learning
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2023.126527

Online continual learning (OCL) is a challenging task that accesses training data only once and trains a deep model to cover a new task while preserving the capability of previous tasks. Due to the poor performance, early works tend to measure the performance in a task-aware (TA) manner, which assumes task labels in the testing time. However, requiring such an assumption is far from the real world. Although recent works also measure the performance without the assumption (i.e., a task-free (TF) manner), there is still a large performance gap between TA and TF in the online environment. In this paper, we observe that the severe performance gap is due to the overlapped information between different tasks. Inspired by this observation, we propose the Task-Aware Network (TANet), which learns both class-specific and task-specific information using a dual-encoder framework. However, it is difficult to learn the global distribution of previous tasks since only a few samples (i.e., exemplars) of the tasks can be accessed. To solve the problem, we utilize the prototype feature that reflects the global distribution of each task. Using the prototype features, TANet learns the task-specific information with Prototype-based Task Contrastive Learning (PTCL) and generates pseudo-task labels. In testing time, we introduce the hybrid task-free (TF) classifier, which only activates the task that the pseudo label indicates. The proposed method achieves state-of-the-art performances in various datasets for OCL.