Learning for amalgamation: A multi-source transfer learning framework for sentiment classification

• Published in: Information sciences 2022-04, Vol.590, p.1-14
• Main Authors: Nguyen, Cuong V., Le, Khiem H., Tran, Anh M., Pham, Quang H., Nguyen, Binh T.
• Format: Article
• Language: English
• Subjects: LIFA; Low-resource NLP; Mixture of experts; Sentiment classification; Transfer learning
• ISSN: 0020-0255; 1872-6291
• DOI: 10.1016/j.ins.2021.12.059

Transfer learning plays an essential role in Deep Learning, which can remarkably improve the performance of the target domain, whose training data is not sufficient. Our work explores beyond the common practice of transfer learning with a single pre-trained model. We focus on the task of Vietnamese sentiment classification and propose LIFA, a framework to learn a unified embedding from several pre-trained models. We further propose two more LIFA variants that encourage the pre-trained models to either cooperate or compete with one another. Studying these variants sheds light on the success of LIFA by showing that sharing knowledge among the models is more beneficial for transfer learning. Moreover, we construct the AISIA-VN-Review-F dataset, the first large-scale Vietnamese sentiment classification database. We conduct extensive experiments on the AISIA-VN-Review-F and existing benchmarks to demonstrate the efficacy of LIFA compared to other techniques. To contribute to the Vietnamese NLP research, we publish our source code and datasets to the research community upon acceptance.