Gorthaur: A Portfolio Approach for Dynamic Selection of Multi-Armed Bandit Algorithms for Recommendation

• Main Authors: Gutowski, Nicolas, Amghar, Tassadit, Camp, Olivier, Chhel, Fabien
• Format: Conference Proceeding
• Language: English
• Subjects: Application of Reinforcement Learning; Contextual Multi-Armed Bandit; Heuristic; Portfolio Approach; Recommender Systems
• ISSN: 2375-0197
• DOI: 10.1109/ICTAI.2019.00161

Recommendation systems must reach a good global accuracy but also diversify their recommendations. Despite the theoretically grounded guarantees, we observe that multi-armed bandit algorithms obtain different results depending on the nature of the real-world applications or the offline datasets that is used. Thus, before choosing an algorithm, it is necessary to carry out a preliminary offline evaluation on the criteria of global accuracy and, if necessary, diversity. However, recommendation systems are notoriously hard to evaluate due to their interactive and dynamic nature. Hence, we have implemented a portfolio approach, entitled Gorthaur, which uses a heuristic to dynamically select multi-armed bandits algorithms used for recommending. Thus, Gorthaur aims at selecting algorithms by maximising the two criteria of global accuracy and diversity. Following our results, we observe that the advantage of using Gorthaur is twofold: 1) Find a trade-off in cases where there is no prior knowledge about the nature of the dataset or the recommendation application we want to deploy; 2) Rapidly shed light on a set of optimal algorithms.