Integrated Personalized and Diversified Search Based on Search Logs

• Published in: IEEE transactions on knowledge and data engineering 2024-02, Vol.36 (2), p.1-14
• Main Authors: Liu, Jiongnan, Dou, Zhicheng, Nie, Jian-Yun, Wen, Ji-Rong
• Format: Article
• Language: English
• Subjects: Behavioral sciences; Customization; Diversification; History; Integration; Java; Measurement; personalized search; Queries; Search engines; Search problems; search result diversification; Training; Transformers
• ISSN: 1041-4347; 1558-2191
• DOI: 10.1109/TKDE.2023.3291006

Personalized search and search result diversification are two possible solutions to cope with the query ambiguity problem in search engines. In most existing studies, they have been investigated separately, but intuitively, they address the problem from two complementary perspectives and should be combined. Some recent work tried to combine them by restricting result diversification to the subtopics corresponding to the user's personal profile. However, diversification can be required even when the subtopics are outside the user's profile. In this paper, we propose a more general approach to integrate them based on users' implicit feedback in query logs. The proposed approach PER+DIV aggregates a document's novelty score and personal relevance score dynamically according to how much the query falls into the user's interests. To train the model based on user clicks in the logs, we consider user click as a result of both personal relevance and result diversity and a new method is proposed to isolate and model these two factors. To evaluate the model, we design several diversified and personalized metrics in addition to the traditional click-based metrics. Experimental results on a large-scale query log dataset show that the proposed integrated method significantly outperforms the existing personalization and diversification approaches.