ComRank: metasearch and automatic ranking of XML retrieval system

Different information retrieval (IR) systems often return very diverse results lists for the same query. This is problematic for users since no one IR system works best for every scenario, and it is difficult for the user to know which system will work best a priori. The challenge of metasearch is t...

Full description

Saved in:
Bibliographic Details
Main Authors: Woodley, A., Geva, S.
Format: Conference Proceeding
Language:English
Subjects:
Abstracts
Automatic control
Data communication
Humans
Information retrieval
Information technology
Metasearch
Software engineering
Web sites
XML
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Different information retrieval (IR) systems often return very diverse results lists for the same query. This is problematic for users since no one IR system works best for every scenario, and it is difficult for the user to know which system will work best a priori. The challenge of metasearch is to merge results lists from several IR systems, with the goal of outperforming each of the constituent systems. This paper presents ComRank, a metasearch system that discriminates in favour of results that (1) originate by consensus amongst several systems; (2) are highly ranked in their original systems and (3) originate from the better performing systems. Importantly, ComRank determines the `better' performing systems without the need for human judgements. Rather, it uses an automatic assessment process that ranks systems by their pseudo-relevance, as derived from highly ranked results in ComRank's list. We apply our methods to the INEX Collection, which is an unexplored domain for these methods, and show that they are comparable to or better than baseline alternatives
DOI:10.1109/CW.2005.30