TESSA—an image testbed for evaluating 2-D spatial similarity algorithms

• Published in: SIGIR forum 1994-09, Vol.28 (2), p.17-36
• Main Author: Gudivada, Venkat N.
• Format: Article
• Language: English
• ISSN: 0163-5840
• DOI: 10.1145/195498.195502

In multimedia database applications, a major class of users' requests require retrieving those images in the database that are spatially similar to the query image. To process such queries, a spatial similarity algorithm is required. A spatial similarity algorithm assesses the degree to which the spatial relationships among the domain objects in a database image conform to those specified in the query image. The recent ubiquitous interest in multimedia information systems has spurred a great interest in spatial similarity algorithms. A standard testbed of images is required not only to systematically evaluate these algorithms but also to compare and contrast them. However, there is no such testbed of images available for this purpose.In this paper, we describe a collection of images, referred to as TESSA, as a testbed for investigating the robustness of spatial similarity algorithms and their further use in evaluating the retrieval effectiveness of these algorithms. The TESSA collection comprises 160 images and are produced by generating 15 variants of each of the 10 original images. Image variants are produced by scale, rotation, and translation, and by an arbitrary composition of these three transformations. The variants are designed to inquire into the robustness of spatial similarity algorithms. To facilitate their further use in evaluating the retrieval effectiveness, each image in TESSA is considered as a query in turn, and an expert is asked to provide a rank ordering of the TESSA images with respect to the query image vis-a-vis expert provided rank ordering. System provided rank ordering for a query image is the rank ordering of TESSA images induced by a spatial similarity algorithm with respect to the query. Retrieval effectiveness of spatial similarity algorithms is then characterized by using R norm measure. This measure is based on both the expert and system provided rank orderings.