Image annotation: the effects of content, lexicon and annotation method

Image annotation is the process of assigning metadata to images, allowing effective retrieval by text-based search techniques. Despite the lots of efforts in automatic multimedia analysis, automatic semantic annotation of multimedia is still inefficient due to the problems in modeling high-level sem...

Full description

Saved in:
Bibliographic Details
Published in:International journal of multimedia information retrieval 2020-09, Vol.9 (3), p.191-203
Main Authors: Theodosiou, Zenonas, Tsapatsoulis, Nicolas
Format: Article
Language:English
Subjects:
Algorithms
Annotations
Classification
Computer Science
Crowdsourcing
Data Mining and Knowledge Discovery
Database Management
Datasets
Deep learning
Image annotation
Image Processing and Computer Vision
Image quality
Image retrieval
Information retrieval
Information Storage and Retrieval
Information Systems Applications (incl.Internet)
Keywords
Machine learning
Multimedia
Multimedia Information Systems
Neural networks
Queries
Regular Paper
Semantics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Image annotation is the process of assigning metadata to images, allowing effective retrieval by text-based search techniques. Despite the lots of efforts in automatic multimedia analysis, automatic semantic annotation of multimedia is still inefficient due to the problems in modeling high-level semantic terms. In this paper, we examine the factors affecting the quality of annotations collected through crowdsourcing platforms. An image dataset was manually annotated utilizing: (1) a vocabulary consists of preselected set of keywords, (2) an hierarchical vocabulary and (3) free keywords. The results show that the annotation quality is affected by the image content itself and the used lexicon. As we expected while annotation using the hierarchical vocabulary is more representative, the use of free keywords leads to increased invalid annotation. Finally, it is shown that images requiring annotations that are not directly related to their content (i.e., annotation using abstract concepts) lead to accrue annotator inconsistency revealing in that way the difficulty in annotating such kind of images is not limited to automatic annotation, but it is a generic problem of annotation.
ISSN:2192-6611
2192-662X
DOI:10.1007/s13735-020-00193-z