Deep Descriptor Learning with Auxiliary Classification Loss for Retrieving Images of Silk Fabrics in the Context of Preserving European Silk Heritage

With the growing number of digitally available collections consisting of images depicting relevant objects from the past in relation with descriptive annotations, the need for suitable information retrieval techniques is becoming increasingly important to support historians in their work. In this co...

Full description

Saved in:
Bibliographic Details
Published in:ISPRS international journal of geo-information 2022-02, Vol.11 (2), p.82
Main Authors: Dorozynski, Mareike, Rottensteiner, Franz
Format: Article
Language:English
Subjects:
Annotations
Artificial neural networks
auxiliary classification loss
Classification
Clustering
Collections
Context
continuous triplet margin
Cultural heritage
Datasets
Deep learning
Evaluation
Experiments
Fabrics
fine-grained similarity
Historians
Image classification
Image management
Image retrieval
Information processing
Information retrieval
Learning
Neural networks
Properties
semantic similarity
Semantics
Silk
Training
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:With the growing number of digitally available collections consisting of images depicting relevant objects from the past in relation with descriptive annotations, the need for suitable information retrieval techniques is becoming increasingly important to support historians in their work. In this context, we address the problem of image retrieval for searching records in a database of silk fabrics. The descriptors, used as an index to the database, are learned by a convolutional neural network, exploiting the available annotations to automatically generate training data. Descriptor learning is combined with auxiliary classification loss with the aim of supporting the clustering in the descriptor space with respect to the properties of the depicted silk objects, such as the place or time of origin. We evaluate our approach on a dataset of fabric images in a kNN-classification, showing promising results with respect to the ability of the descriptors to represent semantic properties of silk fabrics; integrating the auxiliary loss improves the overall accuracy by 2.7% and the average F1 score by 5.6%. It can be observed that the largest improvements can be obtained for variables with imbalanced class distributions. An evaluation on the WikiArt dataset demonstrates the transferability of our approach to other digital collections.
ISSN:2220-9964
2220-9964
DOI:10.3390/ijgi11020082