Fine-Tuning CNN Image Retrieval with No Human Annotation

Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated d...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on pattern analysis and machine intelligence 2019-07, Vol.41 (7), p.1655-1668
Main Authors: Radenovic, Filip, Tolias, Giorgos, Chum, Ondrej
Format: Article
Language:English
Subjects:
Annotations
applications
Artificial neural networks
computing methodologies
Databases, Factual
Humans
Image management
Image Processing, Computer-Assisted - methods
Image reconstruction
Image representation
Image retrieval
Imaging, Three-Dimensional
neural nets
Neural Networks, Computer
pattern recognition
Pattern Recognition, Automated - methods
Power efficiency
processing and computer vision
Solid modeling
Task analysis
Three dimensional models
Three-dimensional displays
Training
Training data
Tuning
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated data, where a high quality of annotation is often crucial. In this work, we propose to fine-tune CNNs for image retrieval on a large collection of unordered images in a fully automated manner. Reconstructed 3D models obtained by the state-of-the-art retrieval and structure-from-motion methods guide the selection of the training data. We show that both hard-positive and hard-negative examples, selected by exploiting the geometry and the camera positions available from the 3D models, enhance the performance of particular-object retrieval. CNN descriptor whitening discriminatively learned from the same training data outperforms commonly used PCA whitening. We propose a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and average pooling and show that it boosts retrieval performance. Applying the proposed method to the VGG network achieves state-of-the-art performance on the standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.
ISSN:0162-8828
1939-3539
2160-9292
DOI:10.1109/TPAMI.2018.2846566