Hash Learning With Variable Quantization for Large-Scale Retrieval

Approximate Nearest Neighbor(ANN) search is the core problem in many large-scale machine learning and computer vision applications such as multimodal retrieval. Hashing is becoming increasingly popular, since it can provide efficient similarity search and compact data representations suitable for ha...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on circuits and systems for video technology 2022-05, Vol.32 (5), p.2624-2637
Main Authors: Cao, Yuan, Chen, Sheng, Gui, Jie, Qi, Heng, Li, Zhiyang, Liu, Chao
Format: Article
Language:English
Subjects:
Algorithms
Approximate nearest neighbor search
Binary codes
Computer vision
Data search
Eigenvalues and eigenfunctions
Forecasting
Hamming distance
Hash based algorithms
hashing
Integers
Iterative methods
Kernel
Machine learning
Measurement
multimodal retrieval
Principal component analysis
quantization
Quantization (signal)
Retrieval
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Approximate Nearest Neighbor(ANN) search is the core problem in many large-scale machine learning and computer vision applications such as multimodal retrieval. Hashing is becoming increasingly popular, since it can provide efficient similarity search and compact data representations suitable for handling such large-scale ANN search problems. Most hashing algorithms concentrate on learning more effective projection functions. However, the accuracy loss in the quantization step has been ignored and barely studied. In this paper, we analyse the importance of various projected dimensions, distribute them into several groups and quantize them with two types of values which can both better preserve the neighborhood structure among data. One is Variable Integer-based Quantization (VIQ) that quantizes each projected dimension with integer values. The other is Variable Codebook-based Quantization (VCQ) that quantizes each projected dimension with corresponding codebook values. We conduct experiments on five common public data sets containing up to one million vectors. The results show that the proposed VCQ and VIQ algorithms can both achieve much higher accuracy than state-of-the-art quantization methods. Furthermore, although VCQ performs better than VIQ, ANN search with VIQ provides much higher search efficiency.
ISSN:1051-8215
1558-2205
DOI:10.1109/TCSVT.2021.3051358