Aggregated Learning: A Vector-Quantization Approach to Learning Neural Network Classifiers

We consider the problem of learning a neural network classifier. Under the information bottleneck (IB) principle, we associate with this classification problem a representation learning problem, which we call "IB learning". We show that IB learning is, in fact, equivalent to a special clas...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2021-06
Main Authors: Soflaei, Masoumeh, Guo, Hongyu, Al-Bashabsheh, Ali, Mao, Yongyi, Zhang, Richong
Format: Article
Language:English
Subjects:
Classifiers
Image classification
Learning
Neural networks
Object recognition
Representations
Vector quantization
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We consider the problem of learning a neural network classifier. Under the information bottleneck (IB) principle, we associate with this classification problem a representation learning problem, which we call "IB learning". We show that IB learning is, in fact, equivalent to a special class of the quantization problem. The classical results in rate-distortion theory then suggest that IB learning can benefit from a "vector quantization" approach, namely, simultaneously learning the representations of multiple input objects. Such an approach assisted with some variational techniques, result in a novel learning framework, "Aggregated Learning", for classification with neural network models. In this framework, several objects are jointly classified by a single neural network. The effectiveness of this framework is verified through extensive experiments on standard image recognition and text classification tasks.
ISSN:2331-8422