A Gaussian RBM with binary auxiliary units

Restricted Boltzmann Machines (RBM) have been widely applied in image processing. For RBM-based models on image recognition and image generation tasks, extracting expressive real-valued features and alleviating the overfitting problem are extremely important. In this paper, we propose a Gaussian Res...

Full description

Saved in:
Bibliographic Details
Published in:International journal of machine learning and cybernetics 2022-09, Vol.13 (9), p.2425-2433
Main Authors: Zhang, Jian, Ding, Shifei, Sun, Tongfeng, Guo, Lili
Format: Article
Language:English
Subjects:
Algorithms
Artificial Intelligence
Artificial neural networks
Complex Systems
Computational Intelligence
Control
Energy
Engineering
Feature selection
Image processing
Mechatronics
Neural networks
Normal distribution
Original Article
Pattern Recognition
Probability
Propagation
Random variables
Robotics
Systems Biology
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Restricted Boltzmann Machines (RBM) have been widely applied in image processing. For RBM-based models on image recognition and image generation tasks, extracting expressive real-valued features and alleviating the overfitting problem are extremely important. In this paper, we propose a Gaussian Restricted Boltzmann Machine with binary Auxiliary units (GARBM), which designs binary auxiliary units in its visible layer and constructs parameterized real-valued features in its hidden layer. Specifically, based on the designed energy function in GARBM, activated auxiliary units are directly used to control probabilities of visible units and hidden units to extract real-valued features. Moreover, auxiliary units and their resulting feature selection mechanism not only alleviate the “gradient-variance” problem, but also provide certain randomness to other units to alleviate overfitting without introducing more hyperparameters. To build more effective deep models, we propose GARBM-based deep neural networks, and the effectiveness of proposed neural networks is verified in experiments.
ISSN:1868-8071
1868-808X
DOI:10.1007/s13042-022-01534-6