Neural Network with Smooth Activation Functions and without Bottlenecks Is Almost Surely a Morse Function

It is proved that a neural network with sigmoidal activation functions is a Morse function for almost all, with respect to the Lebesgue measure, sets of parameters (weights) in the case when the network architecture has no bottlenecks, i.e., layers with fewer neurons than in the adjacent layers. It ...

Full description

Saved in:
Bibliographic Details
Published in:Computational mathematics and mathematical physics 2021-07, Vol.61 (7), p.1162-1168
Main Author: Kurochkin, S. V.
Format: Article
Language:English
Subjects:
Bottlenecks
Computational Mathematics and Numerical Analysis
Computer architecture
Information Science
Mathematics
Mathematics and Statistics
Neural networks
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:It is proved that a neural network with sigmoidal activation functions is a Morse function for almost all, with respect to the Lebesgue measure, sets of parameters (weights) in the case when the network architecture has no bottlenecks, i.e., layers with fewer neurons than in the adjacent layers. It is shown by examples that the requirement for no bottlenecks is essential.
ISSN:0965-5425
1555-6662
DOI:10.1134/S0965542521070101