1-Dimensional Topological Invariants to Estimate Loss Surface Non-Convexity

We utilize the framework of topological data analysis to examine the geometry of loss landscape. With the use of topology and Morse theory, we propose to analyse 1-dimensional topological invariants as a measure of loss function non-convexity up to arbitrary re-parametrization. The proposed approach...

Full description

Saved in:
Bibliographic Details
Published in:Doklady. Mathematics 2023-12, Vol.108 (Suppl 2), p.S325-S332
Main Authors: Voronkova, D. S., Barannikov, S. A., Burnaev, E. V.
Format: Article
Language:English
Subjects:
Algebraic Topology
Algorithms
Artificial Intelligence
Computation and Language
Computational Geometry
Computer Science
Computer Vision and Pattern Recognition
Convexity
Data analysis
Dimensional analysis
Dynamical Systems
Invariants
Machine Learning
Mathematics
Mathematics and Statistics
Neural and Evolutionary Computing
Neural networks
Optimization
Parameterization
Statistics
Symplectic Geometry
Topology
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We utilize the framework of topological data analysis to examine the geometry of loss landscape. With the use of topology and Morse theory, we propose to analyse 1-dimensional topological invariants as a measure of loss function non-convexity up to arbitrary re-parametrization. The proposed approach uses optimization of 2-dimensional simplices in network weights space and allows to conduct both qualitative and quantitative evaluation of loss landscape to gain insights into behavior and optimization of neural networks. We provide geometrical interpretation of the topological invariants and describe the algorithm for their computation. We expect that the proposed approach can complement the existing tools for analysis of loss landscape and shed light on unresolved issues in the field of deep learning.
ISSN:1064-5624
1531-8362
DOI:10.1134/S1064562423701569