The Q -Norm Complexity Measure and the Minimum Gradient Method: A Novel Approach to the Machine Learning Structural Risk Minimization Problem

This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting obje...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on neural networks 2008-08, Vol.19 (8), p.1415-1430
Main Authors: Vieira, D.A.G., Takahashi, R.H.C., Palade, V., Vasconcelos, J.A., Caminhas, W.M.
Format: Article
Language:English
Subjects:
Algorithms
Applied sciences
Artificial Intelligence
Complexity measure
Computer science
control theory
systems
Computer Simulation
Exact sciences and technology
Gradient methods
Machine learning
Machine learning algorithms
Mathematics
Models, Theoretical
multiobjective training algorithms
Neural networks
Neural Networks (Computer)
parallel layer perceptron (PLP)
Pattern Recognition, Automated - methods
regularization methods
Risk management
Sliding mode control
structural risk minimization (SRM)
Supervised learning
Support vector machines
Virtual colonoscopy
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:This paper presents a novel approach for dealing with the structural risk minimization (SRM) applied to a general setting of the machine learning problem. The formulation is based on the fundamental concept that supervised learning is a bi-objective optimization problem in which two conflicting objectives should be minimized. The objectives are related to the empirical training error and the machine complexity. In this paper, one general Q-norm method to compute the machine complexity is presented, and, as a particular practical case, the minimum gradient method (MGM) is derived relying on the definition of the fat-shattering dimension. A practical mechanism for parallel layer perceptron (PLP) network training, involving only quasi-convex functions, is generated using the aforementioned definitions. Experimental results on 15 different benchmarks are presented, which show the potential of the proposed ideas.
ISSN:1045-9227
1941-0093
DOI:10.1109/TNN.2008.2000442