Geometry-Based Ensembles: Toward a Structural Characterization of the Classification Boundary

• Published in: IEEE transactions on pattern analysis and machine intelligence 2009-06, Vol.31 (6), p.1140-1146
• Main Authors: Pujol, O., Masip, D.
• Format: Article
• Language: English
• Subjects: Acoustic signal processing; Acoustics; Additives; Algorithmics. Computability. Computer arithmetics; Algorithms; Applied sciences; Artificial Intelligence; Assembly; Boundaries; Classification; Classifiers; Computational complexity; Computer science; control theory; systems; Computer Simulation; Computer vision; Distance learning; Exact sciences and technology; Fundamental areas of phenomenology (including applications); Large-scale systems; Learning; Machine learning; Models, Theoretical; Nonlinearity; On-line systems; Pattern recognition; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Physics; Piecewise linear techniques; Recognition; Robustness; Spatial databases; Studies; Theoretical computing; Ultrasonic imaging
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2009.31

This article introduces a novel binary discriminative learning technique based on the approximation of the non-linear decision boundary by a piece-wise linear smooth additive model. The decision border is geometrically defined by means of the characterizing boundary points - points that belong to the optimal boundary under a certain notion of robustness. Based on these points, a set of locally robust linear classifiers is defined and assembled by means of a Tikhonov regularized optimization procedure in an additive model to create a final lambda-smooth decision rule. As a result, a very simple and robust classifier with a strong geometrical meaning and non-linear behavior is obtained. The simplicity of the method allows its extension to cope with some of nowadays machine learning challenges, such as online learning, large scale learning or parallelization, with linear computational complexity. We validate our approach on the UCI database. Finally, we apply our technique in online and large scale scenarios, and in six real life computer vision and pattern recognition problems: gender recognition, intravascular ultrasound tissue classification, speed traffic sign detection, Chagas' disease severity detection, clef classification and action recognition using a 3D accelerometer data. The results are promising and this paper opens a line of research that deserves further attention.