Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach

• Published in: IEEE transaction on neural networks and learning systems 2006-03, Vol.17 (2), p.374-384
• Main Authors: Etchells, T.A., Lisboa, P.J.G.
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Boolean algebra; Boolean functions; Breast cancer; Computer science; control theory; systems; Computer Simulation; Connectionism. Neural networks; Data mining; Data visualization; Decision analysis; Decision Support Techniques; Etching; Exact sciences and technology; Extraction; Formalism; Law; Legal factors; Mathematical analysis; Mathematical models; Methodology; Models, Theoretical; Neural networks; Neural Networks (Computer); Numerical Analysis, Computer-Assisted; Pattern Recognition, Automated - methods; rule extraction; Testing
• ISSN: 1045-9227; 2162-237X; 1941-0093; 2162-2388
• DOI: 10.1109/TNN.2005.863472

There is much interest in rule extraction from neural networks and a plethora of different methods have been proposed for this purpose. We discuss the merits of pedagogical and decompositional approaches to rule extraction from trained neural networks, and show that some currently used methods for binary data comply with a theoretical formalism for extraction of Boolean rules from continuously valued logic. This formalism is extended into a generic methodology for rule extraction from smooth decision surfaces fitted to discrete or quantized continuous variables independently of the analytical structure of the underlying model, and in a manner that is efficient even for high input dimensions. This methodology is then tested with Monks' data, for which exact rules are obtained and to Wisconsin's breast cancer data, where a small number of high-order rules are identified whose discriminatory performance can be directly visualized.