Rule mining and classification in imperfect databases

• Main Authors: Hewawasam, K.K.R.G.K., Premaratne, K., Subasingha, S.P., Shyu, M.-L.
• Format: Conference Proceeding
• Language: English
• Subjects: Airports; Application software; Association rules; classification; data ambiguities; Data imperfections; Data mining; Data structures; Dempster-Shafer belief theory; Interpolation; Itemsets; Relational databases; Testing; Training data
• DOI: 10.1109/ICIF.2005.1591917

A rule-based classifier learns rules from a set of training data instances with assigned class labels and then uses those rules to assign a class label for a new incoming data instance. To accommodate data imperfections, a probabilistic relational model would represent the attributes by probabilistic functions. One extension to this model uses belief functions instead. Such an approach can represent a wider range of data imperfections. However, the task of extracting frequent patterns and rules from such a "belief theoretic" relational database has to overcome a potentially enormous computational burden. In this work, we present a data structure that is an alternate representation of a belief theoretic relational database. We then develop efficient algorithms to query for belief of item sets, extract frequent item sets and generate corresponding association rules from this representation. This set of rules is then used as the basis on which an unknown data instance, whose attributes are represented via belief functions, is classified. These algorithms are tested on a data set collected from a test bed that mimics airport threat detection and classification scenario where both data attributes and threat class labels may possess imperfections.