Beyond Novelty Detection: Incongruent Events, When General and Specific Classifiers Disagree

• Published in: IEEE transactions on pattern analysis and machine intelligence 2012-10, Vol.34 (10), p.1886-1901
• Main Authors: Weinshall, Daphna, Zweig, Alon, Hermansky, Hynek, Kombrink, Stefan, Ohl, Frank W., Anemüller, Jörn, Bach, Jörg-Hendrik, Van Gool, Luc, Nater, Fabian, Pajdla, Tomas, Havlena, Michal, Pavel, Misha
• Format: Article
• Language: English
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; categorization; Computer science; control theory; systems; Data models; Electronic mail; Exact sciences and technology; Humans; Image Processing, Computer-Assisted - methods; Information, signal and communications theory; Models, Statistical; Motion; Novelty detection; object recognition; out-of-vocabulary words; Pattern Recognition, Automated - methods; Pattern recognition. Digital image processing. Computational geometry; Probabilistic logic; Signal processing; Signal Processing, Computer-Assisted; Speech and sound recognition and synthesis. Linguistics; Speech processing; Speech Recognition Software; Telecommunications and information theory; Training; Training data; Visualization; Vocabulary
• ISSN: 0162-8828; 1939-3539; 2160-9292
• DOI: 10.1109/TPAMI.2011.279

Unexpected stimuli are a challenge to any machine learning algorithm. Here, we identify distinct types of unexpected events when general-level and specific-level classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: Starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels in the label hierarchy. An incongruent event is an event where the probability computed based on some more specific level is much smaller than the probability computed based on some more general level, leading to conflicting predictions. Algorithms are derived to detect incongruent events from different types of hierarchies, different applications, and a variety of data types. We present promising results for the detection of novel visual and audio objects, and new patterns of motion in video. We also discuss the detection of Out-Of-Vocabulary words in speech recognition, and the detection of incongruent events in a multimodal audiovisual scenario.