Visual explanation of black-box model: Similarity Difference and Uniqueness (SIDU) method

• Published in: Pattern recognition 2022-07, Vol.127, p.108604, Article 108604
• Main Authors: Muddamsetty, Satya M., Jahromi, Mohammad N.S., Ciontos, Andreea E., Fenoy, Laura M., Moeslund, Thomas B.
• Format: Article
• Language: English
• Subjects: Adversarial attack; CNN; Explainable AI (XAI); Eye-tracker
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2022.108604

•A novel XAI explanation method denoted SIDU that can effectively localize entire object regions responsible for prediction in a full extend is introduced.•The method generates masks and computes similarity differences and uniqueness of the predictions score of mask and original image for explaining the model decision.•The method is evaluated on three different types evaluations namely Human-Grounded evaluation, Functionally Grounded and Application-Grounded evaluations to thoroughly assess SIDU.•For the human grounded evaluation, a framework for evaluating explainable AI (XAI) methods using an eye-tracker is introduced.•The robustness of the SIDU’s explanations analyzed in the presence of adversarial attacks. Explainable Artificial Intelligence (XAI) has in recent years become a well-suited framework to generate human understandable explanations of ‘black- box’ models. In this paper, a novel XAI visual explanation algorithm known as the Similarity Difference and Uniqueness (SIDU) method that can effectively localize entire object regions responsible for prediction is presented in full detail. The SIDU algorithm robustness and effectiveness is analyzed through various computational and human subject experiments. In particular, the SIDU algorithm is assessed using three different types of evaluations (Application, Human and Functionally-Grounded) to demonstrate its superior performance. The robustness of SIDU is further studied in the presence of adversarial attack on ’black-box’ models to better understand its performance. Our code is available at: https://github.com/satyamahesh84/SIDU_XAI_CODE.