Reinforcement Explanation Learning
Siddhant Agarwal
Owais Iqbal
Sree Aditya Buridi
Mada Manjusha
Abir Das
NeurIPS 2021 workshop on explainableAI approaches for debugging and diagnosis


Deep Learning has become overly complicated and has enjoyed stellar success in solving several classical problems like image classification, object detection, etc. Several methods for explaining these decisions have been proposed. Black-box methods to generate saliency maps are particularly interesting due to the fact that they do not utilize the internals of the model to explain the decision. Most black-box methods perturb the input and observe the changes in the output. We formulate saliency map generation as a sequential search problem and leverage upon Reinforcement Learning (RL) to accumulate evidence from input images that most strongly support decisions made by a classifier. Such a strategy encourages to search intelligently for the perturbations that will lead to high-quality explanations. While successful black box explanation approaches need to rely on heavy computations and suffer from small sample approximation, the deterministic policy learned by our method makes it a lot more efficient during the inference. Experiments on three benchmark datasets demonstrate the superiority of the proposed approach in inference time over state-of-the-arts without hurting the performance.


Paper & Code

Siddhant Agarwal, Owais Iqbal, Sree Aditya Buridi, Mada Manjusha, Abir Das
Reinforcement Explanation Learning
1st Workshop on explainable AI approaches for debugging and diagnosis at
Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS)
, 2021
[PDF] [Supp] [Video Presentation] [Poster] [Slides] [Code]