Score-CAM++: Class Discriminative Localization with Feature Map Selection

Abstract In recent years, explaining convolutional neural networks (CNN) has received increasing attention since it helps to build humans’ trust on CNNs by exposing their inference basis. In this field, generating intuitive saliency maps that highlight the input regions most related to model decisions is one of the popular approaches. Building on a state-of-the-art saliency method named Score-CAM, we propose Score-CAM++ to generate better saliency maps with higher efficiency (when compared to Score-CAM). Different from Score-CAM, which adopts all the feature maps in target layer to produce saliency maps, we propose the “feature map selection” operation to select the feature maps that capture the “positive” information (i.e., the features whose increment in intensity leads to an increase in target score). Then the selected feature maps are up-sampled as masks to perturb the input and compute the weights of the corresponding feature maps. Finally, the linear combination of the weighted feature maps forms the saliency map. Compared with Score-CAM, the experiments based on VGG-16 show that our method saves 25%-30% of the time when generating saliency maps. Meanwhile, our conducted evaluations, both subjective and objective, show that our method provides better visual explanations when compared to the previous saliency methods.