Deng Pan

Deng Pan

Postdoctoral Researcher at the University of Notre Dame specializing in Responsible AI. My research integrates interpretability, safety, fairness and robustness in both traditional machine learning and modern large language models, aiming to build transparent and accountable AI systems that advance responsible and trustworthy AI.

Explaining Deep Neural Network Models with Adversarial Gradient Integration

Published in Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI-21), 2021

Abstract: Deep neural networks (DNNs) have became one of the most high performing tools in a broad rangeof machine learning areas. However, the multilayer non-linearity of the network architectures preventus from gaining a better understanding of the models’ predictions. Gradient based attributionmethods (e.g., Integrated Gradient (IG)) that decipher input features’ contribution to the predictiontask have been shown to be highly effective yet requiring a reference input as the anchor for explainingmodel’s output. The performance of DNN model interpretation can be quite inconsistent withregard to the choice of references. Here we propose an Adversarial Gradient Integration (AGI) methodthat integrates the gradients from adversarial examples to the target example along the curve of steepestascent to calculate the resulting contributions from all input features. Our method doesn’t rely onthe choice of references, hence can avoid the ambiguity and inconsistency sourced from the referenceselection. We demonstrate the performance of our AGI method and compare with competing methodsin explaining image classification results. Code is available at GitHub.

Recommended citation: Pan, D., Li, X and Zhu, D. (2021). "Explaining Deep Neural Network Models with Adversarial Gradient Integration." The proceedings of 30th International Joint Conference on Artificial Intelligence (IJCAI-21), Montreal, Canada.
Download Paper