最近low-level vision的炼丹经常出现各种主观评测上的效果问题，无法定位出其对于输入数据的对应关系，出现了问题之后很难进行针对性解决。
这个时候一个很自然的问题就是，都2020年了，深度学习的可解释性到底发展到什么地步了？
对于模型的可解释性而言，很难做到像解数学题一样，每一步都能给出有效的解释。
于是就查阅了下模型解释性相关的论文，从2012年开始，主要还是以高引用作为主要的参考因素。
整个的相关论文查阅步骤大致为：
- 先从周博磊和张拳石两个大佬的主页去找相关的文献作为初始种子队列；
- 从初始种子队列里面去寻找相关参考文献，然后添加到种子队列中；
- 从github 资源awesome系列和pytorch模型解释性包captum中去寻找相关的论文添加到种子队列中；
- 按照时间顺序，从经典到最新，将种子队列中的论文在谷歌学术中进行搜索，然后从其引用中去继续寻找相关的论文，添加到种子队列中；
- 重复4，直至种子队列为空；
其中主要以高引用/顶会作为主要考虑因素，对ICCV和CVPR历年来的引用数统计结果来看，破100引用数的论文占比已经很少了，所以将其作为一个引用数高低的一个大致的衡量标准。
当然，查阅过程中难免会有错漏，因此也整理了一个awesome系列awesome_deep_learning_interpretabilit 来进行了版本的记录。
知乎不支持markdown格式的表格，前往github阅读会更为直观一些。
最后筛选出了161篇相关的论文，按照时间顺序和不同的出版物进行排序，结果进行了整理，结果如下。要是只想关注高引用的论文，也可参见github，也有按照引用次数的整理结果。
以下文献列表会不定期更新。
|Year|Publication|Paper|Citation|code|
|:---:|:---:|:---:|:---:|:---:|
|2020|ICLR|Knowledge Isomorphism between Neural Networks|0|
|2020|ICLR|Interpretable Complex-Valued Neural Networks for Privacy Protection|0|
|2019|AI|Explanation in artificial intelligence: Insights from the social sciences|380|
|2019|NMI|Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead|54|
|2019|NeurIPS|This looks like that: deep learning for interpretable image recognition|35|[Pytorch](cfchen-duke/ProtoPNet)|
|2019|NeurIPS|A benchmark for interpretability methods in deep neural networks（同arxiv:1806.10758）|3|
|2019|NeurIPS|Full-gradient representation for neural network visualization|2|
|2019|NeurIPS|On the (In) fidelity and Sensitivity of Explanations|2|
|2019|NeurIPS|Towards Automatic Concept-based Explanations|1|[Tensorflow](amiratag/ACE)|
|2019|NeurIPS|CXPlain: Causal explanations for model interpretation under uncertainty|1|
|2019|CVPR|Interpreting CNNs via Decision Trees|49|
|2019|CVPR|From Recognition to Cognition: Visual Commonsense Reasoning|44|[Pytorch](rowanz/r2c)|
|2019|CVPR|Attention branch network: Learning of attention mechanism for visual explanation|14|
|2019|CVPR|Interpretable and fine-grained visual explanations for convolutional neural networks|8|
|2019|CVPR|Learning to Explain with Complemental Examples|6|
|2019|CVPR|Revealing Scenes by Inverting Structure from Motion Reconstructions|5|[Tensorflow](francescopittaluga/invsfm)|
|2019|CVPR|Multimodal Explanations by Predicting Counterfactuality in Videos|1|
|2019|CVPR|Visualizing the Resilience of Deep Convolutional Network Interpretations|1|
|2019|ICCV|U-CAM: Visual Explanation using Uncertainty based Class Activation Maps|6|
|2019|ICCV|Towards Interpretable Face Recognition|6|
|2019|ICCV|Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded|5|
|2019|ICCV|Understanding Deep Networks via Extremal Perturbations and Smooth Masks|2|[Pytorch](facebookresearch/TorchRay)|
|2019|ICCV|Explaining Neural Networks Semantically and Quantitatively|1|
|2019|ICLR|Hierarchical interpretations for neural network predictions|15|[Pytorch](csinva/hierarchical-dnn-interpretations)|
|2019|ICLR|How Important Is a Neuron?|10|
|2019|ICLR|Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks|7|
|2018|ICML|Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples|47|[Pytorch](tech-srl/lstar_extraction)|
|2019|ICML|Towards A Deep and Unified Understanding of Deep Neural Models in NLP|4|[Pytorch](icml2019paper2428/Towards-A-Deep-and-Unified-Understanding-of-Deep-Neural-Models-in-NLP)|
|2019|ICAIS|Interpreting black box predictions using fisher kernels|7|
|2019|ACMFAT|Explaining explanations in AI|54|
|2019|AAAI|Interpretation of neural networks is fragile|63|[Tensorflow](amiratag/InterpretationFragility)|
|2019|AAAI|Classifier-agnostic saliency map extraction|4|
|2019|AAAI|Can You Explain That? Lucid Explanations Help Human-AI Collaborative Image Retrieval|0|
|2019|AAAIW|Unsupervised Learning of Neural Networks to Explain Neural Networks|9|
|2019|AAAIW|Network Transplanting|4|
|2019|CSUR|A Survey of Methods for Explaining Black Box Models|344|
|2019|JVCIR|Interpretable convolutional neural networks via feedforward design|16|[Keras](davidsonic/Interpretable_CNNs_via_Feedforward_Design)|
|2019|ExplainAI|The (Un)reliability of saliency methods(scihub)|95|
|2019|ACL|Attention is not Explanation|57|
|2019|arxiv|Attention Interpretability Across NLP Tasks|4|
|2019|arxiv|Interpretable CNNs|3|[Pytorch](oyzh888/ICNN)|
|2018|ICLR|Towards better understanding of gradient-based attribution methods for deep neural networks|123|
|2018|ICLR|Learning how to explain neural networks: PatternNet and PatternAttribution|90|
|2018|ICLR|On the importance of single directions for generalization|81|[Pytorch](1Konny/class_selectivity_index)|
|2018|ICLR|Detecting statistical interactions from neural network weights|30|[Pytorch](mtsang/neural-interaction-detection)|
|2018|ICLR|Interpretable counting for visual question answering|21|[Pytorch](sanyam5/irlc-vqa-counting)|
|2018|CVPR|Interpretable Convolutional Neural Networks|154|
|2018|CVPR|Tell me where to look: Guided attention inference network|81|[Chainer](alokwhitewolf/Guided-Attention-Inference-Network)|
|2018|CVPR|Multimodal Explanations: Justifying Decisions and Pointing to the Evidence|78|[Caffe](Seth-Park/MultimodalExplanations)|
|2018|CVPR|Transparency by design: Closing the gap between performance and interpretability in visual reasoning|54|[Pytorch](davidmascharka/tbd-nets)|
|2018|CVPR|Net2vec: Quantifying and explaining how concepts are encoded by filters in deep neural networks|39|
|2018|CVPR|What have we learned from deep representations for action recognition?|20|
|2018|CVPR|Learning to Act Properly: Predicting and Explaining Affordances from Images|17|
|2018|CVPR|Teaching Categories to Human Learners with Visual Explanations|13|[Pytorch](macaodha/explain_teach)|
|2018|CVPR|What do Deep Networks Like to See?|9|
|2018|CVPR|Interpret Neural Networks by Identifying Critical Data Routing Paths|5|[Tensorflow](lidongyue12138/CriticalPathPruning)|
|2018|ECCV|Deep clustering for unsupervised learning of visual features|167|[Pytorch](asanakoy/deep_clustering)|
|2018|ECCV|Explainable neural computation via stack neural module networks|40|[Tensorflow](ronghanghu/snmn)|
|2018|ECCV|Grounding visual explanations|38|
|2018|ECCV|Textual explanations for self-driving vehicles|30|
|2018|ECCV|Interpretable basis decomposition for visual explanation|26|[Pytorch](CSAILVision/IBD)|
|2018|ECCV|Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases|17|
|2018|ECCV|Vqa-e: Explaining, elaborating, and enhancing your answers for visual questions|12|
|2018|ECCV|Choose Your Neuron: Incorporating Domain Knowledge through Neuron-Importance|8|[Pytorch](ramprs/neuron-importance-zsl)|
|2018|ECCV|Diverse feature visualizations reveal invariances in early layers of deep neural networks|5|[Tensorflow](sacadena/diverse_feature_vis)|
|2018|ECCV|ExplainGAN: Model Explanation via Decision Boundary Crossing Transformations|0|
|2018|ICML|Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)|110|[Tensorflow](fursovia/tcav_nlp)|
|2018|ICML|Learning to explain: An information-theoretic perspective on model interpretation|72|
|2018|ACL|Did the Model Understand the Question?|34|[Tensorflow](pramodkaushik/acl18_results)|
|2018|FITEE|Visual interpretability for deep learning: a survey|140|
|2018|NeurIPS|Sanity Checks for Saliency Maps|122|
|2018|NeurIPS|Explanations based on the missing: Towards contrastive explanations with pertinent negatives|35|[Tensorflow](IBM/Contrastive-Explanation-Method)|
|2018|NeurIPS|Towards robust interpretability with self-explaining neural networks|27|[Pytorch](raj-shah/senn)|
|2018|NeurIPS|Attacks meet interpretability: Attribute-steered detection of adversarial samples|26|
|2018|NeurIPS Workshop|Interpretable Convolutional Filters with SincNet|17|
|2018|NeurIPS|DeepPINK: reproducible feature selection in deep neural networks|15|[Keras](younglululu/DeepPINK)|
|2018|NeurIPS|Representer point selection for explaining deep neural networks|11|[Tensorflow](chihkuanyeh/Representer_Point_Selection)|
|2018|AAAI|Anchors: High-precision model-agnostic explanations|200|
|2018|AAAI|Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients|112|[Tensorflow](dtak/adversarial-robustness-public)|
|2018|AAAI|Deep learning for case-based reasoning through prototypes: A neural network that explains its predictions|67|[Tensorflow](OscarcarLi/PrototypeDL)|
|2018|AAAI|Interpreting CNN Knowledge via an Explanatory Graph|54|[Matlab](zqs1022/explanatoryGraph)|
|2018|AAAI|Examining CNN Representations with respect to Dataset Bias|24|
|2018|WACV|Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks|85|
|2018|IJCV|Top-down neural attention by excitation backprop|256|
|2018|TPAMI|Interpreting deep visual representations via network dissection|56|
|2018|DSP|Methods for interpreting and understanding deep neural networks(scihub)|469|
|2018|Access|Peeking inside the black-box: A survey on Explainable Artificial Intelligence (XAI)|131|
|2018|JAIR|Learning Explanatory Rules from Noisy Data|90|[Tensorflow](ai-systems/DILP-Core)|
|2018|MIPRO|Explainable artificial intelligence: A survey|54|
|2018|AIES|Detecting Bias in Black-Box Models Using Transparent Model Distillation|27|
|2018|BMVC|Rise: Randomized input sampling for explanation of black-box models|30|
|2018|arxiv|Manipulating and measuring model interpretability|73|
|2018|arxiv|How convolutional neural network see the world-A survey of convolutional neural network visualization methods|27|
|2018|arxiv|Revisiting the importance of individual units in cnns via ablation|25|
|2018|arxiv|Computationally Efficient Measures of Internal Neuron Importance|1|
|2017|ICML|Understanding Black-box Predictions via Influence Functions|517|[Pytorch](nimarb/pytorch_influence_functions)|
|2017|ICML|Axiomatic attribution for deep networks|448|[Keras](hiranumn/IntegratedGradients)|
|2017|ICML|Learning Important Features Through Propagating Activation Differences|383|
|2017|ICLR|Visualizing deep neural network decisions: Prediction difference analysis|212|[Caffe](lmzintgraf/DeepVis-PredDiff)|
|2017|ICLR|Exploring LOTS in Deep Neural Networks|26|
|2017|NeurIPS|A Unified Approach to Interpreting Model Predictions|591|
|2017|NeurIPS|Real time image saliency for black box classifiers|111|[Pytorch](karanchahal/SaliencyMapper)|
|2017|NeurIPS|SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability|97|
|2017|CVPR|Mining Object Parts from CNNs via Active Question-Answering|15|
|2017|CVPR|Network dissection: Quantifying interpretability of deep visual representations|373|
|2017|CVPR|Improving Interpretability of Deep Neural Networks with Semantic Information|43|
|2017|CVPR|MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network|86|[Torch](zizhaozhang/mdnet-cvpr2017)|
|2017|CVPR|Interpretable 3d human action analysis with temporal convolutional networks|106|
|2017|CVPR|Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering|393|
|2017|CVPR|Knowing when to look: Adaptive attention via a visual sentinel for image captioning|458|[Torch](jiasenlu/AdaptiveAttention)|
|2017|ICCV|Grad-cam: Visual explanations from deep networks via gradient-based localization|1333|[Pytorch](leftthomas/GradCAM)|
|2017|ICCV|Interpretable Explanations of Black Boxes by Meaningful Perturbation|284|[Pytorch](jacobgil/pytorch-explain-black-box)|
|2017|ICCV|Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention|80|
|2017|ICCV|Understanding and comparing deep neural networks for age and gender classification|39|
|2017|ICCV|Learning to disambiguate by asking discriminative questions|10|
|2017|IJCAI|Right for the right reasons: Training differentiable models by constraining their explanations|102|
|2017|IJCAI|Understanding and improving convolutional neural networks via concatenated rectified linear units|35|[Caffe](chakkritte/CReLU)|
|2017|AAAI|Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning|26|[Matlab](zqs1022/partGraphForCNN)|
|2017|ACL|Visualizing and Understanding Neural Machine Translation|56|
|2017|EMNLP|A causal framework for explaining the predictions of black-box sequence-to-sequence models|64|
|2017|CVPRW|Looking under the hood: Deep neural network visualization to interpret whole-slide image analysis outcomes for colorectal polyps|14|
|2017|survey|Interpretability of deep learning models: a survey of results|49|
|2017|arxiv|SmoothGrad: removing noise by adding noise|212|
|2017|arxiv|Interpretable & explorable approximations of black box models|68|
|2017|arxiv|Distilling a neural network into a soft decision tree|126|[Pytorch](kimhc6028/soft-decision-tree)|
|2017|arxiv|Towards interpretable deep neural networks by leveraging adversarial examples|44|
|2017|arxiv|Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models|210|
|2017|arxiv|Contextual Explanation Networks|28|[Pytorch](alshedivat/cen)|
|2017|arxiv|Challenges for transparency|69|
|2017|ACMSOPP|Deepxplore: Automated whitebox testing of deep learning systems|302|
|2017|CEURW|What does explainable AI really mean? A new conceptualization of perspectives|64|
|2017|TVCG|ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models|113|
|2016|NeurIPS|Synthesizing the preferred inputs for neurons in neural networks via deep generator networks|251|[Caffe](Evolving-AI-Lab/synthesizing)|
|2016|NeurIPS|Understanding the effective receptive field in deep convolutional neural networks|310|
|2016|CVPR|Inverting Visual Representations with Convolutional Networks|266|
|2016|CVPR|Visualizing and Understanding Deep Texture Representations|83|
|2016|CVPR|Analyzing Classifiers: Fisher Vectors and Deep Neural Networks|82|
|2016|ECCV|Generating Visual Explanations|224|[Caffe](LisaAnne/ECCV2016)|
|2016|ECCV|Design of kernels in convolutional neural networks for image classification|11|
|2016|ICML|Understanding and improving convolutional neural networks via concatenated rectified linear units|216|
|2016|ICML|Visualizing and comparing AlexNet and VGG using deconvolutional layers|28|
|2016|EMNLP|Rationalizing Neural Predictions|247|[Pytorch](zhaopku/Rationale-Torch)|
|2016|IJCV|Visualizing deep convolutional neural networks using natural pre-images|216|[Matlab](aravindhm/nnpreimage)|
|2016|IJCV|Visualizing Object Detection Features|22|[Caffe](cvondrick/ihog)|
|2016|KDD|Why should i trust you?: Explaining the predictions of any classifier|2255|
|2016|TVCG|Visualizing the hidden activity of artificial neural networks|122|
|2016|TVCG|Towards better analysis of deep convolutional neural networks|184|
|2016|NAACL|Visualizing and understanding neural models in nlp|269|[Torch](jiweil/Visualizing-and-Understanding-Neural-Models-in-NLP)|
|2016|arxiv|Understanding neural networks through representation erasure|137|
|2016|arxiv|Grad-CAM: Why did you say that?|87|
|2016|arxiv|Investigating the influence of noise and distractors on the interpretation of neural networks|24|
|2016|arxiv|Attentive Explanations: Justifying Decisions and Pointing to the Evidence|41|
|2016|arxiv|The Mythos of Model Interpretability|951|
|2016|arxiv|Multifaceted feature visualization: Uncovering the different types of features learned by each neuron in deep neural networks|130|
|2015|ICLR|Striving for Simplicity: The All Convolutional Net|1762|[Pytorch](StefOe/all-conv-pytorch)|
|2015|CVPR|Understanding deep image representations by inverting them|929|[Matlab](aravindhm/deep-goggle)|
|2015|ICCV|Understanding deep features with computer-generated imagery|94|[Caffe](mathieuaubry/features_analysis)|
|2015|ICMLW|Understanding Neural Networks Through Deep Visualization|974|[Tensorflow](jiye-ML/Visualizing-and-Understanding-Convolutional-Networks)|
|2015|AAS|Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model|304|
|2014|ECCV|Visualizing and Understanding Convolutional Networks|8009|[Pytorch](huybery/VisualizingCNN)|
|2014|ICLR|Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps|2014|[Pytorch](huanghao-code/VisCNN_ICLR_2014_Saliency)|
新年新气象，新的2020年，努力的一年从第一天的模型可解释性综述开始。
扫一扫
1183
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
