Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning

Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning

Uday Kamath • John Liu

Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning

*1 Springer

Uday Kamath Ashburn VA, USA

John Liu Nashville TN, USA

ISBN 978-3-030-83355-8        ISBN 978-3-030-83356-5 (eBook)

Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning | SpringerLink

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021

This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

To my parents Krishna and Bharathi, my wife Pratibha, the kids Aaroh and Brandy, my family and friends for their support.

-Uday Kamath

To my wife Catherine and daughter Gabrielle Kaili-May and my parents for their encouragement and patience.

-John Liu

Foreword

The extraordinarily rapid integration of AI into many (if not most) aspects of life, entertainment, and business is transforming and disrupting the processes that power the flow of human experience. The pace of AI adoption everywhere is intense, accompanied by both immense benefits and immense risks. While many of the risks reside in the human, legal, social, and political dimensions of AI applications, the primary source of risk remains in the technical implementations of AI. Most importantly and significantly, the current technical risk landscape is populated by concerns around a taxonomy of AI issues: understandability, explainability, transparency, interpretability, trustworthiness, and more.

In this comprehensive book on explainable AI (XAI) by John Liu and Uday Kamath, we find a valuable and thorough handbook for the AI/ML community, including early learners, experienced practitioners, and researchers. The book covers the various dimensions of the AI technical risk taxonomy, including methods and applications. The result of the authors’ extensive work is an in-depth coverage of many XAI techniques, with real-world practical examples including code, libraries, algorithms, foundational mathematics, and thorough explanations (the essence of XAI).

The XAI techniques presented in-depth here range from traditional whitebox (explainable) models (e.g., regression, rule-based, graphs, network models) to advanced black-box models (e.g., neural networks). In the first case, XAI is addressed through statistical and visualization techniques, feature exploration and engineering, and exploratory data analysis. In the latter case, XAI is addressed through a remarkably powerful and rich set of methods, including feature sensitivity testing through dependence, perturbation, difference, and gradient analyses. Extra attention is given to three special cases for XAI: natural language processing (NLP), computer vision (CV), and time series.

The book aims (and succeeds) to bring the reader up to date with the most modern advances in the XAI field, while also giving significant coverage to the history and complete details of traditional methods. In all the examples, use cases, techniques, and applications, the consistent theme of the book is to provide a comprehensive overview of XAI as perhaps the most critical requirement in AI/ML both for now and in the future, from both technical and compliance perspectives. In that regard, the forward-looking discussions at the end of the book give us a glimpse of emerging XAI research areas that will advance our AI risk-compliance posture, including human-machine collaboration (as in assisted and augmented intelligence applications), causal ML, explainable security, responsible AI, and multidisciplinary research into explainable and interpretable intelligence technologies as they impact humanity.

I expect that this book will be an essential textbook, guidebook, reference book, and how-to book in XAI design discussions, operational implementations, risk and compliance monitoring applications, and essential conversations with technically informed AI end-users, stakeholders, and regulators.

Kirk Borne, Ph.D., Data Scientist, Astrophysicist, Top Influencer, and Chief

Science Officer at DataPrime.ai

Preface

Why This Book?

The field of explainable AI addresses one of the most significant shortcomings of machine learning and deep learning algorithms today: the interpretability of models. As algorithms become more powerful and make predictions with better accuracy, it becomes increasingly important to understand how and why a prediction is made. Without interpretability and explainability, it would be difficult for the users to trust the predictions of real-life applications of AI. Interpretable machine learning and explainability is of critical importance for the following reasons:

• •  We need interpretability to explain the model’s working from both the diagnosis and debugging perspective.

• • We need explanations for the end-user to explain the decisions made by the model and the rationale behind the decisions.

• • Most datasets or models have been shown to have biases, and investigating these biases is imperative for model deployment. Explainability is one way of uncovering these biases in the model.

• • Many industries such as finance and healthcare have legal requirements on transparency, trust, explainability, and faithfulness of models, thus making interpretability of models a prerequisite. In the European Union, some interpretations of the GDPR regulations claim that AI solutions must supply explanations for their conclusions. (Other interpretations say people need only be informed that automated processes are involved in decisions that may affect them.)

There is also a constant flux of new tools that fall in various categories such as application specific toolkits, visualization frameworks, and algorithm libraries. Python is currently the lingua-franca of data scientists and researchers to perform research in the area of interpretability and explainability. There are many libraries that have evolved in Python for interpretable machine learning and explainable AI in the last few years. We found a need for a single resource that gives concrete form to traditional as well as modern techniques in explainable AI through the use of existing tools and libraries for real-world applications. The work aims to be a comprehensive “go to” resource presenting the most important methods of explainability, insights to help put the techniques to use, and real-world examples with code for a hands-on experience.

This book offers its readers a collection of techniques and case studies that should serve as an accessible introduction for those just entering the field or as a handy guide for others ready to create explainable solutions in any domain. Here are some highlights:

• • Comprehensive coverage of XAI techniques as a ready reference to the architectures, algorithms with essential mathematical insights, and meaningful interpretations.

• •  Thorough discussion on exploratory data analysis, data visualization, feature engineering, and selection necessary from a pre-hoc perspective for wide varieties of datasets such as tabular, text, time series, and images.

• • Model validation and estimation techniques, visualization of model performance and selection for classification, regression, and clustering problems are discussed with examples.

• •  25+ interpretable machine learning techniques, or white-box techniques, ranging from traditional to modern, state-of-the-art.

• •  20+ post-hoc techniques, covering diverse areas such as visualization, feature importance, and example-based explanation techniques.

• • 20+ explainable deep learning techniques which can be used in a generic or architecture-specific way for model diagnosis and explanations.

• •  XAI techniques from traditional to advanced in different areas such as time series forecasting, natural language processing, and computer vision.

• • 20+ XAI tools and Python-based libraries bundled together in the context of real-world case studies and Google Colaboratory-based notebooks for the practitioners to get hands-on experience. The book’s primary purpose is to be the single resource that addresses the theory and the practical aspects of interpretability and explainability using the case studies with code, experiments, and analysis to support.

Who This Book Is For

This book is an ideal text for AI practitioners wishing to learn methods that enable them to interpret and explain model predictions. This book addresses a large market as the interpretability problem is significant in healthcare, finance, and many other industries. Current AI/ML researchers will also find this book helpful as they integrate explainability into their research and innovation.

As it is infeasible to cover every topic fully in a single book, we present the key concepts regarding explainable AI. In particular, we focus on the overlap of these areas, leveraging different frameworks and libraries to explore modern research and the application. This book is written to introduce interpretability and explainable techniques with an emphasis on application and practical experience.

What This Book Covers

This book takes an in-depth approach to present the fundamentals of explainable AI through mathematical theory and practical use cases. The content is split into four parts: (1) pre-hoc techniques, (2) intrinsic and interpretable methods, (3) modelagnostic methods, and (4) explainable deep learning methods. Finally, a chapter is dedicated for the survey of interpretable and explainable methods applied to time series, natural language processing, and computer vision.

A brief description of each chapter is given below.

• • In the Introduction to Interpretability and Explainability Chap. 1, we introduce the readers to the field of explainable AI by presenting a brief history, its goals, societal impact, types of explanations, and taxonomy. We provide the readers with different resources ranging from books to courses to aid the readers in their practical journey.

• • Pre-model Interpretability and Explainability Chap. 2 focuses on how to summarize, visualize, and explore the data using graphical and non-graphical techniques as well as provide insights into feature engineering. Since time series, natural language processing, and computer vision need special handling regarding data analysis, each of these topics is covered further in detail.

• • Model Visualization Techniques and Traditional Interpretable Algorithms Chap. 3 is a refresher of basic theories and practical concepts that are important in model validation, evaluation, and performance visualization for both supervised and unsupervised learning. Traditional interpretable algorithms such as linear regression, logistic regression, generalized linear models, generalized additive models, Bayesian techniques, decision trees, and rule inductions are discussed from an interpretability perspective with examples.

• • Model Interpretability: Advances in Interpretable Machine Learning Algorithms Chap. 4 guides the readers to the latest advances made in the area of interpretable algorithms in the last few years overcoming various computational challenges while retaining the advantage of being transparent. The chapter covers most of the glass-box-based methods, decision tree-based techniques, rule-based algorithms, and risk-scoring systems successfully adopted in research and real-world scenarios.

• • Post-hoc Interpretability and Explanations Chap. 5 covers a vast collection of explainable methods created to specifically address the black-box model problem. The chapter organizes the post-hoc methods into three categories: visual explanations-based, feature-importance-based, and examples-based techniques. Each technique in the category is not only summarized but also implemented on real-world dataset to give a practical view of the explanations.

• • Explainable Deep Learning Chap. 6 presents a collection of explanation approaches that are specifically developed for neural networks by leveraging architecture or learning method from the perspective of model validation, debugging, and exploration. Various intrinsic, perturbation, and gradient-based methods are covered in-depth with real examples and visualizations.

• • Explainability in Time Series Forecasting, Natural Language Processing and Computer Vision Chap. 7 discusses everything from traditional to modern techniques and various advances in the respective domains in terms of interpretability and explainability. In addition, each topic presents a case study to compare, contrast, and explore from the point-of-view of a real-world practitioner.

• • XAI: Challenges and Future Chap. 8 highlights the paramount importance of formalizing, quantifying, measuring, and comparing different explanation techniques as well as some of the recent work in the area. Finally, we present some essential topics that need to be addressed and directions that will change XAI in the immediate future.

Next, we want to list topics we will not cover in this book. The book does not cover topics related to ethics, bias, and fairness and their relationships with XAI from a data and modeling perspective. XAI can both be hacked and also be used for hacking. XAI and its implications to security are not covered in this work. Causal interpretability is gaining popularity among researchers and practitioners to address the “why” part of decisions. Since this is a relatively new and evolving area, we have not covered causal machine learning from an interpretability viewpoint.

Ashburn, VA, USA

Uday Kamath

John Liu

Nashville, TN, USA

Acknowledgments

The construction of this book would not have been possible without the tremendous efforts of many people. Firstly, we want to thank Springer, especially our editor, Paul Drougas, and coordinator Shina Harshvardhan, for working very closely with us and see this to fruition. We would specifically like to first thank (alphabetical order) Gabrielle Kaili-May Liu (Junior, MIT, Cambridge), Mitch Naylor (Senior Data Scientist, Smarsh, Nashville), and Vedant Vajre (Senior, Stone Bridge High School, Ashburn) for their help in explainable AI libraries integration and validating experiments for many chapters described in the book. We would also like to thank (alphabetical order) Krishna Choppella (Solutions Architect, BAE Systems, Toronto), Bruce Glassford (Sr. Data Scientist, Smarsh, New York), Kevin Keenan (Sr.Director, Smarsh, Belfast), Joe Porter (Researcher, Nashville), and Prerna Subramanian (PhD Scholar, Queens University, Canada) for providing support and expertise in case studies, chapter reviews, and content feedback. We would also like to thank Dr. Kirk Borne, Anusha Dandapani, and Dr. Andrey Sharapov for graciously accepting our proposal to formally review the book and provide their perspectives as a foreword and reviews.

Contents

• 1  Introduction to Interpretability and Explainability

• 1.1   Black-Box problem

• 1.2   Goals

• 1.3   Brief History

• 1.3.1   Porphyrian Tree

• 1.3.2   Expert Systems

• 1.3.3   Case-Based Reasoning

• 1.3.4   Bayesian Networks

• 1.3.5   Neural Networks

• 1.4   Purpose

• 1.5   Societal Impact

• 1.6   Types of Explanations

• 1.7   Trade-offs

• 1.8   Taxonomy

• 1.8.1   Scope

• 1.8.2   Stage

• 1.9   Flowchart for Interpretable and Explainable Techniques

• 1.10  Resources for Researchers and Practitioners

• 1.10.1  Books

• 1.10.2  Relevant University Courses and Classes

• 1.10.3  Online Resources

• 1.10.4  Survey Papers

• 1.11  Book Layout and Details

• 1.11.1  Structure: Explainable Algorithm

References

• 2  Pre-model Interpretability and Explainability

• 2.1   Data Science Process and EDA

• 2.2   Exploratory Data Analysis

• 2.2.1   EDA Challenges for Explainability

• 2.2.2   EDA: Taxonomy

• 2.2.3   Role of EDA in Explainability

• 2.2.4   Non-graphical: Summary Statistics and Analysis..........

• 2.2.5   Graphical: Univariate and Multivariate Analysis

• 2.2.6   EDA and Time Series

• 2.2.7  EDA and NLP

• 2.2.8   EDA and Computer Vision

• 2.3 Feature Engineering

• 2.3.1   Feature Engineering and Explainability

• 2.3.2   Feature Engineering Taxonomy and Tools

References

• 3 Model Visualization Techniques and Traditional Interpretable

Algorithms

• 3.1   Model Validation, Evaluation, and Hyperparameters

• 3.1.1   Tools and Libraries

• 3.2   Model Selection and Visualization

• 3.2.1   Validation Curve

• 3.2.2   Learning Curve

• 3.3   Classification Model Visualization

• 3.3.1   Confusion Matrix and Classification Report

• 3.3.2   ROC and AUC

• 3.3.3   PRC

• 3.3.4   Discrimination Thresholds

• 3.4   Regression Model Visualization

• 3.4.1   Residual Plots

• 3.4.2   Prediction Error Plots

• 3.4.3   Alpha Selection Plots

• 3.4.4   Cook’s Distance

• 3.5   Clustering Model Visualization

• 3.5.1   Elbow Method

• 3.5.2   Silhouette Coefficient Visualizer

• 3.5.3   Intercluster Distance Maps

• 3.6   Interpretable Machine Learning Properties

• 3.7   Traditional Interpretable Algorithms

• 3.7.1   Tools and Libraries

• 3.7.2   Linear Regression

• 3.7.3   Logistic Regression

• 3.7.4   Generalized Linear Models

• 3.7.5   Generalized Additive Models

• 3.7.6   Naive Bayes

• 3.7.7   Bayesian Networks

• 3.7.8   Decision Trees

• 3.7.9   Rule Induction

References

• 4  Model Interpretability: Advances in Interpretable Machine Learning 121

• 4.1   Interpretable vs. Explainable Algorithms

• 4.2   Tools and Libraries

• 4.3   Ensemble-Based

• 4.3.1   Boosted Rulesets

• 4.3.2   Explainable Boosting Machines (EBM)

• 4.3.3   RuleFit

• 4.3.4   Skope-Rules

• 4.3.5   Iterative Random Forests (iRF)

• 4.4   Decision Tree-Based

• 4.4.1   Optimal Classification Trees

• 4.4.2   Optimal Decision Trees

• 4.5   Rule-Based Techniques

• 4.5.1   Bayesian Or’s of And’s (BOA)

• 4.5.2   Bayesian Case Model

• 4.5.3   Certifiably Optimal RulE ListS (CORELS)

• 4.5.4   Bayesian Rule Lists

• 4.6   Scoring System

• 4.6.1   Supersparse Linear Integer Models

References

• 5  Post-Hoc Interpretability and Explanations

• 5.1   Tools and Libraries

• 5.2   Visual Explanation

• 5.2.1   Partial Dependence Plots

• 5.2.2   Individual Conditional Expectation Plots

• 5.2.3   Ceteris Paribus Plots

• 5.2.4   Accumulated Local Effects Plots

• 5.2.5   Breakdown Plots

• 5.2.6   Interaction Breakdown Plots

• 5.3   Feature Importance

• 5.3.1   Feature Interaction

• 5.3.2   Permutation Feature Importance

• 5.3.3   Ablations: Leave-One-Covariate-Out

• 5.3.4   Shapley Values

• 5.3.5   SHAP

• 5.3.6   KernelSHAP

• 5.3.7   Anchors

• 5.3.8   Global Surrogate

• 5.3.9   LIME

• 5.4   Example-Based

• 5.4.1   Contrastive Explanation

• 5.4.2   kNN

• 5.4.3   Trust Scores

• 5.4.4   Counterfactuals

• 5.4.5   Prototypes/Criticisms

• 5.4.6   Influential Instances

References

• 6  Explainable Deep Learning

• 6.1   Applications

• 6.2   Tools and Libraries

• 6.3    Intrinsic

• 6.3.1   Attention

• 6.3.2   Joint Training

• 6.4   Perturbation

• 6.4.1   LIME

• 6.4.2   Occlusion

• 6.4.3   RISE

• 6.4.4   Prediction Difference Analysis

• 6.4.5   Meaningful Perturbation

• 6.5   Gradient/Backpropagation

• 6.5.1   Activation Maximization

• 6.5.2   Class Model Visualization

• 6.5.3   Saliency Maps

• 6.5.4   DeepLIFT

• 6.5.5  DeepSHAP

• 6.5.6   Deconvolution

• 6.5.7   Guided Backpropagation

• 6.5.8   Integrated Gradients

• 6.5.9   Layer-Wise Relevance Propagation

• 6.5.10  Excitation Backpropagation

• 6.5.11  CAM

• 6.5.12  Gradient-Weighted CAM

• 6.5.13  Testing with Concept Activation Vectors

References

• 7  Explainability in Time Series Forecasting, Natural Language

Processing, and Computer Vision

• 7.1   Time Series Forecasting

• 7.1.1   Tools and Libraries

• 7.1.2   Model Validation and Evaluation

• 7.1.3   Model Metrics

• 7.1.4   Statistical Time Series Models

• 7.1.5   Prophet: Scalable and Interpretable Machine

Learning Approach

• 7.1.6   Deep Learning and Interpretable Time Series Forecasting . 275

• 7.2   Natural Language Processing

• 7.2.1    Explainability, Operationalization, and

Visualization Techniques

• 7.2.2   Explanation Quality Evaluation