模型的可解释性：Interpretability versus explainability

原文：

Interpretability versus explainability - Model Explainability with AWS Artificial Intelligence and Machine Learning Solutions

For AI/ML methods, the terms interpretability and explainability are commonly interchangeable. It is important to distinguish the difference between explainability and interpretability to help organizations determine an AI/ML approach to meet their use case.

Interpretability — If a business wants high model transparency and wants to understand exactly why and how the model is generating predictions, they need to observe the inner mechanics of the AI/ML method. This leads to interpreting the model’s weights and features to determine the given output. This is interpretability.

For example, an economist may want to build a multi-variate regression model to predict an inflation rate, they can view the estimated parameters of the model’s variables to measure the expected output given different data examples. In this case, full transparency is given and the economist can answer the exact why and how of the model’s behavior.

However, high interpretability typically comes at the cost of performance, as seen in the following figure. If a company wants to achieve high performance but still wants to have a general understanding of the model behavior, model explainability starts to play a larger role.

Explainability — Explainability is how to take an ML model and explain the behavior in human terms. With complex models (for example, black boxes), you cannot fully understand how and why the inner mechanics impact the prediction. However, through model agnostic methods (for example, partial dependence plots, SHapley Additive exPlanations (SHAP) dependence plots, or surrogate models) you can discover meaning between input data attributions and model outputs, which enables you to explain the nature and behavior of the AI/ML model.

For example, a news media outlet uses a neural network to assign categories to different articles. The news outlet cannot to interpret the model in depth; however, they use a model agnostic approach to evaluate the input article data versus the model predictions. With this approach, they find that the model is assigning the Sports category to business articles that mention sport organizations. Although the news outlet did not use model interpretability, they were still able to derive an explainable answer to reveal the model’s behavior.

When starting a new AI/ML project, you need to consider whether interpretability is required. Model explainability can be used in any AI/ML use case, but if detailed transparency is required, then your AI/ML method selection becomes limited.

Interpretability versus performance trade-off given common ML algorithms

When datasets are large and the data is related to images or text, neural networks can meet the customer's AI/ML objective with high performance. In such cases, where complex methods are required to maximize performance, data scientists may focus on model explainability instead of interpretability.

When starting a new AI/ML project, address interpretability requirements by asking the following questions:

• Is interpretability a hard business requirement? If there are regulations or business requirements for complete model transparency, you need to select an interpretable model. This enables you to document how the inner mechanisms of the model impact the output and explain the model in human terms.

• Can my dataset be used on a simpler model? Start simple first. If you can meet the objective using an interpretable AI/ML method with full transparency, select that approach. In cases where your data consists of audio, image, or text data types, a more complex AI/ML method may be a better option. Although explaining black box models can be challenging at times, model agnostic methods can provide visibility to derive model explainability in human terms.

A model explainability assessment and communication with AI/ML practitioners (for example, data scientists) can help answer these questions and determine the best model explainability methods to meet your business objective.