Is it possible to reverse-engineer AI algorithms to understand their process? Can we use a model agnostic method that will explain AI without looking into the code at all? The answer to both questions is yes, we can. One method is SHAP.
Explainable Artificial Intelligence will become one of the key services AI will provide. As we know it today, social media forms a representation of society, not a full-blown social model. Our opinions, consumer habits, browsing data, and location history constitute a formidable data source for AI models. The sum of all of the information about our daily activities is challenging to analyze
We publish reviews everywhere. We write reviews about books, movies, equipment, smartphones, cars, and sports—everything in our daily lives. In this chapter, we will analyze IMDb reviews of films. IMDb offers datasets of review information for commercial and non -commercial use.
We will thus focus on SHapley Additive exPlanations (SHAP), part of the Microsoft Azure Machine Learning model interpretability solution. We can use "interpret" or "explain" for explainable AI. Both terms mean that we are providing an explanation or an interpretation of a model. We are also reverse-engineering the process of an algorithm.
SHAP was derived from game theory. Lloyd Stowell Shapley gave his name to this game theory model in the 1950s. In game theory, each player decides to contribute to a coalition of players to produce a total value that will be superior to the sum of their individual values. The Shapley value is the marginal contribution of a given player. The goal is to find and explain the marginal contribution of each participant in a coalition of players.
This article goes through:
- - the key principles of SHAP
- - the mathematical representation of SHAP
- - a snippet in Python to illustrate the method
Key SHAP principles
In this section, we will learn about Shapley values through the principles of symmetry, null players, and additivity
Symmetry
If all of the players in a game have the same contribution, their contributions will be symmetrical. Suppose that, for a flight, the plane cannot take off without a pilot and a copilot. They both have the same contribution.
Null player
A null player does not affect the outcome of a model. Suppose we now consider a player in a basketball team. During this game, the player was supposed to play offense. However, when playing offense, this player's leading partner is suddenly absent. The player is both surprised and lost. For some reason, this player will contribute nothing to the result of that specific game.
Additivity
Our null basketball player was lost in the previous null player game in the previous section. The team manager realizes the talent of another player and adds this player to the team. The new player will add value to the team.
Mathematical representation
In a coalition game (N,v) , we need to find the payoff division . The payoff division is a unique fair distribution of the marginal contribution of each player to that player .
As such, the Shapley value will satisfy the null player, additivity, and symmetry/asymmetry properties we described in the previous sections.
We use words as players. Words are features in movie reviews, for example. Words [AS1] [DR2] are also players in game theory.
Words are sequence sensitive, which makes them an interesting way to illustrate Shapley values.
Consider the following three words in a set, a coalition N :
N = {excellent, not, bad}.
They