Model-Agnostic Methods - Partial Dependence Plot (PDP)&Individual Conditional Expectation (ICE)

一、作为模型代理方法的第一节，先介绍模型代理方法的思路

从world捕捉data，用data训练模型，再用可解释性方法来对模型的结果给出解释。把模型训练和模型解释分开，使得训练模型不再局限在拥有内在可解释性的模型范围内。

二、下面介绍第一种Partial Dependence Plot (PDP)，部分依赖图形。描述的是单个或两个feature对模型outcome的边际影响。

The partial function tells us for given value(s) of features S what the average marginal effect on the prediction is. In this formula, x(i)c are actual feature values from the dataset for the features in which we are not interested, and n is the number of instances in the dataset.

PDP的前提假设是C中的feature与S中的不相关。

对于分类问题，the partial dependence plot displays the probability for a certain class given different values for feature(s) in S. An easy way to deal with multiple classes is to draw one line or plot per class.对S中每个特征的不同值展示不同的概率值，对于多分类，按类别划分。

对于类别型变量, the partial dependence is very easy to calculate. For each of the categories, we get a PDP estimate by forcing all data instances to have the same category.尝试用每个类别覆盖所有instance，计算概率。

There are a number of R packages that implement PDPs. I used the iml package for the examples, but there is also pdp or DALEX. In Python you can use Skater.

这里使用Skater来演示PDP的使用，详情请看https://github.com/oracle/Skater，里面有examples供参考。

三、Individual Conditional Expectation (ICE)

The partial dependence plot for the average effect of a feature is a global method because it does not focus on specific instances, but on an overall average. The equivalent to a PDP for individual data instances is called individual conditional expectation (ICE) plot.PDP只描述模型feature的平均影响，而不关注单个实例的，于是有了ICE，每个instance一行。A PDP is the average of the lines of an ICE plot.。

每个实例对应一条曲线

改进的ICE plot有：

Centered ICE Plot：Sometimes it can be hard to tell whether the ICE curves differ between individuals because they start at different predictions. A simple solution is to center the curves at a certain point in the feature and display only the difference in the prediction to this point.中心化曲线

Derivative ICE Plot:Another way to make it visually easier to spot heterogeneity is to look at the individual derivatives of the prediction function with respect to a feature. The resulting plot is called the derivative ICE plot (d-ICE). The derivatives of a function (or curve) tell you whether changes occur and in which direction they occur.太复杂，不实用。

目前没有Python的包，有R的

ICE plots are implemented in the R packages iml (used for these examples), ICEbox, and pdp. Another R package that does something very similar to ICE is condvis.

值得注意的是PDP和ICE在特征存在相关性时存在很大问题。When features interact with each other in a prediction model, the prediction cannot be expressed as the sum of the feature effects, because the effect of one feature depends on the value of the other feature. Aristotle’s predicate “The whole is greater than the sum of its parts” applies in the presence of interactions.

 

四、Accumulated Local Effects (ALE) Plot

可以是为PDP的无偏替代且速度更快。

这里不作介绍了