Purpose
**To formalize the impact of a training point on a
prediction, we ask the counterfactual: what would happen
if we did not have this training point, or if the values of this
training point were changed slightly?**
Main method
Answering this question by perturbing the data and retraining the model can be prohibitively expensive. To overcome this problem, we use influence functions, a classic technique from robust statistics (Cook & Weisberg, 1980) that tells us how the model parameters change as we upweight a training point by an infinitesimal amount.
Approach
1.Upweighting a training point
how would the model’s predictions change if we did not have this training point?
1.1删去一个样本后,模型参数的变化(参数增大了多少)
将某个样本z从训练集中删去后,模型新参数与原来参数相比,变化了:
其中:
如果把每个样本都去掉后各自训练一个新模型,再分别计算参数变化,显然需要很大的计算量。influence fucntions可以帮助我们解决这个问题。
首先,考虑将某一个样本z的权重增加一点点,计算新的模型的参数: