Andrew Ng Deep Learning Tutorial中Softmax分类器梯度求导过程

Andrew Ng的deep learning tutorial 中 Softmax分类器求梯度的过程省略了，他直接给出

$$\Delta_{\theta^{(k)}}J(\theta) = -\sum_{i=1}^m x^{(i)}\bigl(\mathcal{I} (y^{(i)} = k) - \frac{exp({\theta^{(k)T}}x^{(i)}}{\sum_{j=1}^K exp(\theta^{(j)}x^{(i)})}\bigr) \tag{*}$$
但一眼无法看出结果,所以自己求了一下，几次算错。网上查的时候也有人很少有人推导或推错。折腾一下终于知道怎么推了。推导过程如下：
$$\begin{align} J(\theta) = - \left[ \sum_{i=1}^{m} \sum_{k=1}^{K} \mathcal{I}\left\{y^{(i)} = k\right\} \log \frac{\exp(\theta^{(k)\top} x^{(i)})}{\sum_{j=1}^K \exp(\theta^{(j)\top} x^{(i)})}\right] \end{align} \tag{1}$$
$$P(y^{(i)} = k | x^{(i)} ; \theta) = \frac{\exp(\theta^{(k)\top} x^{(i)})}{\sum_{j=1}^K \exp(\theta^{(j)\top} x^{(i)}) } \tag{2}$$
$$把(2)代入(1)得：\\ J(\theta) = -\sum_{i=1}^{m}\sum_{j=0}^K\mathcal{I}(y^{(i)} = j) \left[ \theta^{(j)}x^{(i)} - log \sum_{p=1}^{K}exp(\theta^{(p)T}x^{(i)})\right]$$
$$可以看出，根据j值的两种可能，所求得的导数不同。接下来分类讨论： I. 当j= k时：$$ $$J(\theta) = -\sum_{i=1}^{m}\left[ \theta^{(k)}x^{(i)} - log \sum_{p=1}^{K}exp(\theta^{(p)T}x^{(i)})\right]\\ \frac{\partial J(\theta)}{\partial \theta^{(k)}} =-\sum_{i=1}^{m}x^{(i)}\left(1-P(y^{(i)} = k | x^{(i)} ; \theta)\right) \tag{3}$$
$$II:当j\neq k时，为使过程清晰不妨设j = q，则\\ J(\theta) = -\sum_{i=1}^{m}\left[ \theta^{(q)}x^{(i)} - log \sum_{p=1}^{K}exp(\theta^{(p)T}x^{(i)})\right]$$ $$\frac{\partial J(\theta)}{\partial \theta^{(k)}} =-\sum_{i=1}^{m}x^{(i)}\left(-P(y^{(i)} = k | x^{(i)} ; \theta)\right) \tag{4}$$ .
综合式(3)(4)可得出结果式(*) 。附上andrew ng softmax回归的链接 http://ufldl.stanford.edu/tutorial/supervised/SoftmaxRegression/。