下面是自己刷视频的过程中整理得到的作业答案。题目是2016春季学期发布的版本 (March - June, 2016)。这里只包含了需要推导的内容(万幸CSDN支持 LATEX ),代码实现稍后找个git上传上来。
Assignment1的答案一共被我分成了4部分,分别包含第1,2,3,4题。这部分包含第1题的答案。
1. Softmax (10 points)
(a). (5 points) Prove that softmax is invariant to constant offsets in the input, that is, for any input vector
x
and any constant
c
,
where x+c means adding the constant c to every dimension of
softmax(x)i=exi∑jexj(1)
Note: In practice, we make use of this property and choose c=−maxixi when computing softmax probabilities for numerical stability (i.e. subtracting its maximum element from all elements of x ).
解:
softmax(x+c)i=exi+c∑jexj+c=ecexiec∑jexj=exi∑jexj=softmax(x)
综上,命题得证
(b). 见代码,略