[torch]create a new criterion(cross entropy)

http://blog.csdn.net/u010167269/article/details/51966427

Introduction

CrossentropyCriterion in torch

CrossEntropyCriterion and ClassNLLCriterion only support with hard targets.

For examples, in multi-class classification task,
suppose we have 4 classes: dog, cat, carrot, book.
For one image whose annotation is “cat”, the label will be $P=(0,1,0,0)$.
And the output of the model is $Q=(0.1,0.7,0.01,0.19)$
CrossEntropyCriterion(torch)
$loss=-log(Q_2)$
Even if the label is soft $P=(0.3,0.5,0.2,0.1)$,
CrossEntropyCriterion(torch) won’t use other classes’ label when calculating the loss.
MyCrossEntropyCriterion
In fact, the crossentropy should be $loss=-\sum_{i}{p_ilog(q_i)}=-p_1log(q_1)-p_2log(q_2)-...$

One more thing:
“DistKLDivCriterion” is the KL divergence(relative entropy),
−∑(pi)log(pi/qi)=∑[(pi)log(p