tf.nn.dropout使用参考

一、 embedding + dropout.

embedded = tf.nn.embedding_lookup(self.var, x)
if self.keep_prob < 1.:
	shape = embedded.get_shape().as_list()
	# Use same dropout masks at each timestep with specifying noise_shape.
    # This slightly improves performance.
    # Please see https://arxiv.org/abs/1512.05287 for the theoretical
    # explanation.
	embedded = tf.nn.dropout(embedded, 'self.keep_prob, noise_shape=(shape[0], 1, shape[2]))
    return embedded       

该代码, 在lookup后的embedded进行了drop_out ，并且使用了noise_shape, 据说是有提升的。

• noise_shape

By default, each element is kept or dropped independently.  If `noise_shape`
is specified, it must be [broadcastable](http://docs.scipy.org/doc/numpy/user/basics.broadcasting.html)
to the shape of `x`, and only dimensions with `noise_shape[i] == shape(x)[i]`
will make independent decisions.  For example, if `shape(x) = [k, l, m, n]`
and `noise_shape = [k, 1, 1, n]`, each batch and channel component will be
kept independently and each row and column will be kept or not kept together.