1. 生成随机数
参考1
tf.random_normal
:从正态分布中输出随机值。tf.random_normal(shape,mean=0.0,stddev=1.0,dtype=tf.float32,seed=None,name=None)
tf.random_uniform
:从均匀分布中返回随机值random_uniform( shape,# 生成的张量的形状 minval=0, maxval=None, dtype=tf.float32, seed=None, name=None )
tf.truncated_normal
:截断的正态分布函数。生成的值遵循一个正态分布,但不会大于平均值2个标准差。truncated_normal( shape,#一个一维整数张量或Python数组。代表张量的形状。 mean=0.0,#数据类型为dtype的张量值或Python值。是正态分布的均值。 stddev=1.0,#数据类型为dtype的张量值或Python值。是正态分布的标准差 dtype=tf.float32,#输出的数据类型。 seed=None,#一个Python整数。是随机种子。 name=None#操作的名称(可选) )
2. 随机采样
参考2,tensorflow有几个函数可以实现候选采样:
- tf.nn.uniform_candidate_sampler:均匀采样
- tf.nn.log_uniform_candidate_sampler:按照log-uniform(Zipfian)分布采样,主要用作处理词作为类别的情况。==在语言学中,词按照出现频率从大到小排序之后,服从 Zipfian 分布。==在使用这个函数之前,需要对类别按照出现频率从大到小排序。
- tf.nn.learned_unigram_candidate_sampler:按照训练数据中类别出现分布进行采样。具体实现方式:
- 初始化一个 [0, range_max] 的数组, 数组元素初始为1;
- 在训练过程中碰到一个类别,就将相应数组元素加 1;
- 每次按照数组归一化得到的概率进行采样
- tf.nn.fixed_unigram_candidate_sampler3:按照自定义的(给定)概率分布进行采样。
- 小总结:
- 如果类别服从均匀分布,就用uniform_candidate_sampler;
- 如果词作类别,且词服从 Zipfian, 就用log_uniform_candidate_sampler;
- 如果能够通过统计或者其他渠道知道类别满足某些分布,或者自己指定分布,可以用 nn.fixed_unigram_candidate_sampler;
- 如果实在不知道类别分布,还可以用 tf.nn.learned_unigram_candidate_sampler根据数据学习分布
2.1 实例代码
参考4.
- 借助
tf.nn.fixed_unigram_candidate_sampler
:用参数unigrams
表示每个类的可能性权重,即分布。分布可以用整数表示import tensorflow as tf import numpy as np sess = tf.Session() V = tf.constant( np.array( [[ 10, 30, 20, 50 ]]), dtype=tf.int64) sampled_ids, true_expected_count, sampled_expected_count = tf.nn.fixed_unigram_candidate_sampler( true_classes = V, #待采样真实数据 num_true = 4, # 待采样真实数据类别总数 num_sampled = 50,#采样个数 unique = False, range_max = 4, unigrams = [ 20, 30, 10, 40 ] # this is P, times 100 ) # 返回的是采样的数据id,即索引 # 根据索引提取真实值 sample = tf.gather( V[ 0 ], sampled_ids ) x = sess.run( sample ) print( x )
- 使用
tf.nn.fixed_unigram_candidate_sampler
,但是分布用浮点数表示import tensorflow as tf import numpy as np sess = tf.Session() k = 50 # number of samples you want V = tf.constant( [ 10, 30, 20, 50 ], dtype = tf.float32 ) # values P = tf.constant( [ 0.2, 0.3, 0.1, 0.4 ], dtype = tf.float32 ) # prob dist cum_dist = tf.cumsum( P ) # create cumulative probability distribution # get random values between 0 and the max of cum_dist # we'll determine where it is in the cumulative distribution rand_unif = tf.random_uniform( shape=( k, ), minval = 0.0, maxval = tf.reduce_max( cum_dist ), dtype = tf.float32 ) # create boolean to signal where the random number is greater than the cum_dist # take advantage of broadcasting to create Cartesian product greater = tf.expand_dims( rand_unif, axis = -1 ) > tf.expand_dims( cum_dist, axis = 0 ) # we get the indices by counting how many are greater in any given row idxs = tf.reduce_sum( tf.cast( greater, dtype = tf.int64 ), 1 ) # then just gather the sample from V by the indices sample = tf.gather( V, idxs ) # run, output print( sess.run( sample ) )
- 借助
tf.distributions.Categorical()
5
以上代码可以用如下一行代码表示:# Probability distribution P = [0.2, 0.3, 0.1, 0.4] # Vector of values V = [10, 30, 20, 50] # Define categorical distribution dist = tf.distributions.Categorical(probs=P) # Generate a sample from categorical distribution - this serves as an index index = dist.sample().eval() # Fetch the value at V[index] as the sample sample = V[index]
sample = V[tf.distributions.Categorical(probs=P).sample().eval()]
如果想要从分布P中采样K个样本,则可用如下代码实现:
划掉的部分代码是低效的,更高效的方法是可以设置samples = [ V[tf.distributions.Categorical(probs=P).sample().eval()] for i in range(K) ]
sample
的参数sample_shape
,采样sample_shape
大小的样本:samples = dist.sample(sample_shape=[17,17]) #采样[17,17]大小的矩阵,每个元素都是从分布P中随机采样得到的
- 借助
tf.multinomial
采样,参考67elems = tf.convert_to_tensor([1,2,3,5]) samples = tf.multinomial(tf.log([[0.1, 0, 0.3, 0.6]]), 1) # note log-probability elems[tf.cast(samples[0][0], tf.int32)].eval() Out: 1 elems[tf.cast(samples[0][0], tf.int32)].eval() Out: 5
tf.random.multinomial( logits, # log-probabilities num_samples, #采样个数,必须是整数 seed=None, name=None, output_dtype=None )
3. 随机打乱数据
使用tf.random_shuffle
8来随机打乱数据。打乱第一维度(axis=0)的数据。
tf.random.shuffle(
value,
seed=None,
name=None
)
https://www.tensorflow.org/api_docs/python/tf/random/fixed_unigram_candidate_sampler ↩︎
https://stackoverflow.com/questions/49713210/how-to-sample-in-tensorflow-by-custom-probability-distribution ↩︎
https://www.tensorflow.org/api_docs/python/tf/distributions/Categorical ↩︎
https://www.tensorflow.org/api_docs/python/tf/random/multinomial ↩︎
https://www.tensorflow.org/api_docs/python/tf/random/shuffle ↩︎