最近使用tensorflow训练程序,train时loss正常,test时总是NAN,最后发现是test的数据没有办法通过hashtable将字符串映射为正常的int(hashtable中没有test的数据标签,导致总是越界,输出默认值-1)所以变nan。
tf.contrib.lookup.index_table_from_tensor 使用详解
Sample Usages:
mapping_strings = tf.constant(["emerson", "lake", "palmer"])
table = tf.contrib.lookup.index_table_from_tensor(
mapping=mapping_strings, num_oov_buckets=1, default_value=-1)
features = tf.constant(["emerson", "lake", "and", "palmer"])
ids = table.lookup(features)
...
tf.tables_initializer().run()
ids.eval() ==> [0, 1, 4, 2]
Args:
mapping: A 1-D Tensor that specifies the mapping of keys to indices. The type of this object must be castable to dtype.
num_oov_buckets: The number of out-of-vocabulary buckets.
default_value: The value to use for out-of-vocabulary feature values. Defaults to -1.
hasher_spec: A HasherSpec to specify the hash function to use for assignment of out-of-vocabulary buckets.
dtype: The type of values passed to lookup. Only string and integers are supported.
name: A name for this op (optional).
Returns:
The lookup table to map an input Tensor to index int64 Tensor.