词嵌入和交叉熵｜Word Embedding and Cross Entropy_crossentropyloss embedding-CSDN博客

本文链接：https://blog.csdn.net/qq_41103204/article/details/127564310

Word Embedding

The Embedding layer can be understood as a lookup table that maps from integer indices (which stand for specific words) to dense vectors (their embeddings).
The dimensionality (or width) of the embedding is a parameter you can experiment with to see what works well for your problem, much in the same way you would experiment with the number of neurons in a Dense layer.
When you create an Embedding layer, the weights for the embedding are randomly initialized (just like any other layer). During training, they are gradually adjusted via backpropagation. Once trained, the learned word embeddings will roughly encode similarities between words (as they were learned for the specific problem your model is trained on).

tf.keras.layers.Embedding(
    input_dim,
    output_dim,
    input_length=None,
    **kwargs
)

Embedding层有三个重要的参数

input_dim：输入维度，这个是指词典的大小
output_dim：输出维度，这个是指embedding的维度
input_length：这个只是输入序列（数据）的长度，比如长度为10的字符串
通常我们先处理好数据，然后调用 embedding 层，所以只需要定义 input_dim 和 output_dim，不需要指定input_length
The output vectors are not computed from the input using any mathematical operation. Instead, each input integer is used as the index to access a table that contains all possible vectors. That is the reason why you need to specify the size of the vocabulary as the first argument (so the table can be initialized)
Embedding Layer 可以理解成为一个 lookup table