理解torch.nn.Embedding

torch.nn.Embedding(num_embeddings: int, embedding_dim: int, padding_idx: Optional[int] = None,
                 max_norm: Optional[float] = None, norm_type: float = 2., scale_grad_by_freq: bool = False,
                 sparse: bool = False, _weight: Optional[Tensor] = None)

参数：

"""
num_embeddings (int) – 词典的大小尺寸，比如总共出现2000个词，那就输入2000。此时index为（0-1999）
embedding_dim (int) – 嵌入向量的维度，即用多少维来表示一个符号。
padding_idx (int, optional) – 填充id，比如，输入长度为100，但是每次的句子长度并不一样，
                              后面就需要用统一的数字填充，而这里就是指定这个数字，这样，
                              网络在遇到填充id时，就不会计算其与其它符号的相关性。（初始化为0）
max_norm (float, optional) – 最大范数，如果嵌入向量的范数超过了这个界限，就要进行再归一化。
norm_type (float, optional) – 指定利用什么范数计算，并用于对比max_norm，默认为2范数。
scale_grad_by_freq (boolean, optional) – 根据单词在mini-batch中出现的频率，对梯度进行放缩。默认为False.
sparse (bool, optional) – 若为True,则与权重矩阵相关的梯度转变为稀疏张量
"""
"""
num_embeddings (int): size of the dictionary of embeddings
embedding_dim (int): the size of each embedding vector
padding_idx (int, optional): If given, pads the output with the embedding vector at :attr:`padding_idx`
                                 (initialized to zeros) whenever it encounters the index.
max_norm (float, optional): If given, each embedding vector with norm larger than :attr:`max_norm`
                            is renormalized to have norm :attr:`max_norm`.
norm_type (float, optional): The p of the p-norm to compute for the :attr:`max_norm` option. Default ``2``.
scale_grad_by_freq (boolean, optional): If given, this will scale gradients by the inverse of frequency of
                                        the words in the mini-batch. Default ``False``.
sparse (bool, optional): If ``True``, gradient w.r.t. :attr:`weight` matrix will be a sparse tensor.
                         See Notes for more details regarding sparse gradients.
"""

        其中较为常用的参数为num_embeddings, embedding_dim

     torch.nn.Embedding(num_embeddings,embedding_dim)的意思是创建一个词嵌入模型，num_embeddings代表词数量, embedding_dim代表为每个词创建一个多少维的向量来表示。

       用一个简单例子来解释：

import torch
import torch.nn as nn
from torch.autograd import Variable

word_index = {'hello': 0, 'what': 1,'are': 2, 'welcome': 3,'you': 4, 'i': 5}
# 首先创建6个词的词嵌套模型，每个词用10维的向量表示
embeds = nn.Embedding(6, 10)
print('embeds weight:',embeds.weight)

word = [[1, 2, 4],
        [4, 2, 3]]
# 表示两个句子,[what are you]和[you are welcome]
embed = embeds(torch.LongTensor(word))
print('embed:',embed)
print('embed size:',embed.size())

hello_idx = torch.LongTensor([word_index['hello']])
hello_idx = Variable(hello_idx)
hello_embed = embeds(hello_idx)
print('hello_embed :',hello_embed)
print('hello_embed size:',hello_embed.size())

输出：

embeds weight: Parameter containing:
tensor([[ 1.4825e+00,  3.0712e-01, -2.3293e-01, -4.1670e-01,  1.2044e+00,
         -2.9397e-01,  1.7059e+00, -1.1970e+00, -6.0050e-02, -5.3276e-02],
        [ 2.5928e-01,  4.3114e-04, -6.0456e-01,  1.1260e+00, -9.2605e-02,
          1.7866e+00,  1.9495e+00,  1.3268e-01,  3.4928e-01, -3.7590e+00],
        [-1.2764e+00, -8.0230e-01,  3.6574e-01, -1.1873e+00, -1.4793e+00,
         -1.9998e+00,  2.9790e+00, -2.0439e-01,  2.4888e+00, -1.6975e+00],
        [-1.7675e-01, -7.6663e-01, -1.3094e-01, -4.7818e-01, -1.4898e+00,
          1.0922e+00, -1.2935e-01, -1.8294e+00, -1.1626e+00,  1.2329e+00],
        [ 9.8236e-01,  1.2997e+00, -6.3932e-01,  1.1221e+00, -8.2662e-02,
         -3.9220e+00,  4.4613e-01, -4.9158e-01, -1.5463e+00, -2.4569e+00],
        [-3.3445e-01,  1.7175e+00,  6.3186e-01,  8.1554e-01, -8.2884e-01,
          5.4600e-01,  1.2782e-01,  8.5362e-01, -1.1766e+00,  2.5298e-01]],
       requires_grad=True)

embed: tensor([[[ 2.5928e-01,  4.3114e-04, -6.0456e-01,  1.1260e+00, -9.2605e-02,
           1.7866e+00,  1.9495e+00,  1.3268e-01,  3.4928e-01, -3.7590e+00],
         [-1.2764e+00, -8.0230e-01,  3.6574e-01, -1.1873e+00, -1.4793e+00,
          -1.9998e+00,  2.9790e+00, -2.0439e-01,  2.4888e+00, -1.6975e+00],
         [ 9.8236e-01,  1.2997e+00, -6.3932e-01,  1.1221e+00, -8.2662e-02,
          -3.9220e+00,  4.4613e-01, -4.9158e-01, -1.5463e+00, -2.4569e+00]],

        [[ 9.8236e-01,  1.2997e+00, -6.3932e-01,  1.1221e+00, -8.2662e-02,
          -3.9220e+00,  4.4613e-01, -4.9158e-01, -1.5463e+00, -2.4569e+00],
         [-1.2764e+00, -8.0230e-01,  3.6574e-01, -1.1873e+00, -1.4793e+00,
          -1.9998e+00,  2.9790e+00, -2.0439e-01,  2.4888e+00, -1.6975e+00],
         [-1.7675e-01, -7.6663e-01, -1.3094e-01, -4.7818e-01, -1.4898e+00,
           1.0922e+00, -1.2935e-01, -1.8294e+00, -1.1626e+00,  1.2329e+00]]],
       grad_fn=<EmbeddingBackward>)

embed size: torch.Size([2, 3, 10])

hello_embed : tensor([[ 1.4825,  0.3071, -0.2329, -0.4167,  1.2044, -0.2940,  1.7059, -1.1970,
         -0.0601, -0.0533]], grad_fn=<EmbeddingBackward>)

hello_embed size: torch.Size([1, 10])

        可以看到embed输出的size是[2,3,10],这表示输入维度为2x3的两个由三个单词组成的句子，其中的每个词由一个10维的向量来表示。单个单词“hello”由一个[1,10]的向量来表示。