经典的lstm(Embedding)的例子

最新推荐文章于 2023-12-27 23:26:21 发布

原创

最新推荐文章于 2023-12-27 23:26:21 发布 · 1.3k 阅读

1 ·

CC 4.0 BY-SA版权

本文通过一个具体的示例展示了如何使用PyTorch中的嵌入层(embedding layer)，包括初始化、权重操作以及梯度计算等过程。文章还介绍了如何通过unsqueeze方法调整张量维度以实现特定的矩阵运算。

from torch.autograd import Variable as V
import torch as t
from torch import nn

n, d, m = 3, 5, 7
embedding = nn.Embedding(n, d, max_norm=True)
W = t.randn((m, d), requires_grad=True)
idx = t.tensor([1, 2])
a = embedding.weight.clone() @ W.t()  # weight must be cloned for this to be differentiable
b = embedding(idx) @ W.t()  # modifies weight in-place
out = (a.unsqueeze(0) + b.unsqueeze(1))
print(a.unsqueeze(0).shape, b.unsqueeze(1).shape)
# with t.no_grad():
print(a.unsqueeze(0), "\n", b.unsqueeze(1), "\n", (a.unsqueeze(0) + b.unsqueeze(1)).shape)
print( (a.unsqueeze(0) + b.unsqueeze(1)))
loss = out.sigmoid().prod()
loss.backward()

/home/wangbin/anaconda3/envs/deep_learning/bin/python3.7 /home/wangbin/anaconda3/envs/deep_learning/project/main.py
torch.Size([1, 3, 7]) torch.Size([2, 1, 7])
tensor([[[ 1.0560, -0.3363,  1.9043, -0.8517,  0.0666, -0.1867, -0.1422],
         [-1.0370, -0.8827,  0.1464, -0.4847, -0.0349, -0.