训练时，优化器Adam报错：ValueError: For primitive[Adam], the var_shape: [64,1024,] must be equal to [256,300,]

模型在反向传播过程中出错，Adam优化器输入尺寸不匹配

模型中对不同的params使用了不同的学习率

optimizer_G = nn.Adam( 
    params=[{'params': model.x0.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x1.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x2.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x3.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x4.trainable_params(), 'lr': dynamic_lr2}, 
            ], 
            learning_rate=config.lr*10 
) 

或许因此导致params不匹配，64,1024是x1和x2的参数维度，256,300是x4的参数维度

****************************************************解答*****************************************************

optimizer中将所有trainable_params分组设置lr后params顺序会改变，求梯度时weights的顺序不能直接使用model.trainable_params，需要和optimizer按照同一个顺序分组。

optimizer_G = nn.Adam( 
    params=[{'params': model.x0.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x1.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x2.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x3.trainable_params(), 'lr': dynamic_lr1}, 
            {'params': model.x4.trainable_params(), 'lr': dynamic_lr2}, 
            ], 
            learning_rate=config.lr*10 
)

如下为正确方式

self.weights = ParameterTuple(net.net.x0.trainable_params() +
                                       net.net.x1.trainable_params() +
                                       net.net.x2.trainable_params() +
                                       net.net.x3.trainable_params() +
                                       net.net.x4.trainable_params())

grads_g = self.grad(self.net, self.weights)(a, b)

如下为错误方式

self.weights = ParameterTuple(net.net.trainable_params())
grads_g = self.grad(self.net, self.weights)(a, b)