TD3出现 Trying to backward through the graph a second time

错误提示调用了两次backward，但明明两个critic网络是独立的，为什么会出现这个问题呢

			q_v1 = self.Critic1_eval(bs, ba)
            td_error1 = F.mse_loss(q_target, q_v1)
            self.ctrain1.zero_grad()
            td_error1.backward()
            self.ctrain1.step()

            q_v2 = self.Critic2_eval(bs, ba)
            td_error2 = F.mse_loss(q_target, q_v2)
            self.ctrain2.zero_grad()
            td_error2.backward()
            self.ctrain2.step()

交换了两者的位置同样第二个出问题。
计算q target的时候要截断detach()