transformer 4 RuntimeError: Expected tensor for argument #1 ‘indices‘ to have scalar type Long

        在使用transformer 4.0时，报错误提示RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.IntTensor instead (while checking arguments for embedding)。该问题主要时由于tensor的类型导致的，解决方法是在相应报错行的前一行对数据类型进行转换。假设输入数据为x，那么增加行为“x = torch.tensor(x).to(torch.int64)”。

        如果修改之后仍然出现该错误，并且发生错误的位置发生变化，如下面述错误示例所示，那么逐一进行对应修改即可。

1 完整错误样例一

        完整错误提示为：  

File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\pipelines.py", line 1874, in __call__
    start, end = self.model(**fw_args)[:2]
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 1621, in forward
    return_dict=return_dict,
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 843, in forward
    input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 198, in forward
    inputs_embeds = self.word_embeddings(input_ids)
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\sparse.py", line 126, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\functional.py", line 1852, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.IntTensor instead (while checking arguments for embedding)

        在错误提示中，有如下一行：

“"D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 198, in forward inputs_embeds = self.word_embeddings(input_ids)”，

        那么只需要在这一行前面增加：

        input_ids = torch.tensor(input_ids).to(torch.int64)

2 完整错误样例二

        重新运行程序，这一行错误跳过，但仍有类似错误：

File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\pipelines.py", line 1874, in __call__
    start, end = self.model(**fw_args)[:2]
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 1622, in forward
    return_dict=return_dict,
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 844, in forward
    input_ids=input_ids, position_ids=position_ids, token_type_ids=token_type_ids, inputs_embeds=inputs_embeds
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 201, in forward
    token_type_embeddings = self.token_type_embeddings(token_type_ids)
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\sparse.py", line 126, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "D:\ProgramData\Anaconda3\lib\site-packages\torch\nn\functional.py", line 1852, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 'indices' to have scalar type Long; but got torch.IntTensor instead (while checking arguments for embedding)

        同样地，错误提示所在行为：

"D:\ProgramData\Anaconda3\lib\site-packages\transformers\models\bert\modeling_bert.py", line 201, in forward token_type_embeddings = self.token_type_embeddings(token_type_ids)。

        那么在这一行之前增加：

        token_type_ids = torch.tensor(token_type_ids).to(torch.int64)

3 解决方法

        综上所述，该问题主要时由于tensor的类型导致的，解决方法是在相应报错行的前一行对数据类型进行转换。假设输入数据为x，那么增加行为“x = torch.tensor(x).to(torch.int64)”。如果修改之后仍然出现该错误，并且发生错误的位置发生变化，如上述错误示例所示，那么逐一进行对应修改即可。