🐛 Bugs / Unexpected behaviors
When I try to use Pytorch3d as a differential operator to do some operation like the taubin_smoothing, if the vertices of the mesh require the grad, then the error ouccrs.
当我尝试使用Pytorch3d作为微分算子来进行一些操作,如taubin_smoothing时,如果网格的顶点需要梯度,那么错误就会出现。
Instructions To Reproduce the Issue 复现代码:
'''
ENV:
torch = 1.12.1+cu113
pytorch3d = 0.7.0 , I install it from https://anaconda.org/pytorch3d/pytorch3d/files
CUDA = 11.4
RTX 3080
'''
import torch
import numpy as np
import trimesh
import pytorch3d as p3d
from pytorch3d.structures import Meshes
from pytorch3d.ops import taubin_smoothing
def load_mesh_pml(mesh_file):
meshset = pml.MeshSet()
meshset.load_new_mesh(mesh_file)
return meshset
if __name__ == '__main__':
mesh_file = "tshirt.ply"
# 参数; taubin_smooth parameters
lam = 0.5
mu = -0.53
num_iter = 10
# 加载mesh, 获取基本信息; Load a trimesh
mesh = trimesh.load_mesh(mesh_file)
edges = mesh.edges
faces = np.asarray(mesh.faces)
verts = np.asarray(mesh.vertices)
# to tensor; Please note When .requires_grad_(True) is set, the bug occurs.
verts_t = torch.tensor(verts).cuda().requires_grad_(True)
faces_t = torch.tensor(faces).cuda()
mesh_t = Meshes(verts=[verts_t], faces=[faces_t])
lap_single = taubin_smoothing(mesh_t, num_iter=num_iter, lambd=lam, mu=mu).verts_list()[0]
BUG
Traceback (most recent call last):
File "xxxx/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3508, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-606074cc5014>", line 1, in <module>
runfile('/laplacian.py', wdir='')
File "/snap/pycharm-community/383/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/snap/pycharm-community/383/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/laplacian.py", line 47, in <module>
lap_single = taubin_smoothing(mesh_t, num_iter=num_iter, lambd=lam, mu=mu).verts_list()[0]
File "xxxx/lib/python3.8/site-packages/pytorch3d/ops/mesh_filtering.py", line 50, in taubin_smoothing
verts = (1 - lambd) * verts + lambd * torch.mm(L, verts) / total_weight
RuntimeError: The backward pass for this operation requires the 'self' tensor to be strided, but a sparse tensor was given instead. Please either use a strided tensor or set requires_grad=False for 'self'
解决方案
进入pytorch3d/ops/mesh_filtering.py taubin_smoothing
将
for _ in range(num_iter):
L = norm_laplacian(verts, edges)
total_weight = torch.sparse.sum(L, dim=1).to_dense().view(-1, 1)
verts = (1 - lambd) * verts + lambd * torch.mm(L, verts) / total_weight
L = norm_laplacian(verts, edges)
total_weight = torch.sparse.sum(L, dim=1).to_dense().view(-1, 1)
verts = (1 - mu) * verts + mu * torch.mm(L, verts) / total_weight
改为
for _ in range(num_iter):
L = norm_laplacian(verts, edges)
total_weight = torch.sparse.sum(L, dim=1)
total_weight = total_weight.unsqueeze(-1).to_dense()
verts = (1 - lambd) * verts + lambd * torch.sparse.mm(L, verts)
verts = verts/total_weight
L = norm_laplacian(verts, edges)
total_weight = torch.sparse.sum(L, dim=1)
total_weight = total_weight.unsqueeze(-1).to_dense()
verts = (1 - mu) * verts + mu * torch.sparse.mm(L, verts) / total_weight
Another Bug during Backward():
When I try to do the backward() as the following:
lap_single = lap_single.norm()
lap_single.backward()
It reports:
Traceback (most recent call last):
File "xxxx//IPython/core/interactiveshell.py", line 3508, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-606074cc5014>", line 1, in <module>
runfile('xxxx/laplacian.py', wdir='xxxx')
File "/snap/pycharm-community/383/plugins/python-ce/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/snap/pycharm-community/383/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "xxxx/laplacian.py", line 50, in <module>
lap_single.norm().sum().backward()
File "xxxx//torch/_tensor.py", line 396, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "xxxx//torch/autograd/__init__.py", line 173, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: Tensors of type SparseTensorImpl do not have strides
solution
I find that it is the total_weight causes this error.
So I have to give up the usage of taubin_smoothing.
I replace the taubin_smoothing with the original Laplacian smoothing.
Again, go to pytorch3d/ops/mesh_filtering.py taubin_smoothing.
ADD:
from pytorch3d.ops import laplacian
and Change the loop code in def taubin_smoothing:
for _ in range(num_iter):
L = laplacian(verts, edges)
verts = verts + torch.sparse.mm(L, verts)
Note that, the num_iter should set to 3 or 5, not to large.
【本人原创,同时在pytorch3d的github上也报告了该问题】