论文阅读 [TPAMI-2022] A General Differentiable Mesh Renderer for Image-Based 3D Reasoning

论文阅读 [TPAMI-2022] A General Differentiable Mesh Renderer for Image-Based 3D Reasoning

论文搜索(studyai.com)

搜索论文: A General Differentiable Mesh Renderer for Image-Based 3D Reasoning

搜索论文: http://www.studyai.com/search/whole-site/?q=A+General+Differentiable+Mesh+Renderer+for+Image-Based+3D+Reasoning

关键字(Keywords)

Three-dimensional displays; Rendering (computer graphics); Two dimensional displays; Standards; Cognition; Task analysis; Vision and scene understanding, modeling and recovery of physical attributes, perceptual reasoning; computer graphics, picture/image generat

机器学习; 机器视觉

无监督学习; 场景解析; 三维场景; 单目重建

摘要(Abstract)

Rendering bridges the gap between 2D vision and 3D scenes by simulating the physical process of image formation.

渲染通过模拟图像形成的物理过程，架起了2D视觉和3D场景之间的桥梁。.

By inverting such renderer, one can think of a learning approach to infer 3D information from 2D images.

通过反转这种渲染器，可以想到一种从2D图像推断3D信息的学习方法。.

However, standard graphics renderers involve a fundamental step called rasterization, which prevents rendering to be differentiable.

然而，标准图形渲染器涉及一个称为光栅化的基本步骤，这会阻止渲染的可微性。.

Unlike the state-of-the-art differentiable renderers (Kato et al.

与最先进的可微渲染器不同（Kato等人。.

2018 and Loper 2018), which only approximate the rendering gradient in the backpropagation, we propose a natually differentiable rendering framework that is able to (1) directly render colorized mesh using differentiable functions and (2) back-propagate efficient supervisions to mesh vertices and their attributes from various forms of image representations.

2018年和Loper 2018年），我们提出了一个自然可微的渲染框架，它能够（1）使用可微函数直接渲染彩色网格，以及（2）从各种形式的图像表示向网格顶点及其属性反向传播有效的监控。.

The key to our framework is a novel formulation that views rendering as an aggregation function that fuses the probabilistic contributions of all mesh triangles with respect to the rendered pixels.

我们的框架的关键是一个新的公式，它将渲染视为一个聚合函数，它融合了所有网格三角形相对于渲染像素的概率贡献。.

Such formulation enables our framework to flow gradients to the occluded and distant vertices, which cannot be achieved by the previous state-of-the-arts.

这样的公式使我们的框架能够将梯度流到被遮挡和远处的顶点，这是以前的技术无法实现的。.

We show that by using the proposed renderer, one can achieve significant improvement in 3D unsupervised single-view reconstruction both qualitatively and quantitatively.

我们表明，通过使用所提出的渲染器，可以在三维无监督单视图重建中实现定性和定量的显著改进。.

Experiments also demonstrate that our approach can handle the challenging tasks in image-based shape fitting, which remain nontrivial to existing differentiable renders…

实验还表明，我们的方法可以处理基于图像的形状拟合中的挑战性任务，这些任务对于现有的可微渲染来说并不重要。。.

作者(Authors)

[‘Shichen Liu’, ‘Tianye Li’, ‘Weikai Chen’, ‘Hao Li’]