GET3D 论文笔记

简介

主要相关阅读：

• 油管视频 https://www.youtube.com/watch?v=whXTP08XMYA
• yuannnn的笔记 https://zhuanlan.zhihu.com/p/568878981

其它相关阅读：

• https://blog.csdn.net/baibaidoudou/article/details/119278314 conditional unconditional的图像生成
• https://nv-tlabs.github.io/DIBRPlus/ DIBR++
• https://developer.nvidia.com/discover/ray-tracing 光线追踪
• https://blog.csdn.net/minstyrain/article/details/123858806 nerf

论文架构

整个论文遵从GAN的结构。其用到了两个论文成果，DMTet和Nvdiffrast。主要步骤如下：
1 模型会生成SDF格式的图片，同时生成其纹理。
2. 利用DMTet，把SDF转换为mesh。
3. 利用Nvdiffrast取得阴影和相机视角。
4. 用两个鉴别器，判断以上两种图是否为真实图片。

整个过程可微，因此可以反向传播，完成从1到4的端到端训练。

概念解释

mesh

mesh是由一个个三角形合成的多面体

nerf

Recently, rapid progress in neural volume rendering [45] and 2D Generative Adversarial Networks
(GANs) [34, 35, 33, 29, 52] has led to the rise of 3D-aware image synthesis [7, 57, 8, 49, 51, 25].
However, this line of work aims to synthesize multi-view consistent images using neural rendering in
the synthesis process and does not guarantee that meaningful 3D shapes can be generated. While a
mesh can potentially be obtained from the underlying neural field