3D GAUSSIAN AS A NEW VISION ERA: A SURVEY(1)

ABSTRACT

3DGS has emerged as a significant advancement in the field of Computer Graphics, offering explicit scene representation and novel view synthesis without the reliance on neural networks, such as Neural Radiance Fields (NeRF).

Figure

Figure 1

The optimization of 3DGS will be first introduced in terms of efficiency, realness, costs, and physics.

3DGS on reconstruction, manipulation, perception, generation, and human applications are comprehensively reviewed.

Figure 2

Taxonomy of existing 3DGS derived methods.

Figure 3

List of commonly used datasets for 3D Gaussians.

Figure 4

An illustration of optimizing 3DGS: (a) efficiency, (b) photorealism, (c) costs, and (d) physics. 

Efficiency

Challenges

Representing scenes with intricate details requires an enormous amount of 3D Gaussians. The tremendous storage space needed for Gaussians not only impedes its application on edge devices but also restrains the rendering speed.

Opportunities

Existing vector quantization and pruning of insignificant Gaussian methods have demonstrated their effectiveness in compressing 3D Gaussians for static scenes.

However, extending them to dynamic scenes and improving the compactness of dynamic representation are still underexplored.

Photorealism

Challenges

Although the projection of 3D Gaussians onto 2D image drastically accelerates the rendering process, it complicates the calculation of occlusion which leads to poor estimation of illumination.

In the meantime, the under-regularized 3DGS fails to capture precise geometry and cannot natively generate accurate normals.

Furthermore, the aliasing issue and artifacts