虚拟现实光场技术

注意：本文摘取自上海科技大学虚拟现实中心教授虞晶怡的演讲，仅做摘录。

A Light Field Journey To Virtual Reality

报告人：虞晶怡(上海科技大学)

---

The Fish Eye Approach

Projecting$P[x, y, z]$to$[a, b]$

$a=x/z$
$b=y/z$
$r^2=a^2+b^2$
$\theta=atan(r)$

Fisheye distortion:

$$\theta_d=\theta(1+k_1\theta^2+k_2\theta^4+k_3\theta^6+k_4\theta^6)$$

Unidistort to $[x^{'}, y^{'}]$

$x^{'}=(\theta_d/r)x$
$y^{'}=(\theta_d/r)y$

Final pixel coordinate $[u, v]$

$u=f(x^{'}+ay^{'})$
$v=fy^{'}$

First 360 Attempt: QuickTime VR

Assumption: All cameras have an identical CoP!
1991 Eric Chen

Image Warping

Probabilistic model for image match verification

Image Quilting/Stitching

Stereo Parallax

Motion Parallax

360 Stereo Panomara

Near Stereo Pairs

Problem Dense Input

JUMP’s solution

Problems: Stereo Inconsistency
Incoherent Stitching

The Origin of Light Field (LF):

5D Plenoptic Function [Adelson ‘92]

Light Field Acquisitions:

Camera Arrays

Light Field Cameras

Days of Future Past

Light Field Rendering[Isakesen ‘00, Ng ‘05]

Problems With Light Field Rendering

Solution: Lumigraph

Question: How to Obtain Geometry?

Capability 1: Refocusing

refocusing via ray blending

Capability 2: Focal Symmetry

Depth-from-Defocus
Occlusion Detection

Capability 3: Dense Angular Sampling

Surface Cameras (SCam) Images

• Scam can be used to separate occluding vs. non occluding regions
• Can also used as a consistency metric

How About Using Multiple LFs

• Multi-View Light Field 3D Scanning

Now What? A Leap of Magic! Light Field HMD

Lenticular LF Displays
Compressive LF Displays: Magic Leap/Hololens

How Does It Work?
Solution: Tensor Decomposition

Coclusion: A Light Field Journey to VR