Learning to Sample
2019 CVPR
摘要
原文 | 译文 |
---|---|
Processing large point clouds is a challenging task. Therefore, the data is often sampled to a size that can be processed more easily. | 处理大规模点云是一项具有挑战性的任务,因此,一般把点云下采样到数量较少的size,方便处理 |
The question is how to sample the data? A popular sampling technique is Farthest Point Sampling (FPS). However, FPS is agnostic to a downstream ap-plication (classification, retrieval, etc.). | 问题是如何对点云进行采样?目前最常用的方法是最远点采样(FPS),但是FPS对于应用的任务是不可知的(FPS是通用的一个采样技术,没有争对点云处理的不同任务做不同的适应) |
The underlying assumption seems to be that minimizing the farthest point distance, as done by FPS, is a good proxy to other objective functions. | FPS潜在的最远点假设似乎对于点云处理的目标函数指一种不错的proxy。 |
We show that it is better to learn how to sample. To do that, we propose a deep network to simplify 3D point clouds. | 本文提出学习如何采样,提出用网络来采样\简化三维点云 |
The network, termed S-NET, takes a point cloud and produces a smaller point cloud that is optimized for a particular task. | 我们称他S-Net,S-Net的输入时一个点云,输出一个更小的对特定任务优化后的点云 |
he simplified point cloud is not guaranteed to be a subset of the original point cloud. Therefore, we match it to a subset of the original points in a post-processing step. | 输出的点云可能不是输入点云的子集,所以我们提出一种匹配方法。 |
We contrast our approach with FPS by experimenting on two standard data sets and show significantly better results for a variety of applications. | 我们在2个数据集上,将本文的采样方法和FPS进行对比实验。 |
创新点
- A task-specific data-driven sampling approach for point clouds
- A Progressive sampling method that orders points according to their relevance for the task
S-Net
按照文中的描述,先把task network预训练好,用的是PointNet。然后用S-Net输入原始点云,输入指定点数的点云,把这个点云经过匹配后送入Task Network里,然后训练S-Net。
训练S-Net用到的loss包括:
(1)
L
f
(
G
,
P
)
=
1
∣
G
∣
∑
g
∈
G
m
i
n
∣
∣
g
−
p
∣
∣
2
2
L_f(G,P)=\frac{1}{|G|}\sum _{g\in G}min||g-p||_{2}^{2}
Lf(G,P)=∣G∣1∑g∈Gmin∣∣g−p∣∣22
对所有生成的点
g
g
g要和原始点云
P
P
P相似。
(2)
L
m
(
G
,
P
)
=
m
a
x
g
∈
G
m
i
n
p
∈
P
∣
∣
g
−
p
∣
∣
2
2
L_m(G,P)=max_{g \in G}min_{p \in P}||g-p||_{2}^{2}
Lm(G,P)=maxg∈Gminp∈P∣∣g−p∣∣22
对于最不相似的点要尽量相似。
(3)
L
b
(
G
,
P
)
=
1
∣
P
∣
∑
p
∈
P
m
i
n
g
∈
G
∣
∣
g
−
p
∣
∣
2
2
L_b(G,P)=\frac{1}{|P|}\sum _{p\in P}min_{g \in G}||g-p||_{2}^{2}
Lb(G,P)=∣P∣1∑p∈Pming∈G∣∣g−p∣∣22
希望生成的点不冲突,尽量分散在输入点云中。
这张图最能反应这篇文章的核心内容,S-Net的优势体现在当点数少的时候,S-Net可以将那些关键点取出,而FPS不行。
Matching
由于不能保证生成的点集
G
G
G 是原始点集
P
P
P的子集,所以需要加入一个匹配的步骤,作者用邻近点搜索方法。
这里要提一下原文中一句话
The matching process is only applied at inference time, as the final step of inference. During training, the generated points are processed by the task network as-is, since the matching is not differentiable and cannot propagate the task loss back to S-NET .
因为取点这一操作是不可逆不可导的,所以task loss是不能回传到S-Net的。之前考虑过从采样这一环节入手改进PointNet++,看到文章题目的时候以为作者也是做了这样的工作,其实不然。最大的问题和困难就在不能BP,如何设计可以学习的采样策略还是没有解决,sad。
ProgressiveNet
这部分是对S-Net的一个扩展,可以产生任意size的点云,作者页做了相关工作。
文献
从这篇文章里我发现了几篇不错有关点云的文章,待读
- [2018 ACCV] Flex-Convolution
- PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
- Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds
- [CVPR 2019] Modeling Point Clouds with Self-Attention and Gumbel Subset Sampling