pointnet-cvpr2017-3D_point_machine_learning

Abstruct

we design a novel type of neural network that directly consumes point clouds called pointenet, provides a unified architecture for applications range from object classsfication, part segmentation, to semantic parsing.

introduction

• Since points clouds or meshs are not in a regular format, most researchers typically transform such data to regular 3D voxel grids or collections of images.
• we add a data-dependent spatial transformer network that attempts to canonicalize the data before the point process them.
• pointnet can approximate any set function that is continuous.
• others’ rearch: multiview CNN， spectral CNN, feature-based DNNs
• deeplearning on unordered sets:

related work

problem statements

Deep learning on Point sets

properites of point sets

• unordered: a network that consumes N 3D point sets needs to be invariant to N! pemutations of the input set in data feeding order.
• interaction among points: capture the local structures
• invariance under transformations.

structure

three key modules:

• the max-pooling layers:as a aymmetric function to aggregate information from all the points
• global information combination structure
• two joint alignment networks :align both input points and point features

symmetry function for unorder input

three strategies exist

• sort input into a canonical order
• treat the input as a sequence to train a RNN(augment the training data by all kinds of permutations)
• use a symmetric function to aggregate the information from each point.

local and clobal information aggregation

• conbine both global features and point features to semetric segmentation

Joint Alignment Network

• point cloud invariant
• because of the great difficulty of optimization caused by differernt dimensions between feature transform martix and spatial transformation matrix, we add regluarization term to our softmax training.(and why)

Theoretical Analysis

universal approximation

Experiments

alllications

3D object classification

semanntic segmentation in scence

Architecture Design Analysis

compared with Alternative Order-invariant Method

Effectiveness if Input and Feature Transformations

• regularization

Robustness Test

Visualizaing PointNet

下面补充一点最新的内容

补充文章

篇名为：Explaining the PointNet: What Has Been Learned Inside the PointNet?
19年的CVPR的work-shop的一篇文章,主要的内容是在探究Pointnet到底在激活些什么东西？为同济大学的一篇研究性文章。
主要贡献在：

• 可视化全局特征的维度，观察它所激活的哪一个结构
• 提出了一个可代替pointnet的网络结构C-pointnet。该网络作用是映射出前几层网络结构哪些在相应的起作用。在实验上表明，C-pointnet比pointnet更加具有可解释性