姿态估计 Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations

Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations

Xianjie Chen and Alan Yuille

Abstract

We present a method for estimating articulated human pose from a single static image based on a graphical model with novel pairwise relations that make adaptive use of local image measurements. More precisely, we specify a graphical model for human pose which exploits the fact the local image measurements can be used both to detect parts (or joints) and also to predict the spatial relationships between them (Image Dependent Pairwise Relations). These spatial relationships are represented by a mixture model. We use Deep Convolutional Neural Networks (DCNNs) to learn conditional probabilities for the presence of parts and their spatial relationships within image patches. Hence our model combines the representational flexibility of graphical models with the efficiency and statistical power of DCNNs. Our method significantly outperforms the state of the art methods on the LSP and FLIC datasets and also performs very well on the Buffy dataset without any training.

 
Poster

 
PDF

 
Results & Evaluation Code

 
Full Code

 
Trained Model

@InProceedings{Chen_NIPS14,
  title        = {Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations},
  author       = {Xianjie Chen and Alan Yuille},
  booktitle    = {Advances in Neural Information Processing Systems (NIPS)},
  year         = {2014},
}

---

Key Ideas

1.  Intuition: We can reliably predict the relative positions of a part's neighbors (as well as the presence of the part itself) by  only observing the local image patch around it.

2. Deep Convolutional Neural Network is suitable to extract information about  pairwise part relations, as well as  part presence, from local image patches, which can be used in the unary and pairwise terms of the Graphical Model.

---

Estimation Examples

---

Performance

Comparison of  strict PCP results on the  Leeds Sport Pose (LSP) Dataset using  Observer-Centric (OC) annotations.

Numbers are from the corresponding papers or errata.

Method	Torso	Head	Upper Arms	Lower Arms	Upper Legs	Lower Legs	Mean
Ours	92.7	87.8	69.2	55.4	82.9	77.0	75.0
Pishchulin et al., ICCV'13	88.7	85.6	61.5	44.9	78.8	73.4	69.2
Ouyang et al., CVPR'14	85.8	83.1	63.3	46.6	76.5	72.2	68.6
Ramakrishna et al., ECCV'14	88.1	80.9	62.3	39.1	78.9	73.4	67.6
Eichner&Ferrari, ACCV'12	86.2	80.1	56.5	37.4	74.3	69.3	64.3
Pishchulin et al., CVPR'13	87.5	78.1	54.2	33.9	75.7	68.0	62.9
Yang&Ramanan, CVPR'11	84.1	77.1	52.5	35.9	69.5	65.6	60.8
Kiefel&Gehler, ECCV'14	84.4	78.4	53.3	27.4	74.4	67.1	60.7

Comparison of  strict PCP results on the  Leeds Sport Pose (LSP) Dataset using Person-Centric (PC) annotations. Note that both our method and  Tompson et al., NIPS'14* include the  Extended Leeds Sport Pose (ex_LSP) Dataset as training data.

Numbers are from the  performance evaluation by Pishchulin et al.

Method	Torso	Head	Upper Arms	Lower Arms	Upper Legs	Lower Legs	Mean
Ours*	96.0	85.6	69.7	58.1	77.2	72.2	73.6
Tompson et al., NIPS'14*	90.3	83.7	63.0	51.2	70.4	61.1	66.6
Pishchulin et al., ICCV'13	88.7	85.1	46.0	35.2	63.6	58.4	58.0
Wang&Li, CVPR'13	87.5	79.1	43.1	32.1	56.0	55.8	54.1

Comparison of  strict PCP results on the  Frames Labeled In Cinema (FLIC) Dataset using Observer-Centric (OC) annotations.

Numbers are from our evaluation using the prediction results released by the authors.

Method	Upper Arms	Lower Arms	Mean
Ours	97.0	86.8	91.9
Tompson et al., NIPS'14	93.7	80.9	87.3
MODEC, CVPR'13	84.4	52.1	68.3

Comparison of PDJ curves of elbows and wrists on the Frames Labeled In Cinema (FLIC) Datasetusing Observer-Centric (OC) annotations. The curves are for Tompson et al., NIPS'14, DeepPose, CVPR'14 and MODEC, CVPR'13.

Figure Data: flic_elbows.fig | flic_wrists.fig

Cross-dataset PCP results on the  Buffy Stickmen Dataset using Observer-Centric (OC) annotations.

Numbers are from the corresponding papers.

Method	Upper Arms	Lower Arms	Mean
Ours*	96.8	89.0	92.9
Ours* strict	94.5	84.1	89.3
Yang, PAMI'13	97.8	68.6	83.2
Yang, PAMI'13 strict	94.3	57.5	75.9
Sapp, ECCV'10	95.3	63.0	79.2
FLPM, ECCV'12	93.2	60.6	76.9
Eichner, IJCV'12	93.2	60.3	76.8

Cross-dataset PDJ curves of elbows and wrists on the Buffy Stickmen Dataset using Observer-Centric (OC) annotations. Note that both our method and DeepPose are trained on the FLIC dataset. Compared with the curves on the FLIC dataset, the margin between our method and DeepPose significantly increases, which implies that our model generalizes better.

Figure Data: cross_dataset_buffy_elbows.fig | cross_dataset_buffy_elbows.fig

---

Related Pages

Nice Performance Evaluation by Pishchulin et al.

Buffy Stickmen Dataset (Buffy)

Leeds Sports Pose Dataset (LSP)

Extended Leeds Sports Pose Dataset (ex_LSP)

Frames Labeled In Cinema Dataset (FLIC)

from: http://www.stat.ucla.edu/~xianjie.chen/projects/pose_estimation/pose_estimation.html