Abstract

We present a method for estimating articulated human pose from a single static image based on a graphical model with novel pairwise relations that make adaptive use of local image measurements. More precisely, we specify a graphical model for human pose which exploits the fact the local image measurements can be used both to detect parts (or joints) and also to predict the spatial relationships between them (Image Dependent Pairwise Relations). These spatial relationships are represented by a mixture model. We use Deep Convolutional Neural Networks (DCNNs) to learn conditional probabilities for the presence of parts and their spatial relationships within image patches. Hence our model combines the representational flexibility of graphical models with the efficiency and statistical power of DCNNs. Our method significantly outperforms the state of the art methods on the LSP and FLIC datasets and also performs very well on the Buffy dataset without any training.

NIPS 14 Paper PDF
PDF

@InProceedings{Chen_NIPS14,
  title        = {Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations},
  author       = {Xianjie Chen and Alan Yuille},
  booktitle    = {Advances in Neural Information Processing Systems (NIPS)},
  year         = {2014},
}


Key Ideas

1. Intuition: We can reliably predict the relative positions of a part's neighbors (as well as the presence of the part itself) by only observing the local image patch around it. Motivation
2. Deep Convolutional Neural Network is suitable to extract information about pairwise part relations, as well as part presence, from local image patches, which can be used in the unary and pairwise terms of the Graphical Model. Deep Convolutional Neural Network

Estimation Examples

Pose Estimation Examples

Performance

Comparison of strict PCP results on the Leeds Sport Pose (LSP) Dataset using Observer-Centric (OC) annotations.
Method Torso Head Upper Arms Lower Arms Upper Legs Lower Legs Mean
Ours 92.7 87.8 69.2 55.4 82.9 77.0 75.0
Pishchulin et al., ICCV'13 88.7 85.6 61.5 44.9 78.8 73.4 69.2
Ouyang et al., CVPR'14 85.8 83.1 63.3 46.6 76.5 72.2 68.6
Ramakrishna et al., ECCV'14 88.1 80.9 62.3 39.1 78.9 73.4 67.6
Eichner&Ferrari, ACCV'12 86.2 80.1 56.5 37.4 74.3 69.3 64.3
Pishchulin et al., CVPR'13 87.5 78.1 54.2 33.9 75.7 68.0 62.9
Yang&Ramanan, CVPR'11 84.1 77.1 52.5 35.9 69.5 65.6 60.8
Kiefel&Gehler, ECCV'14 84.4 78.4 53.3 27.4 74.4 67.1 60.7
Numbers are from the corresponding papers or errata.
Comparison of strict PCP results on the Leeds Sport Pose (LSP) Dataset using Person-Centric (PC) annotations. Note that both our method and Tompson et al., NIPS'14* include the Extended Leeds Sport Pose (ex_LSP) Dataset as training data.
Method Torso Head Upper Arms Lower Arms Upper Legs Lower Legs Mean
Ours* 96.0 85.6 69.7 58.1 77.2 72.2 73.6
Tompson et al., NIPS'14* 90.3 83.7 63.0 51.2 70.4 61.1 66.6
Pishchulin et al., ICCV'13 88.7 85.1 46.0 35.2 63.6 58.4 58.0
Wang&Li, CVPR'13 87.5 79.1 43.1 32.1 56.0 55.8 54.1
Numbers are from the performance evaluation by Pishchulin et al.
Comparison of strict PCP results on the Frames Labeled In Cinema (FLIC) Dataset using Observer-Centric (OC) annotations.
Method Upper Arms Lower Arms Mean
Ours 97.0 86.8 91.9
Tompson et al., NIPS'14 93.7 80.9 87.3
MODEC, CVPR'13 84.4 52.1 68.3
Numbers are from our evaluation using the prediction results released by the authors.
Comparison of PDJ curves of elbows and wrists on the Frames Labeled In Cinema (FLIC) Dataset using Observer-Centric (OC) annotations. The curves are for Tompson et al., NIPS'14, DeepPose, CVPR'14 and MODEC, CVPR'13.
FLIC PDJ curves

Figure Data: flic_elbows.fig | flic_wrists.fig


Cross-dataset PCP results on the Buffy Stickmen Dataset using Observer-Centric (OC) annotations.
Method Upper Arms Lower Arms Mean
Ours* 96.8 89.0 92.9
Ours* strict 94.5 84.1 89.3
Yang, PAMI'13 97.8 68.6 83.2
Yang, PAMI'13 strict 94.3 57.5 75.9
Sapp, ECCV'10 95.3 63.0 79.2
FLPM, ECCV'12 93.2 60.6 76.9
Eichner, IJCV'12 93.2 60.3 76.8
Numbers are from the corresponding papers.
Cross-dataset PDJ curves of elbows and wrists on the Buffy Stickmen Dataset using Observer-Centric (OC) annotations. Note that both our method and DeepPose are trained on the FLIC dataset. Compared with the curves on the FLIC dataset, the margin between our method and DeepPose significantly increases, which implies that our model generalizes better.
Buffy PDJ curves

Figure Data: cross_dataset_buffy_elbows.fig | cross_dataset_buffy_elbows.fig


Related Pages

Nice Performance Evaluation by Pishchulin et al.

Buffy Stickmen Dataset (Buffy)

Leeds Sports Pose Dataset (LSP)

Extended Leeds Sports Pose Dataset (ex_LSP)

Frames Labeled In Cinema Dataset (FLIC)