SMPL in more detail
Optional · supplementary note 5 of 5 for DLCV 05 · Human Pose and Keypoint Estimation. Nothing in the main text depends on it; citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
This note explains the terms of the SMPL body model [39] summarized in the main text.
Template and skinning. SMPL starts from an artist-made template mesh with vertices in a rest pose, a skeleton of joints plus a root, and blend weights that say how strongly each joint’s rotation moves each vertex. Linear blend skinning moves every vertex by a weighted average of the rigid transformations of the joints that influence it. On its own, blend skinning produces well-known artifacts, such as collapsing elbows and shoulders when joints bend strongly.
Shape blend shapes. adds a linear combination of orthonormal principal components of body-shape variation, learned from registered scans of many people. A few coefficients already capture most of the variation in height, weight and proportions.
Pose blend shapes. adds vertex offsets that depend linearly on the elements of the joints’ rotation matrices, computed relative to the rest pose. These offsets correct the blend-skinning artifacts and model pose-dependent deformations. Because the rest-pose rotations are subtracted, is exactly zero in the rest pose [39], which keeps shape and pose effects separate.
Joints from shape. People with different shapes have joints in different places, so SMPL computes the rest-pose joint locations from the shaped mesh with a learned linear regressor : [39]. After posing, the 3D joints (and any keypoints defined as regressed from vertices) follow the mesh, so they always respect the person’s bone lengths.
Why this helps pose estimation. A network that predicts can only produce bodies the model can express, and the 72 pose numbers are rotations rather than free 3D positions. Projecting the model’s joints into the image with an estimated camera gives 2D keypoints that can be compared with 2D labels, which is the reprojection loss HMR [40] uses to train on images without 3D ground truth.
References
- A. Mathis, P. Mamidanna, K. M. Cury, T. Abe, V. N. Murthy, M. W. Mathis and M. Bethge, “DeepLabCut: Markerless pose estimation of user-defined body parts with deep learning,” Nature Neuroscience, vol. 21, no. 9, 2018. doi · arXiv
- C. Zheng, W. Wu, C. Chen, T. Yang, S. Zhu, J. Shen, N. Kehtarnavaz and M. Shah, “Deep learning-based human pose estimation: A survey,” ACM Computing Surveys, vol. 56, no. 1, 2024. doi · arXiv
- Y. Chen, Y. Tian and M. He, “Monocular human pose estimation: A survey of deep learning-based methods,” Computer Vision and Image Understanding, vol. 192, 102897, 2020. doi · arXiv
- COCO Consortium, “Keypoint evaluation,” COCO dataset official documentation (accessed 2026-11-01). cocodataset.org
- Y. Tian, H. Zhang, Y. Liu and L. Wang, “Recovering 3D human mesh from monocular images: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, 2023. doi · arXiv
- A. Toshev and C. Szegedy, “DeepPose: Human pose estimation via deep neural networks,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014. doi · arXiv
- J. Tompson, A. Jain, Y. LeCun and C. Bregler, “Joint training of a convolutional network and a graphical model for human pose estimation,” in Advances in Neural Information Processing Systems (NeurIPS), 2014. NeurIPS · arXiv
- A. Newell, K. Yang and J. Deng, “Stacked hourglass networks for human pose estimation,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2016. doi · arXiv
- B. Xiao, H. Wu and Y. Wei, “Simple baselines for human pose estimation and tracking,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2018. doi · arXiv
- F. Zhang, X. Zhu, H. Dai, M. Ye and C. Zhu, “Distribution-aware coordinate representation for human pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. doi · arXiv
- X. Sun, B. Xiao, F. Wei, S. Liang and Y. Wei, “Integral human pose regression,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2018. doi · arXiv
- Y. Li, S. Yang, P. Liu, S. Zhang, Y. Wang, Z. Wang, W. Yang and S.-T. Xia, “SimCC: A simple coordinate classification perspective for human pose estimation,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2022. doi · arXiv
- J. Li, S. Bian, A. Zeng, C. Wang, B. Pang, W. Liu and C. Lu, “Human pose regression with residual log-likelihood estimation,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2021. doi · arXiv
- S.-E. Wei, V. Ramakrishna, T. Kanade and Y. Sheikh, “Convolutional pose machines,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. doi · arXiv
- K. Sun, B. Xiao, D. Liu and J. Wang, “Deep high-resolution representation learning for human pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
- Y. Xu, J. Zhang, Q. Zhang and D. Tao, “ViTPose: Simple vision transformer baselines for human pose estimation,” in Advances in Neural Information Processing Systems (NeurIPS), 2022. NeurIPS · arXiv
- K. He, G. Gkioxari, P. Dollár and R. Girshick, “Mask R-CNN,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi · arXiv
- Z. Cao, T. Simon, S.-E. Wei and Y. Sheikh, “Realtime multi-person 2D pose estimation using part affinity fields,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. doi · arXiv
- A. Newell, Z. Huang and J. Deng, “Associative embedding: End-to-end learning for joint detection and grouping,” in Advances in Neural Information Processing Systems (NeurIPS), 2017. NeurIPS · arXiv
- B. Cheng, B. Xiao, J. Wang, H. Shi, T. S. Huang and L. Zhang, “HigherHRNet: Scale-aware representation learning for bottom-up human pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. doi · arXiv
- Z. Geng, K. Sun, B. Xiao, Z. Zhang and J. Wang, “Bottom-up human pose estimation via disentangled keypoint regression,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. doi · arXiv
- X. Zhou, D. Wang and P. Krähenbühl, “Objects as points,” arXiv:1904.07850, 2019. arXiv
- P. Lu, T. Jiang, Y. Li, X. Li, K. Chen and W. Yang, “RTMO: Towards high-performance one-stage real-time multi-person pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. doi · arXiv
- J. Yang, A. Zeng, S. Liu, F. Li, R. Zhang and L. Zhang, “Explicit box detection unifies end-to-end multi-person pose estimation,” in Proc. International Conference on Learning Representations (ICLR), 2023. OpenReview · arXiv
- Y. Yang and D. Ramanan, “Articulated human detection with flexible mixtures of parts,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 12, 2013. doi
- M. Andriluka, L. Pishchulin, P. Gehler and B. Schiele, “2D human pose estimation: New benchmark and state of the art analysis,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014. doi · CVF
- COCO Consortium,
cocoeval.py(keypoint OKS computation and the per-keypoint sigmas), official COCO API source code (accessed 2026-11-01). GitHub - J. Martinez, R. Hossain, J. Romero and J. J. Little, “A simple yet effective baseline for 3d human pose estimation,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi · arXiv
- M. R. Ronchi and P. Perona, “Benchmarking and error diagnosis in multi-instance pose estimation,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi · arXiv
- T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2014. doi · arXiv
- J. Li, C. Wang, H. Zhu, Y. Mao, H.-S. Fang and C. Lu, “CrowdPose: Efficient crowded scenes pose estimation and a new benchmark,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
- S. Jin, L. Xu, J. Xu, C. Wang, W. Liu, C. Qian, W. Ouyang and P. Luo, “Whole-body human pose estimation in the wild,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2020. doi · arXiv
- M. Andriluka, U. Iqbal, E. Insafutdinov, L. Pishchulin, A. Milan, J. Gall and B. Schiele, “PoseTrack: A benchmark for human pose estimation and tracking,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. doi · arXiv
- C. Ionescu, D. Papava, V. Olaru and C. Sminchisescu, “Human3.6M: Large scale datasets and predictive methods for 3D human sensing in natural environments,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 7, 2014. doi · project page
- H. Yu, Y. Xu, J. Zhang, W. Zhao, Z. Guan and D. Tao, “AP-10K: A benchmark for animal pose estimation in the wild,” in Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021. NeurIPS · arXiv
- Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei and Y. Sheikh, “OpenPose: Realtime multi-person 2D pose estimation using part affinity fields,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, 2021. doi · arXiv
- D. Pavllo, C. Feichtenhofer, D. Grangier and M. Auli, “3D human pose estimation in video with temporal convolutions and semi-supervised training,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
- K. Iskakov, E. Burkov, V. Lempitsky and Y. Malkov, “Learnable triangulation of human pose,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2019. doi · arXiv
- M. Loper, N. Mahmood, J. Romero, G. Pons-Moll and M. J. Black, “SMPL: A skinned multi-person linear model,” ACM Transactions on Graphics (Proc. SIGGRAPH Asia), vol. 34, no. 6, 2015. doi
- A. Kanazawa, M. J. Black, D. W. Jacobs and J. Malik, “End-to-end recovery of human shape and pose,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. doi · arXiv
- S. Goel, G. Pavlakos, J. Rajasegaran, A. Kanazawa and J. Malik, “Humans in 4D: Reconstructing and tracking humans with transformers,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2023. doi · arXiv
- R. Khirodkar, T. Bagautdinov, J. Martinez, S. Zhaoen, A. James, P. Selednik, S. Anderson and S. Saito, “Sapiens: Foundation for human vision models,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2024. doi · arXiv
- T. Jiang, P. Lu, L. Zhang, N. Ma, R. Han, C. Lyu, Y. Li and K. Chen, “RTMPose: Real-time multi-person pose estimation based on MMPose,” arXiv:2303.07399, 2023. arXiv