完整搜尋頁面 →

Simple Baselines(Xiao 等人,2018)

選讀・DLCV 05・人體姿態與關鍵點估計 的論文導讀(第 1/3 篇)。引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

本頁內容

補充資料 · 5

  1. 以 Taylor 步驟做子像素解碼
  2. 座標分類與以似然為基礎的迴歸
  3. 翻轉測試與左右對調的記帳
  4. 深入 OKS:未標註的人、擁擠區域與尺度 s
  5. SMPL 細說

論文導讀 · 3

  1. Simple Baselines(Xiao 等人,2018)
  2. OpenPose:部分親和場(Cao 等人,2017)
  3. 「現代觀點」綜述導讀

電腦視覺深度學習

  1. 02 · 深度影像分類
  2. 03 · 物件偵測:從 R-CNN 到 DETR
  3. 09 · 深度影像復原:低階視覺
查看完整課程樹 →

挑戰

到了 2018 年,頂尖的姿態估計器已變得很複雜:多階段沙漏、級聯金字塔網路、精修模組。很難分辨哪些部分真正重要。Xiao、Wu 與 Wei [9] 問:最簡單而合理的設計能走多遠?他們把它作為評估新想法的基線,同時適用於單張影像的姿態與影片中的姿態追蹤。

比較圖中架構,特別是骨幹後接反卷積熱圖預測的簡單設計;由上而下的人物裁切仍是流程一部分。
原論文圖 1 — 比較圖中架構,特別是骨幹後接反卷積熱圖預測的簡單設計;由上而下的人物裁切仍是流程一部分。

圖片來源: [9], arXiv:1804.06208v2, PDF 第 3 頁. 原始 PDF.

方法

取 ImageNet 預訓練的 ResNet,什麼都不拿掉,在最後階段之上加三層反卷積層(含批次正規化與 ReLU);用 1×11 \times 1 卷積預測 KK 張熱圖;以 MSE 對高斯目標訓練。使用標準的由上而下流程,搭配 Faster R-CNN 人物偵測器、翻轉測試與四分之一偏移解碼器。追蹤時,用光流把每個人的關鍵點傳播到下一個影格以產生額外的框,並用基於光流的 OKS 相似度做貪婪配對,連結跨影格的人。

結果

這個簡單的網路在 COCO 上與當時較複雜的模型不相上下,甚至更好;論文最佳的單一模型在 COCO test-dev 上達到 73.7 AP,其追蹤系統在發表時是 PoseTrack 基準上的最先進。

判讀證據

消融實驗(表 2)在 COCO val2017 上一次只改變一個因素:熱圖解析度(三層反卷積得到 64×4864 \times 48 的圖,比兩層得到 32×2432 \times 24 的圖高 2.5 AP)、反卷積核大小(影響很小,核大小 4 與 2 之間差 0.3 AP)、骨幹深度(越深的 ResNet 穩定地越有幫助)與輸入尺寸。與 Hourglass 和 CPN 的比較(表 3)使用的數字是引自 CPN 論文,而不是重新實作的;作者明說了這一點,並指出差異可能來自實作細節。在比較任何由上而下的數字之前,還有兩個設定必須一致:人物偵測器(論文的偵測器在 COCO val2017 上的人物 AP 為 56.4,Hourglass 與 CPN 使用的偵測器為 55.3)與輸入尺寸(256×192256 \times 192 對 384×288384 \times 288)。

限制與問題

它是由上而下的方法,所以繼承了偵測器所有的失敗,成本也隨人數成長。它的熱圖仍是步幅 4,所以解碼精度取決於本篇所述的後處理。HRNet(其中一些作者相同)後來證明,讓高解析度貫穿整個網路,比到最後才恢復它更好 [15]。

結論

它是現代由上而下姿態估計器的範本,ViTPose 等後續系統也明確保留了它簡單的反卷積解碼器 [16]。

參考文獻

  1. A. Mathis, P. Mamidanna, K. M. Cury, T. Abe, V. N. Murthy, M. W. Mathis and M. Bethge, “DeepLabCut: Markerless pose estimation of user-defined body parts with deep learning,” Nature Neuroscience, vol. 21, no. 9, 2018. doi · arXiv
  2. C. Zheng, W. Wu, C. Chen, T. Yang, S. Zhu, J. Shen, N. Kehtarnavaz and M. Shah, “Deep learning-based human pose estimation: A survey,” ACM Computing Surveys, vol. 56, no. 1, 2024. doi · arXiv
  3. Y. Chen, Y. Tian and M. He, “Monocular human pose estimation: A survey of deep learning-based methods,” Computer Vision and Image Understanding, vol. 192, 102897, 2020. doi · arXiv
  4. COCO Consortium, “Keypoint evaluation,” COCO dataset official documentation (accessed 2026-11-01). cocodataset.org
  5. Y. Tian, H. Zhang, Y. Liu and L. Wang, “Recovering 3D human mesh from monocular images: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 12, 2023. doi · arXiv
  6. A. Toshev and C. Szegedy, “DeepPose: Human pose estimation via deep neural networks,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014. doi · arXiv
  7. J. Tompson, A. Jain, Y. LeCun and C. Bregler, “Joint training of a convolutional network and a graphical model for human pose estimation,” in Advances in Neural Information Processing Systems (NeurIPS), 2014. NeurIPS · arXiv
  8. A. Newell, K. Yang and J. Deng, “Stacked hourglass networks for human pose estimation,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2016. doi · arXiv
  9. B. Xiao, H. Wu and Y. Wei, “Simple baselines for human pose estimation and tracking,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2018. doi · arXiv
  10. F. Zhang, X. Zhu, H. Dai, M. Ye and C. Zhu, “Distribution-aware coordinate representation for human pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. doi · arXiv
  11. X. Sun, B. Xiao, F. Wei, S. Liang and Y. Wei, “Integral human pose regression,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2018. doi · arXiv
  12. Y. Li, S. Yang, P. Liu, S. Zhang, Y. Wang, Z. Wang, W. Yang and S.-T. Xia, “SimCC: A simple coordinate classification perspective for human pose estimation,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2022. doi · arXiv
  13. J. Li, S. Bian, A. Zeng, C. Wang, B. Pang, W. Liu and C. Lu, “Human pose regression with residual log-likelihood estimation,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2021. doi · arXiv
  14. S.-E. Wei, V. Ramakrishna, T. Kanade and Y. Sheikh, “Convolutional pose machines,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. doi · arXiv
  15. K. Sun, B. Xiao, D. Liu and J. Wang, “Deep high-resolution representation learning for human pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
  16. Y. Xu, J. Zhang, Q. Zhang and D. Tao, “ViTPose: Simple vision transformer baselines for human pose estimation,” in Advances in Neural Information Processing Systems (NeurIPS), 2022. NeurIPS · arXiv
  17. K. He, G. Gkioxari, P. Dollár and R. Girshick, “Mask R-CNN,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi · arXiv
  18. Z. Cao, T. Simon, S.-E. Wei and Y. Sheikh, “Realtime multi-person 2D pose estimation using part affinity fields,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. doi · arXiv
  19. A. Newell, Z. Huang and J. Deng, “Associative embedding: End-to-end learning for joint detection and grouping,” in Advances in Neural Information Processing Systems (NeurIPS), 2017. NeurIPS · arXiv
  20. B. Cheng, B. Xiao, J. Wang, H. Shi, T. S. Huang and L. Zhang, “HigherHRNet: Scale-aware representation learning for bottom-up human pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. doi · arXiv
  21. Z. Geng, K. Sun, B. Xiao, Z. Zhang and J. Wang, “Bottom-up human pose estimation via disentangled keypoint regression,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. doi · arXiv
  22. X. Zhou, D. Wang and P. Krähenbühl, “Objects as points,” arXiv:1904.07850, 2019. arXiv
  23. P. Lu, T. Jiang, Y. Li, X. Li, K. Chen and W. Yang, “RTMO: Towards high-performance one-stage real-time multi-person pose estimation,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. doi · arXiv
  24. J. Yang, A. Zeng, S. Liu, F. Li, R. Zhang and L. Zhang, “Explicit box detection unifies end-to-end multi-person pose estimation,” in Proc. International Conference on Learning Representations (ICLR), 2023. OpenReview · arXiv
  25. Y. Yang and D. Ramanan, “Articulated human detection with flexible mixtures of parts,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 12, 2013. doi
  26. M. Andriluka, L. Pishchulin, P. Gehler and B. Schiele, “2D human pose estimation: New benchmark and state of the art analysis,” in Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014. doi · CVF
  27. COCO Consortium, cocoeval.py (keypoint OKS computation and the per-keypoint sigmas), official COCO API source code (accessed 2026-11-01). GitHub
  28. J. Martinez, R. Hossain, J. Romero and J. J. Little, “A simple yet effective baseline for 3d human pose estimation,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi · arXiv
  29. M. R. Ronchi and P. Perona, “Benchmarking and error diagnosis in multi-instance pose estimation,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017. doi · arXiv
  30. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár and C. L. Zitnick, “Microsoft COCO: Common objects in context,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2014. doi · arXiv
  31. J. Li, C. Wang, H. Zhu, Y. Mao, H.-S. Fang and C. Lu, “CrowdPose: Efficient crowded scenes pose estimation and a new benchmark,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
  32. S. Jin, L. Xu, J. Xu, C. Wang, W. Liu, C. Qian, W. Ouyang and P. Luo, “Whole-body human pose estimation in the wild,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2020. doi · arXiv
  33. M. Andriluka, U. Iqbal, E. Insafutdinov, L. Pishchulin, A. Milan, J. Gall and B. Schiele, “PoseTrack: A benchmark for human pose estimation and tracking,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. doi · arXiv
  34. C. Ionescu, D. Papava, V. Olaru and C. Sminchisescu, “Human3.6M: Large scale datasets and predictive methods for 3D human sensing in natural environments,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 7, 2014. doi · project page
  35. H. Yu, Y. Xu, J. Zhang, W. Zhao, Z. Guan and D. Tao, “AP-10K: A benchmark for animal pose estimation in the wild,” in Advances in Neural Information Processing Systems (NeurIPS) Datasets and Benchmarks Track, 2021. NeurIPS · arXiv
  36. Z. Cao, G. Hidalgo, T. Simon, S.-E. Wei and Y. Sheikh, “OpenPose: Realtime multi-person 2D pose estimation using part affinity fields,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 1, 2021. doi · arXiv
  37. D. Pavllo, C. Feichtenhofer, D. Grangier and M. Auli, “3D human pose estimation in video with temporal convolutions and semi-supervised training,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. doi · arXiv
  38. K. Iskakov, E. Burkov, V. Lempitsky and Y. Malkov, “Learnable triangulation of human pose,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2019. doi · arXiv
  39. M. Loper, N. Mahmood, J. Romero, G. Pons-Moll and M. J. Black, “SMPL: A skinned multi-person linear model,” ACM Transactions on Graphics (Proc. SIGGRAPH Asia), vol. 34, no. 6, 2015. doi
  40. A. Kanazawa, M. J. Black, D. W. Jacobs and J. Malik, “End-to-end recovery of human shape and pose,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. doi · arXiv
  41. S. Goel, G. Pavlakos, J. Rajasegaran, A. Kanazawa and J. Malik, “Humans in 4D: Reconstructing and tracking humans with transformers,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2023. doi · arXiv
  42. R. Khirodkar, T. Bagautdinov, J. Martinez, S. Zhaoen, A. James, P. Selednik, S. Anderson and S. Saito, “Sapiens: Foundation for human vision models,” in Proc. European Conference on Computer Vision (ECCV), LNCS, 2024. doi · arXiv
  43. T. Jiang, P. Lu, L. Zhang, N. Ma, R. Han, C. Lyu, Y. Li and K. Chen, “RTMPose: Real-time multi-person pose estimation based on MMPose,” arXiv:2303.07399, 2023. arXiv