完整搜尋頁面 →

對齊臉部:最小平方相似變換

選讀・DLCV 02A・臉部辨識與度量學習 的補充資料(第 1/6 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

本頁內容

補充資料 · 6

  1. 對齊臉部:最小平方相似變換
  2. 尺度 s 要多大?
  3. 並排比較邊界損失:一個公式與它的特殊情況
  4. 一個類別不只一個中心:子中心、品質感知邊界與 Partial FC
  5. 量測極小的錯誤率
  6. 深度學習之前:Eigenfaces 與 Fisherfaces

論文導讀 · 3

  1. ArcFace(Deng 等人,CVPR 2019;TPAMI 2022)
  2. NIST FRVT Part 3:族群效應(Grother、Ngan 與 Hanaoka,2019)
  3. 現代觀點綜述導讀

電腦視覺深度學習

  1. 02 · 深度影像分類
  2. 03 · 物件偵測:從 R-CNN 到 DETR
  3. 09 · 深度影像復原:低階視覺
查看完整課程樹 →

這則補充推導正文使用的對齊擬合,並列出它可能出的問題。

問題。 給定 nn 個偵測到的特徵點 pi=(ui,vi)\mathbf{p}_i = (u_i, v_i) 與模板位置 qi=(ui′,vi′)\mathbf{q}_i = (u'_i, v'_i),求使下式最小的 a,b,tx,tya, b, t_x, t_y:

E(a,b,tx,ty)=∑i=1n[(aui−bvi+tx−ui′)2+(bui+avi+ty−vi′)2].E(a, b, t_x, t_y) = \sum_{i=1}^{n} \Big[ (a u_i - b v_i + t_x - u'_i)^2 + (b u_i + a v_i + t_y - v'_i)^2 \Big].

每個特徵點對線性方程組 Aβ=rA\boldsymbol{\beta} = \mathbf{r}(其中 β=(a,b,tx,ty)⊤\boldsymbol{\beta} = (a, b, t_x, t_y)^\top)貢獻兩列:一列是 (ui,−vi,1,0)(u_i, -v_i, 1, 0),右邊是 ui′u'_i;另一列是 (vi,ui,0,1)(v_i, u_i, 0, 1),右邊是 vi′v'_i。最小平方解滿足正規方程式 A⊤A β=A⊤rA^\top A\,\boldsymbol{\beta} = A^\top \mathbf{r},np.linalg.lstsq 會穩定地解出它。

封閉解。 先把兩組點都置中:pˉ=1n∑ipi\bar{\mathbf{p}} = \frac1n\sum_i \mathbf{p}_i,p~i=pi−pˉ\tilde{\mathbf{p}}_i = \mathbf{p}_i - \bar{\mathbf{p}},q~i\tilde{\mathbf{q}}_i 同理。則最佳的 aa 與 bb 為

a=∑ip~i⊤q~i∑i∥p~i∥2,b=∑i(u~iv~i′−v~iu~i′)∑i∥p~i∥2,a = \frac{\sum_i \tilde{\mathbf{p}}_i^\top \tilde{\mathbf{q}}_i}{\sum_i \lVert \tilde{\mathbf{p}}_i \rVert^2}, \qquad b = \frac{\sum_i (\tilde u_i \tilde v'_i - \tilde v_i \tilde u'_i)}{\sum_i \lVert \tilde{\mathbf{p}}_i \rVert^2},

其中 (u~i,v~i)(\tilde u_i, \tilde v_i) 與 (u~i′,v~i′)(\tilde u'_i, \tilde v'_i) 分別是 p~i\tilde{\mathbf{p}}_i 與 q~i\tilde{\mathbf{q}}_i 的座標,平移量為 t=qˉ−[a−bba]pˉ\mathbf{t} = \bar{\mathbf{q}} - \begin{bmatrix} a & -b \\ b & a \end{bmatrix}\bar{\mathbf{p}}。縮放比例與角度則為 σ=a2+b2\sigma = \sqrt{a^2 + b^2} 與 ϕ=atan2⁡(b,a)\phi = \operatorname{atan2}(b, a)。bb 的分子是二維外積之和,所以當最佳旋轉恰為零時 b=0b = 0。

為什麼不用更有彈性的扭轉? DeepFace 走了相反的路,使用明確的三維臉部建模與分段仿射扭轉 [6],但後來大多數系統都採用簡單的二維相似變換。完整的仿射變換(六個參數)還能在 xx 與 yy 方向把臉剪切與拉伸成不同程度;投影變換(八個參數)能做的更多。只有五個帶雜訊的特徵點時,這些額外的自由度多半只是在擬合雜訊,並扭曲臉部的比例,而比例正是辨識一個人的依據之一。相似變換則保持角度與距離比例。

特殊情況。

  • 鏡射。 參數化 [a−bba]\begin{bmatrix} a & -b \\ b & a \end{bmatrix} 不可能把影像鏡射,這正是我們想要的;鏡射過的臉是不同的影像(不過水平翻轉有時被用作測試時的資料擴增)。
  • 退化的配置。 如果所有特徵點(幾乎)是同一個點,則 ∑i∥p~i∥2≈0\sum_i \lVert \tilde{\mathbf{p}}_i \rVert^2 \approx 0,縮放比例沒有定義;這會發生在很小的臉或偵測失敗時,應在對齊前就拒絕。
  • 側臉。 偏航角很大時,一隻眼睛與一個嘴角被藏起來,偵測器對它們的估計只是猜測。五點正面模板會把臉拉成不自然的裁切。系統要不就使用特定姿勢的模板,要不就接受對齊品質隨姿勢下降。
  • 一個壞的特徵點。 最小平方對離群值很敏感:一個偏離 20 像素的特徵點就會讓整個擬合移位。檢查殘差(如程式碼所印出的)並拒絕殘差大的擬合,是簡單的保護措施。

參考文獻

  1. M. Wang and W. Deng, “Deep face recognition: A survey,” Neurocomputing, vol. 429, pp. 215–244, 2021. doi
  2. P. Grother, M. Ngan and K. Hanaoka, “Face Recognition Vendor Test (FRVT) Part 3: Demographic Effects,” NIST Interagency Report 8280, National Institute of Standards and Technology, 2019. doi
  3. National Institute of Standards and Technology, “Face Recognition Technology Evaluation (FRTE) 1:1 Verification,” evaluation web page, accessed 2026. NIST
  4. K. Zhang, Z. Zhang, Z. Li and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499–1503, 2016. doi
  5. J. Deng, J. Guo, E. Ververas, I. Kotsia and S. Zafeiriou, “RetinaFace: Single-shot multi-level face localisation in the wild,” in Proc. IEEE/CVF CVPR, 2020. CVF
  6. Y. Taigman, M. Yang, M. Ranzato and L. Wolf, “DeepFace: Closing the gap to human-level performance in face verification,” in Proc. IEEE CVPR, 2014. CVF
  7. F. Schroff, D. Kalenichenko and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” in Proc. IEEE CVPR, 2015. CVF
  8. S. Chopra, R. Hadsell and Y. LeCun, “Learning a similarity metric discriminatively, with application to face verification,” in Proc. IEEE CVPR, vol. 1, pp. 539–546, 2005. doi
  9. R. Hadsell, S. Chopra and Y. LeCun, “Dimensionality reduction by learning an invariant mapping,” in Proc. IEEE CVPR, vol. 2, pp. 1735–1742, 2006. doi
  10. A. Hermans, L. Beyer and B. Leibe, “In defense of the triplet loss for person re-identification,” arXiv:1703.07737, 2017. arXiv
  11. F. Wang, X. Xiang, J. Cheng and A. L. Yuille, “NormFace: L2 hypersphere embedding for face verification,” in Proc. ACM International Conference on Multimedia, pp. 1041–1049, 2017. doi
  12. W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj and L. Song, “SphereFace: Deep hypersphere embedding for face recognition,” in Proc. IEEE CVPR, 2017. CVF
  13. H. Wang, Y. Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li and W. Liu, “CosFace: Large margin cosine loss for deep face recognition,” in Proc. IEEE/CVF CVPR, 2018. CVF
  14. F. Wang, J. Cheng, W. Liu and H. Liu, “Additive margin softmax for face verification,” IEEE Signal Processing Letters, vol. 25, no. 7, pp. 926–930, 2018. doi
  15. J. Deng, J. Guo, N. Xue and S. Zafeiriou, “ArcFace: Additive angular margin loss for deep face recognition,” in Proc. IEEE/CVF CVPR, 2019. CVF
  16. J. Deng, J. Guo, J. Yang, N. Xue, I. Kotsia and S. Zafeiriou, “ArcFace: Additive angular margin loss for deep face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 5962–5979, 2022. doi
  17. Y. Wen, K. Zhang, Z. Li and Y. Qiao, “A discriminative feature learning approach for deep face recognition,” in Proc. ECCV, pp. 499–515, 2016. doi
  18. J. Deng, J. Guo, T. Liu, M. Gong and S. Zafeiriou, “Sub-center ArcFace: Boosting face recognition by large-scale noisy web faces,” in Proc. ECCV, pp. 741–757, 2020. doi
  19. Q. Meng, S. Zhao, Z. Huang and F. Zhou, “MagFace: A universal representation for face recognition and quality assessment,” in Proc. IEEE/CVF CVPR, 2021. CVF
  20. M. Kim, A. K. Jain and X. Liu, “AdaFace: Quality adaptive margin for face recognition,” in Proc. IEEE/CVF CVPR, 2022. CVF
  21. X. An, J. Deng, J. Guo, Z. Feng, X. Zhu, J. Yang and T. Liu, “Killing two birds with one stone: Efficient and robust training of face recognition CNNs by Partial FC,” in Proc. IEEE/CVF CVPR, 2022. CVF
  22. A. Martin, G. Doddington, T. Kamm, M. Ordowski and M. Przybocki, “The DET curve in assessment of detection task performance,” in Proc. Eurospeech, pp. 1895–1898, 1997. ISCA
  23. G. B. Huang, M. Mattar, T. Berg and E. Learned-Miller, “Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments,” in Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition, Marseille, 2008. HAL
  24. I. Kemelmacher-Shlizerman, S. M. Seitz, D. Miller and E. Brossard, “The MegaFace benchmark: 1 million faces for recognition at scale,” in Proc. IEEE CVPR, 2016. CVF
  25. University of Washington, “MegaFace and MF2: Million-scale face recognition,” project web page, accessed 2026. project page
  26. Y. Guo, L. Zhang, Y. Hu, X. He and J. Gao, “MS-Celeb-1M: A dataset and benchmark for large-scale face recognition,” in Proc. ECCV, pp. 87–102, 2016. doi
  27. C. Whitelam, E. Taborsky, A. Blanton, B. Maze, J. Adams, T. Miller, N. Kalka, A. K. Jain, J. A. Duncan, K. Allen, J. Cheney and P. Grother, “IARPA Janus Benchmark-B face dataset,” in Proc. IEEE CVPR Workshops, 2017. CVF
  28. B. Maze, J. Adams, J. A. Duncan, N. Kalka, T. Miller, C. Otto, A. K. Jain, W. T. Niggel, J. Anderson, J. Cheney and P. Grother, “IARPA Janus Benchmark – C: Face dataset and protocol,” in Proc. International Conference on Biometrics (ICB), pp. 158–165, 2018. doi
  29. Z. Zhu, G. Huang, J. Deng, Y. Ye, J. Huang, X. Chen, J. Zhu, T. Yang, J. Lu, D. Du and J. Zhou, “WebFace260M: A benchmark unveiling the power of million-scale deep face recognition,” in Proc. IEEE/CVF CVPR, 2021. CVF
  30. K. Peng, A. Mathur and A. Narayanan, “Mitigating dataset harms requires stewardship: Lessons from 1000 papers,” in Proc. NeurIPS Datasets and Benchmarks Track, 2021. proceedings
  31. G. Bae, M. de La Gorce, T. Baltrušaitis, C. Hewitt, D. Chen, J. Valentin, R. Cipolla and J. Shen, “DigiFace-1M: 1 million digital face images for face recognition,” in Proc. IEEE/CVF WACV, 2023. CVF
  32. M. Kim, F. Liu, A. Jain and X. Liu, “DCFace: Synthetic face generation with dual condition diffusion model,” in Proc. IEEE/CVF CVPR, 2023. CVF
  33. F. Boutros, V. Struc, J. Fierrez and N. Damer, “Synthetic data for face recognition: Current state and future prospects,” Image and Vision Computing, vol. 135, art. 104688, 2023. doi
  34. M. Wang, W. Deng, J. Hu, X. Tao and Y. Huang, “Racial Faces in the Wild: Reducing racial bias by information maximization adaptation network,” in Proc. IEEE/CVF ICCV, pp. 692–702, 2019. doi
  35. J. Buolamwini and T. Gebru, “Gender Shades: Intersectional accuracy disparities in commercial gender classification,” in Proc. Conference on Fairness, Accountability and Transparency, PMLR 81, pp. 77–91, 2018. PMLR
  36. P. Drozdowski, C. Rathgeb, A. Dantcheva, N. Damer and C. Busch, “Demographic bias in biometrics: A survey on an emerging challenge,” IEEE Transactions on Technology and Society, vol. 1, no. 2, pp. 89–103, 2020. doi
  37. Z. Yu, Y. Qin, X. Li, C. Zhao, Z. Lei and G. Zhao, “Deep learning for face anti-spoofing: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, pp. 5609–5631, 2023. doi
  38. European Commission, “What personal data is considered sensitive?,” data protection guidance referring to Article 9 of the GDPR, accessed 2026. European Commission
  39. European Commission, “AI Act” (Regulation (EU) 2024/1689), policy page, accessed 2026. European Commission
  40. M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of Cognitive Neuroscience, vol. 3, no. 1, pp. 71–86, 1991. doi
  41. P. N. Belhumeur, J. P. Hespanha and D. J. Kriegman, “Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711–720, 1997. doi