完整搜尋頁面 →

單應性:投影轉換

選讀・IP 02・數位影像基礎 的補充資料(第 4/5 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

用正對白板拍一張照,再從房間角落拍一張。白板在第一張裡是長方形,在第二張裡是梯形,但板上每個點在另一張照片裡仍有唯一的對應點。兩張照片之間的這個對應就是單應性(homography)。它是正文仿射轉換再多一個自由度:透視(perspective),也就是讓平行線交會。

在齊次座標下,單應性是一個作用在向量 (x,y,1)(x, y, 1) 上的 3×33 \times 3 矩陣:

[u′v′w′]=[h11h12h13h21h22h23h31h32h33][xy1],x′=u′w′,y′=v′w′,\begin{bmatrix} u' \\ v' \\ w' \end{bmatrix} = \begin{bmatrix} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}, \qquad x' = \frac{u'}{w'}, \quad y' = \frac{v'}{w'},

其中 (x,y)(x, y) 是第一張影像中的點,(x′,y′)(x', y') 是它在第二張影像中的位置,hijh_{ij} 是矩陣 H\mathbf{H} 的元素,(u′,v′,w′)(u', v', w') 是尚未正規化的齊次座標。最後除以 w′w' 這一步,使整個映射對 (x,y)(x, y) 是非線性的。把九個元素同時乘上同一個非零常數,映射不變,所以只有比例有意義:H\mathbf{H} 有 9−1=89 - 1 = 8 個自由度。如果最後一列是 (0,0,1)(0, 0, 1),則 w′=1w' = 1,除法不起作用,就回到仿射轉換(6 個自由度)。多出來的兩個數 h31h_{31} 與 h32h_{32} 就是透視項。單應性會保持直線仍是直線,但一般不保持平行。

它之所以重要:同一個平面上的點,被針孔相機(pinhole camera)從兩個視角拍到時,彼此恰好由單應性相連;只做旋轉的相機所拍的任何場景,兩張影像之間也是如此 [20]。文件掃描、影像拼接與許多對位工作都屬於這種情形。

由點對估計 H\mathbf{H}。每一組點對 (xi,yi)↔(xi′,yi′)(x_i, y_i) \leftrightarrow (x_i', y_i') 對九個元素提供兩個線性方程式。把 x′=u′/w′x' = u'/w' 的分母乘開,得到 x′(h31x+h32y+h33)=h11x+h12y+h13x'(h_{31}x + h_{32}y + h_{33}) = h_{11}x + h_{12}y + h_{13},y′y' 同理。把 nn 組點對的列疊成一個 2n×92n \times 9 的矩陣 A\mathbf{A},未知向量 h=(h11,…,h33)\mathbf{h} = (h_{11}, \dots, h_{33}) 必須滿足 Ah=0\mathbf{A}\mathbf{h} = \mathbf{0}。八個方程式(4 組點對,且任三點不共線)就能決定 h\mathbf{h} 到只差一個比例。若點對更多且含雜訊,就取使 ∥Ah∥\lVert \mathbf{A}\mathbf{h} \rVert 最小的單位向量,也就是 A\mathbf{A} 最小奇異值對應的右奇異向量。這就是直接線性轉換(direct linear transform,DLT)。實際的匹配含有錯誤的點對,所以通常把它包進 RANSAC:用 4 組隨機點對擬合 H\mathbf{H},計算有多少其他點對與它一致,保留最好的(見 IP 12 的 RANSAC:在有離群值時擬合模型)。

程式碼 · Python · 29 行
import numpy as np
from skimage.transform import ProjectiveTransform

def apply_h(H, pts):                       # pts: (n, 2) -> (n, 2)
    p = np.c_[pts, np.ones(len(pts))] @ H.T
    return p[:, :2] / p[:, 2:3]            # divide by the third coordinate

def dlt(src, dst):
    rows = []
    for (x, y), (u, v) in zip(src, dst):   # two equations per point pair
        rows.append([-x, -y, -1, 0, 0, 0, u * x, u * y, u])
        rows.append([0, 0, 0, -x, -y, -1, v * x, v * y, v])
    _, _, Vt = np.linalg.svd(np.array(rows))
    H = Vt[-1].reshape(3, 3)               # singular vector of the smallest singular value
    return H / H[2, 2]                     # fix the free scale

H_true = np.array([[1.1, 0.2, 5.0], [-0.1, 0.9, 3.0], [1e-3, 2e-3, 1.0]])
rng = np.random.default_rng(0)
src = rng.uniform(0, 100, (6, 2))
dst = apply_h(H_true, src)

H_dlt = dlt(src, dst)
tf = ProjectiveTransform()
if not tf.estimate(src, dst):
    raise ValueError("point configuration does not determine a homography")
print("max |H_dlt - H_true|      :", np.abs(H_dlt - H_true).max())
print("max |H_skimage - H_true|  :", np.abs(tf.params / tf.params[2, 2] - H_true).max())
print("max point error (DLT)     :", np.abs(apply_h(H_dlt, src) - dst).max())
print("max |skimage - DLT| (mapped):", np.abs(tf(src) - apply_h(H_dlt, src)).max())

估計器使用專案環境支援的 ProjectiveTransform().estimate(src, dst) 介面 [21];布林回傳值表示估計是否成功。以下輸出的最後幾位可能隨線性代數函式庫而變化。

執行後印出:

程式碼 · 4 行
max |H_dlt - H_true|      : 1.9229062786507711e-13
max |H_skimage - H_true|  : 1.9539925233402755e-14
max point error (DLT)     : 5.7198690228688065e-12
max |skimage - DLT| (mapped): 5.7127635955112055e-12

點是精確的時候,兩種估計方法都把矩陣還原到捨入誤差等級。常見的錯誤有:忘了除以 w′w'、比較矩陣前沒有先固定比例,以及把三個共線的點餵給 DLT,這時 H\mathbf{H} 無法被唯一決定。Hartley 與 Zisserman [20] 是這個主題的標準教科書。

參考文獻

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 2. publisher page
  2. M. Unser, “Sampling—50 years after Shannon,” Proceedings of the IEEE, vol. 88, no. 4, pp. 569–587, 2000. doi
  3. R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 6, pp. 1153–1160, 1981. doi
  4. P. Thévenaz, T. Blu, and M. Unser, “Interpolation revisited,” IEEE Transactions on Medical Imaging, vol. 19, no. 7, pp. 739–758, 2000. doi
  5. A. Rosenfeld and J. L. Pfaltz, “Distance functions on digital pictures,” Pattern Recognition, vol. 1, pp. 33–61, 1968. doi
  6. B. Zitová and J. Flusser, “Image registration methods: a survey,” Image and Vision Computing, vol. 21, no. 11, pp. 977–1000, 2003. doi
  7. M. Delbracio, D. Kelly, M. S. Brown, and P. Milanfar, “Mobile computational photography: A tour,” arXiv:2102.09000, 2021. arXiv
  8. R. Ramanath, W. E. Snyder, Y. Yoo, and M. S. Drew, “Color image processing pipeline,” IEEE Signal Processing Magazine, vol. 22, no. 1, pp. 34–43, 2005. doi
  9. Z. Wang, J. Chen, and S. C. H. Hoi, “Deep learning for image super-resolution: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
  10. B. Moser, F. Raue, S. Frolov, J. Hees, S. Palacio, and A. Dengel, “Hitchhiker’s guide to super-resolution: Introduction and recent advances,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. arXiv
  11. Y. Chen, S. Liu, and X. Wang, “Learning continuous image representation with local implicit image function,” CVPR, 2021. arXiv
  12. X. Wang, L. Xie, C. Dong, and Y. Shan, “Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data,” arXiv:2107.10833, 2021. arXiv
  13. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. project page
  14. G. Zhai and X. Min, “Perceptual image quality assessment: A survey,” Science China Information Sciences, vol. 63, no. 11, 2020. doi
  15. R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” CVPR, 2018. arXiv
  16. K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Comparison of full-reference image quality models for optimization of image processing systems,” International Journal of Computer Vision, 2021. arXiv
  17. C. Chen, Q. Chen, J. Xu, and V. Koltun, “Learning to see in the dark,” CVPR, 2018. arXiv
  18. A. Ignatov, L. Van Gool, and R. Timofte, “Replacing mobile camera ISP with a single deep learning model,” arXiv:2002.05509, 2020. arXiv
  19. Q. Huynh-Thu and M. Ghanbari, “Scope of validity of PSNR in image/video quality assessment,” Electronics Letters, vol. 44, no. 13, pp. 800–801, 2008. doi
  20. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed., Cambridge University Press, 2004. doi
  21. scikit-image contributors, “ProjectiveTransform.estimate,” scikit-image 0.25.2 documentation, accessed 5 November 2026. API 文件.