Full search page →

Homography: the projective transform

Optional · supplementary note 4 of 5 for IP 02 · Digital Image Fundamentals. Nothing in the main text depends on it; citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

Take a photo of a whiteboard from straight on, then another from a corner of the room. The board looks like a rectangle in the first photo and like a trapezoid in the second, yet every point on the board has a single, well-defined partner in the other photo. The map between the two photos is a homography. It is the affine transform of the main text with one more degree of freedom: perspective, which makes parallel lines converge.

In homogeneous coordinates a homography is a 3×33 \times 3 matrix acting on the vector (x,y,1)(x, y, 1):

[u′v′w′]=[h11h12h13h21h22h23h31h32h33][xy1],x′=u′w′,y′=v′w′,\begin{bmatrix} u' \\ v' \\ w' \end{bmatrix} = \begin{bmatrix} h_{11} & h_{12} & h_{13} \\ h_{21} & h_{22} & h_{23} \\ h_{31} & h_{32} & h_{33} \end{bmatrix} \begin{bmatrix} x \\ y \\ 1 \end{bmatrix}, \qquad x' = \frac{u'}{w'}, \quad y' = \frac{v'}{w'},

where (x,y)(x, y) is a point in the first image, (x′,y′)(x', y') its image in the second, the hijh_{ij} are the entries of the matrix H\mathbf{H}, and (u′,v′,w′)(u', v', w') are the unnormalized homogeneous coordinates. The final division by w′w' is what makes the map non-linear in (x,y)(x, y). Multiplying all nine entries by the same non-zero constant gives the same map, so only the ratios matter: H\mathbf{H} has 9−1=89 - 1 = 8 degrees of freedom. If the last row is (0,0,1)(0, 0, 1), then w′=1w' = 1, the division does nothing, and we are back to the affine transform (6 degrees of freedom). The two extra numbers h31h_{31} and h32h_{32} are the perspective terms. Straight lines stay straight under a homography, but parallel lines in general do not stay parallel.

The reason it matters: points on one flat surface, seen by a pinhole camera from two viewpoints, are related exactly by a homography, and so are the images of any scene taken by a camera that only rotates [20]. That covers document scanning, image mosaics and many alignment tasks.

Estimating H\mathbf{H} from point pairs. Each pair (xi,yi)↔(xi′,yi′)(x_i, y_i) \leftrightarrow (x_i', y_i') gives two linear equations in the nine entries. Clearing the denominator of x′=u′/w′x' = u'/w' gives x′(h31x+h32y+h33)=h11x+h12y+h13x'(h_{31}x + h_{32}y + h_{33}) = h_{11}x + h_{12}y + h_{13}, and likewise for y′y'. Stack the rows of all nn pairs into a matrix A\mathbf{A} of size 2n×92n \times 9, and the unknown vector h=(h11,…,h33)\mathbf{h} = (h_{11}, \dots, h_{33}) must satisfy Ah=0\mathbf{A}\mathbf{h} = \mathbf{0}. Eight equations (4 pairs, no three on a line) fix h\mathbf{h} up to scale. With more, noisy pairs, take the unit vector that makes ∥Ah∥\lVert \mathbf{A}\mathbf{h} \rVert smallest: it is the right singular vector of A\mathbf{A} belonging to the smallest singular value. This is the direct linear transform (DLT). Real matches contain wrong pairs, so it is normally wrapped in RANSAC: fit H\mathbf{H} from 4 random pairs, count how many other pairs agree, keep the best (see RANSAC: fitting a model despite outliers in IP 12).

Code · Python · 29 lines
import numpy as np
from skimage.transform import ProjectiveTransform

def apply_h(H, pts):                       # pts: (n, 2) -> (n, 2)
    p = np.c_[pts, np.ones(len(pts))] @ H.T
    return p[:, :2] / p[:, 2:3]            # divide by the third coordinate

def dlt(src, dst):
    rows = []
    for (x, y), (u, v) in zip(src, dst):   # two equations per point pair
        rows.append([-x, -y, -1, 0, 0, 0, u * x, u * y, u])
        rows.append([0, 0, 0, -x, -y, -1, v * x, v * y, v])
    _, _, Vt = np.linalg.svd(np.array(rows))
    H = Vt[-1].reshape(3, 3)               # singular vector of the smallest singular value
    return H / H[2, 2]                     # fix the free scale

H_true = np.array([[1.1, 0.2, 5.0], [-0.1, 0.9, 3.0], [1e-3, 2e-3, 1.0]])
rng = np.random.default_rng(0)
src = rng.uniform(0, 100, (6, 2))
dst = apply_h(H_true, src)

H_dlt = dlt(src, dst)
tf = ProjectiveTransform()
if not tf.estimate(src, dst):
    raise ValueError("point configuration does not determine a homography")
print("max |H_dlt - H_true|      :", np.abs(H_dlt - H_true).max())
print("max |H_skimage - H_true|  :", np.abs(tf.params / tf.params[2, 2] - H_true).max())
print("max point error (DLT)     :", np.abs(apply_h(H_dlt, src) - dst).max())
print("max |skimage - DLT| (mapped):", np.abs(tf(src) - apply_h(H_dlt, src)).max())

The estimator uses the ProjectiveTransform().estimate(src, dst) interface available in the project environment [21]. The Boolean return value reports whether estimation succeeded.

The run prints (last digits can vary with the linear-algebra library):

Code · 4 lines
max |H_dlt - H_true|      : 1.9229062786507711e-13
max |H_skimage - H_true|  : 1.9539925233402755e-14
max point error (DLT)     : 5.7198690228688065e-12
max |skimage - DLT| (mapped): 5.7127635955112055e-12

With exact points, both estimators recover the matrix to rounding error. The common mistakes are to forget the division by w′w', to compare matrices without fixing the scale first, and to feed the DLT three collinear points, which leaves H\mathbf{H} undetermined. Hartley and Zisserman [20] give the standard textbook treatment.

References

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 2. publisher page
  2. M. Unser, “Sampling—50 years after Shannon,” Proceedings of the IEEE, vol. 88, no. 4, pp. 569–587, 2000. doi
  3. R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 6, pp. 1153–1160, 1981. doi
  4. P. Thévenaz, T. Blu, and M. Unser, “Interpolation revisited,” IEEE Transactions on Medical Imaging, vol. 19, no. 7, pp. 739–758, 2000. doi
  5. A. Rosenfeld and J. L. Pfaltz, “Distance functions on digital pictures,” Pattern Recognition, vol. 1, pp. 33–61, 1968. doi
  6. B. Zitová and J. Flusser, “Image registration methods: a survey,” Image and Vision Computing, vol. 21, no. 11, pp. 977–1000, 2003. doi
  7. M. Delbracio, D. Kelly, M. S. Brown, and P. Milanfar, “Mobile computational photography: A tour,” arXiv:2102.09000, 2021. arXiv
  8. R. Ramanath, W. E. Snyder, Y. Yoo, and M. S. Drew, “Color image processing pipeline,” IEEE Signal Processing Magazine, vol. 22, no. 1, pp. 34–43, 2005. doi
  9. Z. Wang, J. Chen, and S. C. H. Hoi, “Deep learning for image super-resolution: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
  10. B. Moser, F. Raue, S. Frolov, J. Hees, S. Palacio, and A. Dengel, “Hitchhiker’s guide to super-resolution: Introduction and recent advances,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. arXiv
  11. Y. Chen, S. Liu, and X. Wang, “Learning continuous image representation with local implicit image function,” CVPR, 2021. arXiv
  12. X. Wang, L. Xie, C. Dong, and Y. Shan, “Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data,” arXiv:2107.10833, 2021. arXiv
  13. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. project page
  14. G. Zhai and X. Min, “Perceptual image quality assessment: A survey,” Science China Information Sciences, vol. 63, no. 11, 2020. doi
  15. R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” CVPR, 2018. arXiv
  16. K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Comparison of full-reference image quality models for optimization of image processing systems,” International Journal of Computer Vision, 2021. arXiv
  17. C. Chen, Q. Chen, J. Xu, and V. Koltun, “Learning to see in the dark,” CVPR, 2018. arXiv
  18. A. Ignatov, L. Van Gool, and R. Timofte, “Replacing mobile camera ISP with a single deep learning model,” arXiv:2002.05509, 2020. arXiv
  19. Q. Huynh-Thu and M. Ghanbari, “Scope of validity of PSNR in image/video quality assessment,” Electronics Letters, vol. 44, no. 13, pp. 800–801, 2008. doi
  20. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed., Cambridge University Press, 2004. doi
  21. scikit-image contributors, “ProjectiveTransform.estimate,” scikit-image 0.25.2 documentation, accessed 5 November 2026. API documentation.