Homography: the projective transform
Optional · supplementary note 4 of 5 for IP 02 · Digital Image Fundamentals. Nothing in the main text depends on it; citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
Take a photo of a whiteboard from straight on, then another from a corner of the room. The board looks like a rectangle in the first photo and like a trapezoid in the second, yet every point on the board has a single, well-defined partner in the other photo. The map between the two photos is a homography. It is the affine transform of the main text with one more degree of freedom: perspective, which makes parallel lines converge.
In homogeneous coordinates a homography is a matrix acting on the vector :
where is a point in the first image, its image in the second, the are the entries of the matrix , and are the unnormalized homogeneous coordinates. The final division by is what makes the map non-linear in . Multiplying all nine entries by the same non-zero constant gives the same map, so only the ratios matter: has degrees of freedom. If the last row is , then , the division does nothing, and we are back to the affine transform (6 degrees of freedom). The two extra numbers and are the perspective terms. Straight lines stay straight under a homography, but parallel lines in general do not stay parallel.
The reason it matters: points on one flat surface, seen by a pinhole camera from two viewpoints, are related exactly by a homography, and so are the images of any scene taken by a camera that only rotates [20]. That covers document scanning, image mosaics and many alignment tasks.
Estimating from point pairs. Each pair gives two linear equations in the nine entries. Clearing the denominator of gives , and likewise for . Stack the rows of all pairs into a matrix of size , and the unknown vector must satisfy . Eight equations (4 pairs, no three on a line) fix up to scale. With more, noisy pairs, take the unit vector that makes smallest: it is the right singular vector of belonging to the smallest singular value. This is the direct linear transform (DLT). Real matches contain wrong pairs, so it is normally wrapped in RANSAC: fit from 4 random pairs, count how many other pairs agree, keep the best (see RANSAC: fitting a model despite outliers in IP 12).
Code · Python · 29 lines
import numpy as np
from skimage.transform import ProjectiveTransform
def apply_h(H, pts): # pts: (n, 2) -> (n, 2)
p = np.c_[pts, np.ones(len(pts))] @ H.T
return p[:, :2] / p[:, 2:3] # divide by the third coordinate
def dlt(src, dst):
rows = []
for (x, y), (u, v) in zip(src, dst): # two equations per point pair
rows.append([-x, -y, -1, 0, 0, 0, u * x, u * y, u])
rows.append([0, 0, 0, -x, -y, -1, v * x, v * y, v])
_, _, Vt = np.linalg.svd(np.array(rows))
H = Vt[-1].reshape(3, 3) # singular vector of the smallest singular value
return H / H[2, 2] # fix the free scale
H_true = np.array([[1.1, 0.2, 5.0], [-0.1, 0.9, 3.0], [1e-3, 2e-3, 1.0]])
rng = np.random.default_rng(0)
src = rng.uniform(0, 100, (6, 2))
dst = apply_h(H_true, src)
H_dlt = dlt(src, dst)
tf = ProjectiveTransform()
if not tf.estimate(src, dst):
raise ValueError("point configuration does not determine a homography")
print("max |H_dlt - H_true| :", np.abs(H_dlt - H_true).max())
print("max |H_skimage - H_true| :", np.abs(tf.params / tf.params[2, 2] - H_true).max())
print("max point error (DLT) :", np.abs(apply_h(H_dlt, src) - dst).max())
print("max |skimage - DLT| (mapped):", np.abs(tf(src) - apply_h(H_dlt, src)).max())The estimator uses the ProjectiveTransform().estimate(src, dst) interface available in the project environment [21]. The Boolean return value reports whether estimation succeeded.
The run prints (last digits can vary with the linear-algebra library):
Code · 4 lines
max |H_dlt - H_true| : 1.9229062786507711e-13
max |H_skimage - H_true| : 1.9539925233402755e-14
max point error (DLT) : 5.7198690228688065e-12
max |skimage - DLT| (mapped): 5.7127635955112055e-12With exact points, both estimators recover the matrix to rounding error. The common mistakes are to forget the division by , to compare matrices without fixing the scale first, and to feed the DLT three collinear points, which leaves undetermined. Hartley and Zisserman [20] give the standard textbook treatment.
References
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 2. publisher page
- M. Unser, “Sampling—50 years after Shannon,” Proceedings of the IEEE, vol. 88, no. 4, pp. 569–587, 2000. doi
- R. Keys, “Cubic convolution interpolation for digital image processing,” IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 29, no. 6, pp. 1153–1160, 1981. doi
- P. Thévenaz, T. Blu, and M. Unser, “Interpolation revisited,” IEEE Transactions on Medical Imaging, vol. 19, no. 7, pp. 739–758, 2000. doi
- A. Rosenfeld and J. L. Pfaltz, “Distance functions on digital pictures,” Pattern Recognition, vol. 1, pp. 33–61, 1968. doi
- B. Zitová and J. Flusser, “Image registration methods: a survey,” Image and Vision Computing, vol. 21, no. 11, pp. 977–1000, 2003. doi
- M. Delbracio, D. Kelly, M. S. Brown, and P. Milanfar, “Mobile computational photography: A tour,” arXiv:2102.09000, 2021. arXiv
- R. Ramanath, W. E. Snyder, Y. Yoo, and M. S. Drew, “Color image processing pipeline,” IEEE Signal Processing Magazine, vol. 22, no. 1, pp. 34–43, 2005. doi
- Z. Wang, J. Chen, and S. C. H. Hoi, “Deep learning for image super-resolution: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
- B. Moser, F. Raue, S. Frolov, J. Hees, S. Palacio, and A. Dengel, “Hitchhiker’s guide to super-resolution: Introduction and recent advances,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023. arXiv
- Y. Chen, S. Liu, and X. Wang, “Learning continuous image representation with local implicit image function,” CVPR, 2021. arXiv
- X. Wang, L. Xie, C. Dong, and Y. Shan, “Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data,” arXiv:2107.10833, 2021. arXiv
- Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. project page
- G. Zhai and X. Min, “Perceptual image quality assessment: A survey,” Science China Information Sciences, vol. 63, no. 11, 2020. doi
- R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” CVPR, 2018. arXiv
- K. Ding, K. Ma, S. Wang, and E. P. Simoncelli, “Comparison of full-reference image quality models for optimization of image processing systems,” International Journal of Computer Vision, 2021. arXiv
- C. Chen, Q. Chen, J. Xu, and V. Koltun, “Learning to see in the dark,” CVPR, 2018. arXiv
- A. Ignatov, L. Van Gool, and R. Timofte, “Replacing mobile camera ISP with a single deep learning model,” arXiv:2002.05509, 2020. arXiv
- Q. Huynh-Thu and M. Ghanbari, “Scope of validity of PSNR in image/video quality assessment,” Electronics Letters, vol. 44, no. 13, pp. 800–801, 2008. doi
- R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd ed., Cambridge University Press, 2004. doi
- scikit-image contributors, “ProjectiveTransform.estimate,” scikit-image 0.25.2 documentation, accessed 5 November 2026. API documentation.