SIFT: make local features repeatable before trying to match them
Optional · paper guide 2 of 2 for IP 12 · Feature Extraction. Citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
On this page
- The Moore boundary-following procedure
- Slope chain codes
- Minimum-perimeter polygons in more detail
- The medial axis transform
- Computing shape numbers of a given order
- Moments of a 1-D boundary function
- Euler’s formula for polygonal networks
- Keypoint localization
- RANSAC: fitting a model despite outliers
- Learned features in more detail
Image Processing
Challenge
Matching raw image patches fails when scale, orientation or illumination changes. Lowe’s IJCV 2004 paper [6] joins repeatable keypoint detection with a distinctive local descriptor and a matching strategy. Its contribution is the pipeline, not just a vector length. Read image gradients and Gaussian scale space first, then distinguish detecting the same physical location from recognizing its local appearance.
Method

Source: Lowe [6], IJCV 2004 author manuscript, PDF page 15, figure 7. Original PDF.
Detect a location and a scale. Build Gaussian-smoothed images and differences between adjacent scales:
is the input image, a spatial position, a Gaussian kernel with scale , convolution, the smoothed image, the difference-of-Gaussians response, and the ratio of adjacent scales. Search for extrema against neighbors in space and scale, refine their locations, and reject low-contrast candidates and unstable responses along edges. The edge rejection addresses poor localization along a ridge, not a rule that every feature near an edge is unusable.
Assign orientation, then describe. A local gradient histogram supplies a dominant orientation; strong secondary orientations can produce additional keypoints at the same location and scale. Express the descriptor’s spatial coordinates and gradients relative to this orientation. Figure 7 then pools weighted gradient magnitudes into spatial cells and orientation bins, with interpolation to reduce sensitivity to bin boundaries. Normalize the descriptor, clip large entries and normalize again. These steps reduce effects of contrast and dominant gradients; they do not establish invariance to arbitrary lighting changes.
Match with an ambiguity check. Let and be descriptor distances to the closest and second-closest candidate. Accepting only sufficiently small rejects ambiguous matches: a close first candidate is less convincing if another is almost as close. This ratio is not a probability that the match is correct. The paper’s recognition pipeline also groups compatible matches and verifies pose; the descriptor by itself does not establish geometric consistency.
Results
Figure 11 supports the distance-ratio test. In its experiment, rejecting ratios above 0.8 eliminates 90% of false matches while discarding less than 5% of correct ones. The setting includes random scale/orientation changes, a 30-degree depth rotation, 2% image noise and a database of 40,000 keypoints. These are the paper’s conditional results, not guaranteed error rates for every image pair or nearest-neighbor implementation.
Figures 3–6 and 9 separate detection repeatability, orientation stability and final matching under controlled transformations. Read their denominators and transformation settings before comparing percentages: successful detection and correct matching are different events.
Ablations and design evidence
Figure 8 varies descriptor grid width and orientation-bin count under a 50-degree viewpoint change with 4% noise. The selected representation balances distinctiveness against sensitivity to deformation; simply increasing dimensions is not always helpful. The scale-sampling experiments likewise separate additional detections from repeatable, correctly matched detections. These studies explain design choices rather than proving one universal parameter setting.
Conclusion
SIFT makes a local coordinate frame before comparing appearance. Use it as an interpretable baseline for images with enough textured, overlapping content, and inspect geometric verification after descriptor matching. The chapter’s RANSAC note [21] explains a complementary way to reject geometrically inconsistent correspondences; it is not a claim that the original SIFT recognition pipeline is identical to every modern SIFT-plus-RANSAC implementation.
Problems
Repeated patterns can remain ambiguous, textureless regions may provide few keypoints, and strong viewpoint changes distort a patch beyond scale and rotation normalization. Occlusion removes shared evidence. A larger number of matches does not necessarily mean a better registration: inspect their spatial coverage, consistency and failure cases. Learned alternatives may change these trade-offs, but comparing them requires a shared detection/matching and geometry protocol.
References
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 12. publisher page
- M.-K. Hu, “Visual pattern recognition by moment invariants,” IRE Transactions on Information Theory, vol. 8, no. 2, pp. 179–187, 1962. doi
- R. M. Haralick, K. Shanmugam, and I. Dinstein, “Textural features for image classification,” IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-3, no. 6, pp. 610–621, 1973. doi
- C. Harris and M. Stephens, “A combined corner and edge detector,” in Proc. Alvey Vision Conference, pp. 23.1–23.6, 1988. doi
- J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image and Vision Computing, vol. 22, no. 10, pp. 761–767, 2004. doi
- D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004. doi
- H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359, 2008. doi
- E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” in Proc. IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571, 2011. doi
- D. Zhang and G. Lu, “Review of shape representation and description techniques,” Pattern Recognition, vol. 37, no. 1, pp. 1–19, 2004. doi
- T. Tuytelaars and K. Mikolajczyk, “Local invariant feature detectors: A survey,” Foundations and Trends in Computer Graphics and Vision, vol. 3, no. 3, pp. 177–280, 2008. doi
- L. Liu, J. Chen, P. Fieguth, G. Zhao, R. Chellappa, and M. Pietikäinen, “From BoW to CNN: Two decades of texture representation for texture classification,” International Journal of Computer Vision, vol. 127, pp. 74–109, 2019. arXiv
- J. Ma, X. Jiang, A. Fan, J. Jiang, and J. Yan, “Image matching from handcrafted to deep features: A survey,” International Journal of Computer Vision, vol. 129, pp. 23–79, 2021. doi
- Y. Jin, D. Mishkin, A. Mishchuk, J. Matas, P. Fua, K. M. Yi, and E. Trulls, “Image matching across wide baselines: From paper to practice,” International Journal of Computer Vision, 2021. doi · arXiv
- D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperPoint: Self-supervised interest point detection and description,” arXiv:1712.07629, 2017. arXiv
- P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperGlue: Learning feature matching with graph neural networks,” arXiv:1911.11763, 2019. arXiv
- M. J. Tyszkiewicz, P. Fua, and E. Trulls, “DISK: Learning local features with policy gradient,” arXiv:2006.13566, 2020. arXiv
- J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou, “LoFTR: Detector-free local feature matching with Transformers,” arXiv:2104.00680, 2021. arXiv
- P. Lindenberger, P.-E. Sarlin, and M. Pollefeys, “LightGlue: Local feature matching at light speed,” arXiv:2306.13643, 2023. arXiv
- M. Oquab, T. Darcet, T. Moutakanni, et al., “DINOv2: Learning robust visual features without supervision,” arXiv:2304.07193, 2023. arXiv
- J. Edstedt, Q. Sun, G. Bökman, M. Wadenbäck, and M. Felsberg, “RoMa: Robust dense feature matching,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. arXiv
- M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981. doi
- scikit-image contributors, “skimage.measure.ransac,” scikit-image 0.26 documentation (API reference). docs