Full search page →

Guide to the Modern view surveys

Optional · paper guide 1 of 2 for IP 12 · Feature Extraction. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

Choose a reading by topic: Shape, Local keypoints, Texture, Matching.

Five surveys and benchmarks frame this chapter’s Modern view.

Feature extraction: invariance versus distinctiveness
Teaching reading map — the cards compare questions addressed by different papers. They are complementary categories, not a performance ranking or a diagram copied from one source.

Shape descriptors

Zhang and Lu’s review [9] sorts shape descriptors along two axes: contour-based versus region-based, and global (one vector for the whole shape) versus structural (the shape broken into primitives). Everything in the first half of this chapter fits that grid: chain codes and polygons are structural contour methods; Fourier descriptors and boundary moments are global contour methods; area, Euler number and Hu moments are global region methods; skeletons are structural region methods. For each technique the review describes the implementation and lists its advantages and disadvantages; the practical message is that no single descriptor is best for every application, so the choice depends on which invariances and how much detail the task needs.

Local feature detectors

Tuytelaars and Mikolajczyk [10] survey interest-point and region detectors: corner detectors (Harris and its scale- and affine-adapted versions), blob detectors (Laplacian/DoG, Hessian), and region detectors (MSER and others). They organize the field around properties a good local feature should have: repeatability, distinctiveness, locality, quantity, accuracy and efficiency. They point out that repeatability, the most important property, can be reached in two ways: by invariance (model large deformations mathematically and design the detector to ignore them) or by robustness (tolerate small ones such as noise, blur and compression). They also note that some properties compete: distinctiveness and locality cannot both be maximized, because a more local feature sees less intensity pattern and is harder to match correctly. Which compromise is right depends on the application. The two speed-oriented descendants of SIFT are good examples of trading accuracy for time: SURF [7] approximates Gaussian derivatives with box filters on integral images, and ORB [8] combines the FAST corner test with an oriented binary descriptor compared by Hamming distance, fast enough for phones.

Texture

Liu et al. [11] review roughly two decades of texture representation for classification, organized into bag-of-words pipelines (local descriptors, coding and pooling), CNN-based methods, and attribute-based methods. Co-occurrence statistics [3] sit at the root of this history: they are the first widely used descriptors of pairs of pixels. The survey, which covers more than 200 publications, traces how the field moved from hand-designed local descriptors encoded as bags of words to representations built on CNN features, and it closes with open questions and directions for future work.

Image matching, handcrafted to deep

Ma et al. [12] follow the full matching pipeline (detection, description, and matching with outlier removal) from handcrafted methods to trainable ones, and compare them experimentally on standard datasets. They show that learning has entered every stage of the pipeline, not only the descriptor. Jin et al. [13] built a benchmark that scores methods by the accuracy of the final camera poses rather than by intermediate metrics. Their key finding is a cautionary one: once each method’s settings (ratio-test threshold, RANSAC parameters, number of keypoints) are tuned properly, classical pipelines such as SIFT can still beat methods perceived as the state of the art, which suggests that some reported gains reflected untuned baselines.

Reading these papers critically

Separate a detector’s repeatability from a descriptor’s discriminative power and a matcher’s geometric verification. A pipeline can fail before descriptor comparison if the same point is not detected, or after comparison if repeated texture produces convincing but incorrect candidates. Shape, texture and local-feature surveys organize different units of evidence; their taxonomies should not be collapsed into one universal feature ranking.

A meaningful comparison states the image transformation, visibility, localization tolerance and matching protocol. Rotation or scale invariance can improve repeatability while discarding information that distinguishes similar structures. Learned matching can exploit context yet still depend on training scenes, keypoint density and outlier handling. Use repeatability, correspondence precision and downstream geometric success as separate diagnostics. The surveys locate assumptions and historical alternatives; the original benchmark protocols determine which reported results can be compared. For this chapter, the next step is to trace one correspondence through detection, description, matching and verification rather than attributing the entire outcome to a descriptor name.

References

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 12. publisher page
  2. M.-K. Hu, “Visual pattern recognition by moment invariants,” IRE Transactions on Information Theory, vol. 8, no. 2, pp. 179–187, 1962. doi
  3. R. M. Haralick, K. Shanmugam, and I. Dinstein, “Textural features for image classification,” IEEE Transactions on Systems, Man, and Cybernetics, vol. SMC-3, no. 6, pp. 610–621, 1973. doi
  4. C. Harris and M. Stephens, “A combined corner and edge detector,” in Proc. Alvey Vision Conference, pp. 23.1–23.6, 1988. doi
  5. J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust wide-baseline stereo from maximally stable extremal regions,” Image and Vision Computing, vol. 22, no. 10, pp. 761–767, 2004. doi
  6. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004. doi
  7. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust features (SURF),” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359, 2008. doi
  8. E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, “ORB: An efficient alternative to SIFT or SURF,” in Proc. IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571, 2011. doi
  9. D. Zhang and G. Lu, “Review of shape representation and description techniques,” Pattern Recognition, vol. 37, no. 1, pp. 1–19, 2004. doi
  10. T. Tuytelaars and K. Mikolajczyk, “Local invariant feature detectors: A survey,” Foundations and Trends in Computer Graphics and Vision, vol. 3, no. 3, pp. 177–280, 2008. doi
  11. L. Liu, J. Chen, P. Fieguth, G. Zhao, R. Chellappa, and M. Pietikäinen, “From BoW to CNN: Two decades of texture representation for texture classification,” International Journal of Computer Vision, vol. 127, pp. 74–109, 2019. arXiv
  12. J. Ma, X. Jiang, A. Fan, J. Jiang, and J. Yan, “Image matching from handcrafted to deep features: A survey,” International Journal of Computer Vision, vol. 129, pp. 23–79, 2021. doi
  13. Y. Jin, D. Mishkin, A. Mishchuk, J. Matas, P. Fua, K. M. Yi, and E. Trulls, “Image matching across wide baselines: From paper to practice,” International Journal of Computer Vision, 2021. doi · arXiv
  14. D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperPoint: Self-supervised interest point detection and description,” arXiv:1712.07629, 2017. arXiv
  15. P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “SuperGlue: Learning feature matching with graph neural networks,” arXiv:1911.11763, 2019. arXiv
  16. M. J. Tyszkiewicz, P. Fua, and E. Trulls, “DISK: Learning local features with policy gradient,” arXiv:2006.13566, 2020. arXiv
  17. J. Sun, Z. Shen, Y. Wang, H. Bao, and X. Zhou, “LoFTR: Detector-free local feature matching with Transformers,” arXiv:2104.00680, 2021. arXiv
  18. P. Lindenberger, P.-E. Sarlin, and M. Pollefeys, “LightGlue: Local feature matching at light speed,” arXiv:2306.13643, 2023. arXiv
  19. M. Oquab, T. Darcet, T. Moutakanni, et al., “DINOv2: Learning robust visual features without supervision,” arXiv:2304.07193, 2023. arXiv
  20. J. Edstedt, Q. Sun, G. Bökman, M. Wadenbäck, and M. Felsberg, “RoMa: Robust dense feature matching,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. arXiv
  21. M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981. doi
  22. scikit-image contributors, “skimage.measure.ransac,” scikit-image 0.26 documentation (API reference). docs