Guide to the Modern view papers
Optional · paper guide 1 of 1 for IP 13 · Image Pattern Classification. Citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
Choose a reading by topic: Statistical models, Learned features, Transfer, Evaluation.
The Modern view of this chapter rests on the following reviews and studies.

Statistical pattern recognition before deep learning
Jain, Duin and Mao’s review [2] is still the clearest map of the classical territory: representation, feature extraction and selection, Bayes and nearest-neighbor rules, linear and nonlinear classifiers, classifier combination, and error estimation. Its main takeaway, that the feature representation and the evaluation protocol matter more than the choice of classifier, aged well. Deep learning did not refute it; it automated the representation part.
The CNN era
Rawat and Wang [11] review CNNs for image classification from their origins in the late 1980s to 2017, organizing more than 300 papers by architecture, regularization, optimization and practical tricks. The line they trace starts with LeCun et al.’s gradient-trained networks for document recognition [6]. It accelerates with AlexNet [7], which used GPUs, ReLU, dropout and data augmentation to win the ImageNet Large Scale Visual Recognition Challenge in 2012 [10]. VGG [8] showed that depth built from small kernels helps. ResNet [9] made very deep networks trainable by learning residual functions through identity shortcut connections. Each of these is still recognizably the architecture of Figure 13.6: convolution, nonlinearity, pooling, a linear classifier, trained by backpropagation.
Transformers in vision
The Vision Transformer (ViT) [12] cuts an image into patches, treats each patch as a token and processes the sequence with self-attention instead of convolution. With enough pre-training data it matches or exceeds CNNs. Khan et al.’s survey [13] covers the many vision transformer variants for classification, detection, segmentation and more, and discusses their dependence on large-scale pre-training. ConvNeXt [14] then showed that a pure CNN, modernized step by step with design choices borrowed from transformers, is competitive again. The practical lesson: architecture families matter less than training data, scale and recipe.
Foundation features and zero-shot classification
Two 2021–2023 developments changed what “training a classifier” means. CLIP [15] trains an image encoder and a text encoder together so that matching image–caption pairs have similar embeddings. A new classifier can then be built without any labelled images: embed prompts such as a photo of a {class}, and assign each image to the class whose text embedding is most similar. That is a minimum-distance classifier whose prototypes come from language. DINOv2 [16] learns general-purpose visual features by self-supervision on a large curated image set, without labels. Its frozen features support strong classification with only a linear layer or a nearest-neighbor rule on top. In both cases the expensive part, the feature extractor, is trained once. The classifier on top is often one of the simple rules from the first half of this chapter.
Data augmentation and regularization
Shorten and Khoshgoftaar’s survey [18] organizes augmentation methods into basic image manipulations (geometric and color transformations, kernel filters, mixing images, random erasing) and learned approaches (adversarial training, GAN-based synthesis, neural style transfer, meta-learning of augmentation policies). Its takeaway matches Figure 13.9: augmentation is one of the cheapest and most effective ways to reduce overfitting when labelled data are limited.
Evaluation pitfalls and dataset bias
Torralba and Efros [19] asked a simple question: can a classifier tell which dataset an image came from? It could, well above chance, showing that each benchmark has its own “signature”. Models trained on one dataset also generalized poorly to another. Liu and He [20] repeated the experiment with modern networks and very large, diverse web-scale datasets. Modern networks still identify the source dataset with high accuracy on held-out images, and they do it with generalizable features rather than memorization. Dataset bias did not disappear with scale. Two practical consequences follow. Accuracy on one benchmark is a statement about that benchmark, not about the visual world. And whenever possible, test on data collected independently of the training data.
Reading these papers critically
The historical change is not simply replacing one final classifier with another. Deep models change the learned representation, available pretraining data and optimization procedure; foundation features change how labels or text can define the decision problem. To compare with a minimum-distance or statistical classifier, first specify whether both systems receive the same feature representation. Otherwise the result evaluates an entire pipeline change.
The benchmark papers in this guide address the reliability of the evidence itself. New test sets probe dataset shift; revised labels probe ambiguity or annotation mistakes; repeated runs probe optimization variability. None alone measures every kind of generalization. A clean comparison records pretraining overlap, adaptation data, model-selection rules and label conventions. Survey taxonomies help identify a method family, but a chronological list of architectures cannot establish which component caused an accuracy gain. Read the representation papers with the evaluation papers, then choose an experiment that separates the particular hypothesis you want to test.
References
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 13. publisher page
- A. K. Jain, R. P. W. Duin, and J. Mao, “Statistical Pattern Recognition: A Review,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4–37, 2000. doi:10.1109/34.824819
- D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, vol. 60, 2004. doi:10.1023/B:VISI.0000029664.99615.94
- F. Rosenblatt, “The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain,” Psychological Review, vol. 65, 1958. doi:10.1037/h0042519
- D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning Representations by Back-Propagating Errors,” Nature, vol. 323, 1986. doi:10.1038/323533a0
- Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-Based Learning Applied to Document Recognition,” Proceedings of the IEEE, vol. 86, 1998. doi:10.1109/5.726791
- A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet Classification with Deep Convolutional Neural Networks,” Advances in Neural Information Processing Systems 25 (NIPS), 2012. paper
- K. Simonyan and A. Zisserman, “Very Deep Convolutional Networks for Large-Scale Image Recognition,” arXiv:1409.1556, 2014. arXiv
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” arXiv:1512.03385, 2015. arXiv
- O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, et al., “ImageNet Large Scale Visual Recognition Challenge,” arXiv:1409.0575, 2014. arXiv
- W. Rawat and Z. Wang, “Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review,” Neural Computation, vol. 29, no. 9, 2017. doi:10.1162/neco_a_00990
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” ICLR, 2021. arXiv
- S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, “Transformers in Vision: A Survey,” ACM Computing Surveys, 2022. arXiv
- Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s,” CVPR, 2022. arXiv
- A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, et al., “Learning Transferable Visual Models From Natural Language Supervision,” arXiv:2103.00020, 2021. arXiv
- M. Oquab, T. Darcet, T. Moutakanni, et al., “DINOv2: Learning Robust Visual Features without Supervision,” arXiv:2304.07193, 2023. arXiv
- N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” Journal of Machine Learning Research, vol. 15, pp. 1929–1958, 2014. JMLR
- C. Shorten and T. M. Khoshgoftaar, “A Survey on Image Data Augmentation for Deep Learning,” Journal of Big Data, vol. 6, 2019. doi:10.1186/s40537-019-0197-0
- A. Torralba and A. A. Efros, “Unbiased Look at Dataset Bias,” CVPR, 2011. paper page
- Z. Liu and K. He, “A Decade’s Battle on Dataset Bias: Are We There Yet?,” ICLR, 2025. arXiv