Guide to the Modern view surveys
Optional · paper guide 1 of 1 for DLCV 01 · Deep Learning for Image Processing and Computer Vision. Citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
Choose a reading by topic: Shape, Local keypoints, Texture, Matching.

CNN architectures
Li et al. [30] review CNNs from their history through the convolution operation itself, classic and advanced architectures (the LeNet–ResNet line, lightweight and attention-augmented networks), 1-D, 2-D and multi-dimensional convolutions, and applications. They also cover the improvements made to each component (layer design, activation and loss functions, regularization, optimization, fast computation), run experiments to derive rules of thumb for choosing functions and hyperparameters, and close with open issues and promising directions. It is the best single place to see the CNN design space laid out component by component.
Vision Transformers
Khan et al. [31] (ACM Computing Surveys) organize Transformer work by task: the foundations (self-attention, large-scale pretraining, bidirectional encoding), then recognition (classification, detection, segmentation, action recognition), generative modeling, multimodal tasks (VQA, visual reasoning, grounding), video, low-level vision (super-resolution, enhancement, colorization) and 3-D point clouds. Their central contrast matches this tutorial’s: Transformers model long-range dependencies with minimal inductive bias, so they need large-scale pretraining or architectural priors (windows, hierarchies) to be data-efficient.
Self-supervised visual learning
Jing and Tian [32] (TPAMI) survey the pre-2020 generation of self-supervised methods, organized by pretext task (generation-based, context-based, free-semantic-label-based and cross-modal), along with the standard protocol of evaluating features by transfer to downstream tasks. Balestriero et al. [33] cover the modern generation in a practical “cookbook” and group methods into four families: deep metric learning (contrastive), self-distillation (the DINO family), canonical-correlation-style methods, and masked image modeling (MAE). Their emphasis on the many hidden “knobs” (augmentations, projector heads, collapse prevention) explains why reproducing self-supervised results is hard.
Foundation models for vision
Awais et al. [34] (TPAMI) categorize vision foundation models by modality pairing (vision with text, audio, depth), training objective (contrastive versus generative) and prompt type: textual, visual (points, boxes, masks, as in SAM) and heterogeneous. Their open challenges are evaluation and benchmarking, real-world understanding and contextual reasoning, bias, adversarial robustness and interpretability. Zhang et al. [35] (TPAMI) focus on vision–language models for visual recognition: architectures and pretraining objectives, transfer methods (prompt tuning, adapters) and knowledge distillation into detectors and segmenters. They argue that web-scale image–text data reduces the dependence on crowd-sourced labels and lets one model serve many tasks.
Reading these papers critically
These categories answer different levels of the design problem. A CNN or transformer names an architectural family; self-supervision names how a learning signal is constructed; a foundation model describes reusable pretrained capability and its adaptation. A transformer can be self-supervised and serve as a foundation encoder, so the categories are not mutually exclusive competitors. Reading them as one ranking confuses architecture, objective and scale.
For a reported transfer result, track the pretraining data, augmentations, projector or prediction heads, whether the encoder is frozen, and which downstream labels are used. A linear probe and full fine-tuning measure different adaptation budgets. Survey tables collect useful context but rarely control every one of these variables across all entries. The strongest practical use is to locate a design choice, understand its proposed mechanism and then inspect the original paper’s controlled comparisons. Open problems such as data efficiency, bias and robustness remain separate evaluation questions even when one shared model supports many tasks.
References
- Shyandram, “深度學習的數位影像處理介紹 (An introduction to digital image processing with deep learning),” Medium post, 2024; updated 2026. link
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018. publisher page
- S. Theodoridis and K. Koutroumbas, Pattern Recognition, 4th ed., Academic Press, 2008. publisher page
- Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, “Backpropagation applied to handwritten zip code recognition,” Neural Computation, vol. 1, no. 4, pp. 541–551, 1989. doi:10.1162/neco.1989.1.4.541
- Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998. doi:10.1109/5.726791
- O. Russakovsky, J. Deng, H. Su, et al., “ImageNet large scale visual recognition challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015. doi:10.1007/s11263-015-0816-y · arXiv:1409.0575
- A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” NeurIPS, 2012. proceedings
- K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” ICLR, 2015. arXiv:1409.1556
- C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” CVPR, 2015. doi:10.1109/CVPR.2015.7298594 · arXiv:1409.4842
- K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” CVPR, 2016. doi:10.1109/CVPR.2016.90 · arXiv:1512.03385
- A. Krizhevsky, “Learning multiple layers of features from tiny images,” technical report, University of Toronto, 2009 (the CIFAR-10/100 datasets). dataset page
- W. Luo, Y. Li, R. Urtasun, and R. Zemel, “Understanding the effective receptive field in deep convolutional neural networks,” NeurIPS, 2016. arXiv:1701.04128
- J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” CVPR, 2015. arXiv:1411.4038
- O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” MICCAI, 2015. arXiv:1505.04597
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” NeurIPS, 2017. ML anthology
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” ICLR, 2021. arXiv:2010.11929
- Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical vision Transformer using shifted windows,” ICCV, 2021. doi:10.1109/ICCV48922.2021.00986 · arXiv:2103.14030
- Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A ConvNet for the 2020s,” CVPR, 2022, arXiv:2201.03545; and S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie, “ConvNeXt V2: Co-designing and scaling ConvNets with masked autoencoders,” CVPR, 2023, arXiv:2301.00808.
- M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision Transformers,” ICCV, 2021. doi:10.1109/ICCV48922.2021.00951 · arXiv:2104.14294
- K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” CVPR, 2022. doi:10.1109/CVPR52688.2022.01553 · arXiv:2111.06377
- M. Oquab, T. Darcet, T. Moutakanni, et al., “DINOv2: Learning robust visual features without supervision,” Transactions on Machine Learning Research, 2024, arXiv:2304.07193; and O. Siméoni, H. V. Vo, M. Seitzer, F. Baldassarre, M. Oquab, et al., “DINOv3,” arXiv preprint, 2025, arXiv:2508.10104.
- L. Zhu, B. Liao, Q. Zhang, X. Wang, W. Liu, and X. Wang, “Vision Mamba: Efficient visual representation learning with bidirectional state space model,” ICML, 2024, arXiv:2401.09417; and Y. Liu, Y. Tian, Y. Zhao, H. Yu, L. Xie, Y. Wang, Q. Ye, J. Jiao, and Y. Liu, “VMamba: Visual state space model,” NeurIPS, 2024, arXiv:2401.10166.
- S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards real-time object detection with region proposal networks,” NeurIPS, 2015, arXiv:1506.01497; and N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with Transformers,” ECCV, 2020, arXiv:2005.12872.
- A. Kirillov, E. Mintun, N. Ravi, et al., “Segment anything,” ICCV, 2023, arXiv:2304.02643; N. Ravi, V. Gabeur, Y.-T. Hu, R. Hu, et al., “SAM 2: Segment anything in images and videos,” ICLR, 2025, arXiv:2408.00714; and N. Carion, L. Gustafson, Y.-T. Hu, S. Debnath, et al., “SAM 3: Segment anything with concepts,” ICLR, 2026, arXiv:2511.16719.
- A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, et al., “Learning transferable visual models from natural language supervision,” ICML, 2021. arXiv:2103.00020
- H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” NeurIPS, 2023. arXiv:2304.08485
- J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” NeurIPS, 2020. arXiv:2006.11239
- R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” CVPR, 2022, arXiv:2112.10752; and W. Peebles and S. Xie, “Scalable diffusion models with Transformers,” ICCV, 2023, arXiv:2212.09748.
- T. Brooks, A. Holynski, and A. A. Efros, “InstructPix2Pix: Learning to follow image editing instructions,” CVPR, 2023. arXiv:2211.09800
- Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: Analysis, applications, and prospects,” IEEE Transactions on Neural Networks and Learning Systems, vol. 33, 2022. doi:10.1109/TNNLS.2021.3084827 · arXiv:2004.02806
- S. Khan, M. Naseer, M. Hayat, S. W. Zamir, F. S. Khan, and M. Shah, “Transformers in vision: A survey,” ACM Computing Surveys, vol. 54, no. 10s, 2022. doi:10.1145/3505244 · arXiv:2101.01169
- L. Jing and Y. Tian, “Self-supervised visual feature learning with deep neural networks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, 2021. doi:10.1109/TPAMI.2020.2992393 · arXiv:1902.06162
- R. Balestriero, M. Ibrahim, V. Sobal, et al., “A cookbook of self-supervised learning,” arXiv preprint, 2023. arXiv:2304.12210
- M. Awais, M. Naseer, S. Khan, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, and F. S. Khan, “Foundation models defining a new era in vision: A survey and outlook,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, 2025. doi:10.1109/TPAMI.2024.3506283 · arXiv:2307.13721
- J. Zhang, J. Huang, S. Jin, and S. Lu, “Vision-language models for vision tasks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, 2024. doi:10.1109/TPAMI.2024.3369699 · arXiv:2304.00685
- scikit-learn developers, “sklearn.datasets.load_digits” (a copy of the test set of the UCI optical recognition of handwritten digits dataset). docs
- S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” ICML, PMLR 37, pp. 448–456, 2015. PMLR · arXiv:1502.03167
- S. Santurkar, D. Tsipras, A. Ilyas, and A. Madry, “How does batch normalization help optimization?” NeurIPS, vol. 31, 2018. proceedings · arXiv:1805.11604
- G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” NIPS 2014 Deep Learning Workshop, arXiv preprint, 2015. arXiv:1503.02531