Paper guides
Optional reading guides to the papers behind DLCV 12B · One-Step Generation, one page each: the problem, the idea, what it showed and what to check when you read it.
Navigate: this page and this track: Navigate: this page and this track
On this page
- Progressive Distillation: two teacher moves in one student move (ICLR 2022)
- Consistency Models: the same endpoint along a path (ICML 2023)
- InstaFlow: reflow, then endpoint distillation (ICLR 2024)
- Latent Consistency Models: endpoints in a guided latent flow (2023)
- Adversarial Diffusion Distillation: a teacher and a discriminator (ECCV 2024)
- DMD: matching a teacher-defined distribution (CVPR 2024)
- DMD2: tracking the student and using real data (NeurIPS 2024)
Deep Learning for Computer Vision
These optional introductions explain the design, evidence and limitations of the key papers. Their benchmark values are quoted from the papers, not reproduced by the small experiments in this chapter.
- Progressive Distillation: two teacher moves in one student move (ICLR 2022)
An accurate diffusion sampler can require many network evaluations.
- Consistency Models: the same endpoint along a path (ICML 2023)
A diffusion sampler walks along a probability-flow trajectory to a near-clean endpoint.
- InstaFlow: reflow, then endpoint distillation (ICLR 2024)
Directly regressing a one-step student onto a text-to-image diffusion sampler can be difficult.
- Latent Consistency Models: endpoints in a guided latent flow (2023)
Pixel-space consistency models demonstrate fast generation, but high-resolution text-to-image synthesis commonly uses a pretrained latent diffusion model.
- Adversarial Diffusion Distillation: a teacher and a discriminator (ECCV 2024)
A student trained only to imitate diffusion predictions may fail to generate sharp, realistic images at very small step counts.
- DMD: matching a teacher-defined distribution (CVPR 2024)
Endpoint distillation ties a student to the teacher’s image for each random seed.
- DMD2: tracking the student and using real data (NeurIPS 2024)
DMD’s endpoint regularizer stabilizes training but requires expensive teacher-generated pairs and partly binds the student to a selected teacher map.
References
- Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” ICLR, 2021. paper
- Y. Lipman, M. Havasi, P. Holderrieth, et al., “Flow Matching Guide and Code,” arXiv tutorial, 2024. paper
- T. Karras, M. Aittala, T. Aila, S. Laine, “Elucidating the Design Space of Diffusion-Based Generative Models,” NeurIPS, 2022. source
- X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
- J. Song, C. Meng, S. Ermon, “Denoising Diffusion Implicit Models,” ICLR, 2021. paper
- C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, J. Zhu, “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps,” NeurIPS, 2022. paper
- T. Salimans, J. Ho, “Progressive Distillation for Fast Sampling of Diffusion Models,” ICLR, 2022. paper
- Y. Song, P. Dhariwal, M. Chen, I. Sutskever, “Consistency Models,” ICML, 2023. paper
- Y. Song, P. Dhariwal, “Improved Techniques for Training Consistency Models,” ICLR, 2024. paper
- S. Luo, Y. Tan, L. Huang, J. Li, H. Zhao, “Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference,” arXiv preprint, 2023. paper
- R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
- X. Liu, C. Gong, Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” ICLR, 2023. paper
- X. Liu, X. Zhang, J. Ma, J. Peng, Q. Liu, “InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation,” ICLR, 2024. paper
- A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
- A. Sauer, F. Boesel, T. Dockhorn, A. Blattmann, P. Esser, R. Rombach, “Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation,” SIGGRAPH Asia Conference Papers, 2024. paper · published version
- T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, T. Park, “One-step Diffusion with Distribution Matching Distillation,” CVPR, 2024. paper
- T. H. Nguyen, A. Tran, “SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation,” CVPR, 2024. paper
- M. Zhou, H. Zheng, Z. Wang, M. Yin, H. Huang, “Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation,” ICML, 2024. paper
- T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, “Improved Distribution Matching Distillation for Fast Image Synthesis,” NeurIPS, 2024. paper
- K. Frans, D. Hafner, S. Levine, P. Abbeel, “One Step Diffusion via Shortcut Models,” ICLR, 2025. paper
- Z. Geng, M. Deng, X. Bai, J. Z. Kolter, K. He, “Mean Flows for One-step Generative Modeling,” NeurIPS, 2025. paper
- M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS, 2017. source
- A. Radford, J. W. Kim, C. Hallacy, et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. paper
- D. Ghosh, H. Hajishirzi, L. Schmidt, “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment,” NeurIPS Datasets and Benchmarks, 2023. paper
- A. Krizhevsky, V. Nair and G. Hinton, “CIFAR-10 and CIFAR-100 datasets,” official dataset page; companion technical report: A. Krizhevsky, “Learning Multiple Layers of Features from Tiny Images,” 2009. dataset and report
- J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
- T. Lin, M. Maire, S. Belongie, et al., “Microsoft COCO: Common Objects in Context,” ECCV, 2014. paper
- L. Yang, Z. Zhang, Y. Song, et al., “Diffusion Models: A Comprehensive Survey of Methods and Applications,” ACM Computing Surveys, 2023. paper
- Z. Yu, J. Yuan, H. Yang, et al., “DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation,” arXiv preprint, 2026. paper
- R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
- C. Schuhmann, R. Beaumont, R. Vencu, et al., “LAION-5B: An open large-scale dataset for training next generation image-text models,” NeurIPS Datasets and Benchmarks, 2022. paper