完整搜尋頁面 →

論文導讀

選讀:DLCV 12B・單步生成 所依據論文的導讀,每篇一頁:說明它要解決的問題、核心想法、結果,以及閱讀時該注意什麼。

導覽:本頁與本系列: 導覽:本頁與本系列

本頁內容

補充資料 · 2

  1. 瞬時速度與區間速度
  2. 無教師的三種意思

論文導讀 · 7

  1. Progressive Distillation:教師兩步壓成學生一步(ICLR 2022)
  2. Consistency Models:沿同一路徑預測相同端點(ICML 2023)
  3. InstaFlow:先 reflow,再蒸餾端點(ICLR 2024)
  4. Latent Consistency Models:受引導潛在流的端點(2023)
  5. Adversarial Diffusion Distillation:教師與判別器(ECCV 2024)
  6. DMD:匹配教師定義的分布(CVPR 2024)
  7. DMD2:追蹤學生,加入真實資料(NeurIPS 2024)

電腦視覺深度學習

  1. 02 · 深度影像分類
  2. 03 · 物件偵測:從 R-CNN 到 DETR
  3. 09 · 深度影像復原:低階視覺
查看完整課程樹 →

以下選讀說明重要論文的設計、證據與限制。基準數字來自原文,不是本章小實驗重現的成績。

  1. Progressive Distillation:教師兩步壓成學生一步(ICLR 2022)

    準確的擴散取樣器常需多次網路評估。

  2. Consistency Models:沿同一路徑預測相同端點(ICML 2023)

    Consistency Models [8] 讓網路從機率流路徑任意位置預測近乾淨端點,將單步生成改為直接預測;預算較多時也能多次細化。

  3. InstaFlow:先 reflow,再蒸餾端點(ICLR 2024)

    文字生成教師的雜訊—影像映射可能高度彎曲、難以直接迴歸。

  4. Latent Consistency Models:受引導潛在流的端點(2023)

    LCM [10] 將一致性蒸餾移到預訓練潛在擴散系統,並把無分類器引導納入學生。

  5. Adversarial Diffusion Distillation:教師與判別器(ECCV 2024)

    少步時只模仿擴散預測,可能產生不夠清晰的影像;大型對抗生成器從頭訓練也困難。

  6. DMD:匹配教師定義的分布(CVPR 2024)

    DMD [16] 不要求每個隨機種子都對應教師同一張圖,而匹配輸出分布,允許學生使用不同的雜訊—影像映射。

  7. DMD2:追蹤學生,加入真實資料(NeurIPS 2024)

    DMD2 [19] 問:移除昂貴的教師端點資料後,為什麼分布匹配會不穩?

參考文獻

  1. Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” ICLR, 2021. paper
  2. Y. Lipman, M. Havasi, P. Holderrieth, et al., “Flow Matching Guide and Code,” arXiv tutorial, 2024. paper
  3. T. Karras, M. Aittala, T. Aila, S. Laine, “Elucidating the Design Space of Diffusion-Based Generative Models,” NeurIPS, 2022. source
  4. X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
  5. J. Song, C. Meng, S. Ermon, “Denoising Diffusion Implicit Models,” ICLR, 2021. paper
  6. C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, J. Zhu, “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps,” NeurIPS, 2022. paper
  7. T. Salimans, J. Ho, “Progressive Distillation for Fast Sampling of Diffusion Models,” ICLR, 2022. paper
  8. Y. Song, P. Dhariwal, M. Chen, I. Sutskever, “Consistency Models,” ICML, 2023. paper
  9. Y. Song, P. Dhariwal, “Improved Techniques for Training Consistency Models,” ICLR, 2024. paper
  10. S. Luo, Y. Tan, L. Huang, J. Li, H. Zhao, “Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference,” arXiv preprint, 2023. paper
  11. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
  12. X. Liu, C. Gong, Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” ICLR, 2023. paper
  13. X. Liu, X. Zhang, J. Ma, J. Peng, Q. Liu, “InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation,” ICLR, 2024. paper
  14. A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
  15. A. Sauer, F. Boesel, T. Dockhorn, A. Blattmann, P. Esser, R. Rombach, “Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation,” SIGGRAPH Asia Conference Papers, 2024. paper · published version
  16. T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, T. Park, “One-step Diffusion with Distribution Matching Distillation,” CVPR, 2024. paper
  17. T. H. Nguyen, A. Tran, “SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation,” CVPR, 2024. paper
  18. M. Zhou, H. Zheng, Z. Wang, M. Yin, H. Huang, “Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation,” ICML, 2024. paper
  19. T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, “Improved Distribution Matching Distillation for Fast Image Synthesis,” NeurIPS, 2024. paper
  20. K. Frans, D. Hafner, S. Levine, P. Abbeel, “One Step Diffusion via Shortcut Models,” ICLR, 2025. paper
  21. Z. Geng, M. Deng, X. Bai, J. Z. Kolter, K. He, “Mean Flows for One-step Generative Modeling,” NeurIPS, 2025. paper
  22. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS, 2017. source
  23. A. Radford, J. W. Kim, C. Hallacy, et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. paper
  24. D. Ghosh, H. Hajishirzi, L. Schmidt, “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment,” NeurIPS Datasets and Benchmarks, 2023. paper
  25. A. Krizhevsky, V. Nair and G. Hinton, “CIFAR-10 and CIFAR-100 datasets,” official dataset page; companion technical report: A. Krizhevsky, “Learning Multiple Layers of Features from Tiny Images,” 2009. dataset and report
  26. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
  27. T. Lin, M. Maire, S. Belongie, et al., “Microsoft COCO: Common Objects in Context,” ECCV, 2014. paper
  28. L. Yang, Z. Zhang, Y. Song, et al., “Diffusion Models: A Comprehensive Survey of Methods and Applications,” ACM Computing Surveys, 2023. paper
  29. Z. Yu, J. Yuan, H. Yang, et al., “DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation,” arXiv preprint, 2026. paper
  30. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
  31. C. Schuhmann, R. Beaumont, R. Vencu, et al., “LAION-5B: An open large-scale dataset for training next generation image-text models,” NeurIPS Datasets and Benchmarks, 2022. paper