DMD:匹配教師定義的分布(CVPR 2024)
選讀・DLCV 12B・單步生成 的論文導讀(第 6/7 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
挑戰
DMD [16] 不要求每個隨機種子都對應教師同一張圖,而匹配輸出分布,允許學生使用不同的雜訊—影像映射。
生成器密度難以計算,參數更新又持續改變分布。只用固定教師分數,無法描述學生密度,還可能把樣本集中到少數模態。
方法

圖片來源: [16], arXiv:2311.18828v4, PDF 第 3 頁. 原始 PDF.
生成器。 由預訓練去雜訊器初始化,輸入隨機 及文字/類別條件 ,輸出 ,形狀沿用影像或潛在表示。確定性網路仍可因不同 產生多樣輸出。
加雜訊比較。 提供較平滑、重疊的分布空間,也讓擴散模型可估計分數。時間抽樣與權重控制不同尺度差異的影響。
目標分數。 凍結教師估計 ;圖中「real score」仍是預訓練模型估計,並非精確真實資料密度。
學生分數。 另一去雜訊器以目前生成樣本及已知新增雜訊訓練,估計 。學生變動時也要更新它;生成器用停止梯度的兩分數差取得方向,不必反傳教師的完整 Jacobian。
分布梯度。 實際更新正比於 。 調整雜訊程度貢獻。學生分數修正其已有機率質量,因此不同於只最大化每張圖的教師似然。
迴歸正規化。 原始 DMD 另用確定性教師產生雜訊—端點配對,以重建/感知損失穩定訓練、維持覆蓋。因此原方法同時有分布自由度與配對限制,不能稱為完全沒有軌跡監督。
交替更新。 分數網路學目前樣本密度;生成器以分布梯度及端點項更新;教師不動。部署只保留生成器及必要解碼器,輔助分數網路是訓練成本。
結果
ImageNet-64 的一步 FID 為 2.62。零樣本 COCO-30k、512 解析度的學生 FID 11.49、時間 0.09 秒;原文教師列為 8.78、2.59 秒。此時間依硬體與引導設定而定,快速學生仍未匹配每個教師品質分數。
原文另研究較高引導訓練,須保留各列的引導與初始化。不能把一列品質配上另一列延遲;條件與分布品質也可能隨引導往不同方向變動。
消融實驗
表 2:CIFAR-10 移除分布匹配為 FID 3.82、移除迴歸 5.58、完整 2.66;ImageNet 對應為 9.21/5.61/2.62。此配方下兩項皆有貢獻。
其他時間權重的 CIFAR FID 為 3.60、3.71,預設 2.66,顯示雜訊程度的梯度尺度重要。移除迴歸也出現多樣性下降,但不代表理想分布目標必然需要迴歸;近似分數與更新動態也是因素,DMD2 [19] 進一步檢驗。
結論
教師分數與學得的學生分數提供分布方向;原始端點迴歸能穩定訓練,同時限制分布目標原本允許的部分自由度。
限制與問題
小生成器損失不代表輔助分數準確。應分別監測樣本分布及去雜訊擬合,注意追蹤落後、教師偏差、有限雜訊與乾淨覆蓋的差異。離線端點要付出儲存與教師取樣成本,兩個分數網路的條件/引導也必須相符。DMD2 針對穩定性移除端點資料,並加入真實影像訊號。
參考文獻
- Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” ICLR, 2021. paper
- Y. Lipman, M. Havasi, P. Holderrieth, et al., “Flow Matching Guide and Code,” arXiv tutorial, 2024. paper
- T. Karras, M. Aittala, T. Aila, S. Laine, “Elucidating the Design Space of Diffusion-Based Generative Models,” NeurIPS, 2022. source
- X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
- J. Song, C. Meng, S. Ermon, “Denoising Diffusion Implicit Models,” ICLR, 2021. paper
- C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, J. Zhu, “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps,” NeurIPS, 2022. paper
- T. Salimans, J. Ho, “Progressive Distillation for Fast Sampling of Diffusion Models,” ICLR, 2022. paper
- Y. Song, P. Dhariwal, M. Chen, I. Sutskever, “Consistency Models,” ICML, 2023. paper
- Y. Song, P. Dhariwal, “Improved Techniques for Training Consistency Models,” ICLR, 2024. paper
- S. Luo, Y. Tan, L. Huang, J. Li, H. Zhao, “Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference,” arXiv preprint, 2023. paper
- R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
- X. Liu, C. Gong, Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” ICLR, 2023. paper
- X. Liu, X. Zhang, J. Ma, J. Peng, Q. Liu, “InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation,” ICLR, 2024. paper
- A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
- A. Sauer, F. Boesel, T. Dockhorn, A. Blattmann, P. Esser, R. Rombach, “Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation,” SIGGRAPH Asia Conference Papers, 2024. paper · published version
- T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, T. Park, “One-step Diffusion with Distribution Matching Distillation,” CVPR, 2024. paper
- T. H. Nguyen, A. Tran, “SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation,” CVPR, 2024. paper
- M. Zhou, H. Zheng, Z. Wang, M. Yin, H. Huang, “Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation,” ICML, 2024. paper
- T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, “Improved Distribution Matching Distillation for Fast Image Synthesis,” NeurIPS, 2024. paper
- K. Frans, D. Hafner, S. Levine, P. Abbeel, “One Step Diffusion via Shortcut Models,” ICLR, 2025. paper
- Z. Geng, M. Deng, X. Bai, J. Z. Kolter, K. He, “Mean Flows for One-step Generative Modeling,” NeurIPS, 2025. paper
- M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS, 2017. source
- A. Radford, J. W. Kim, C. Hallacy, et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. paper
- D. Ghosh, H. Hajishirzi, L. Schmidt, “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment,” NeurIPS Datasets and Benchmarks, 2023. paper
- A. Krizhevsky, V. Nair and G. Hinton, “CIFAR-10 and CIFAR-100 datasets,” official dataset page; companion technical report: A. Krizhevsky, “Learning Multiple Layers of Features from Tiny Images,” 2009. dataset and report
- J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
- T. Lin, M. Maire, S. Belongie, et al., “Microsoft COCO: Common Objects in Context,” ECCV, 2014. paper
- L. Yang, Z. Zhang, Y. Song, et al., “Diffusion Models: A Comprehensive Survey of Methods and Applications,” ACM Computing Surveys, 2023. paper
- Z. Yu, J. Yuan, H. Yang, et al., “DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation,” arXiv preprint, 2026. paper
- R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
- C. Schuhmann, R. Beaumont, R. Vencu, et al., “LAION-5B: An open large-scale dataset for training next generation image-text models,” NeurIPS Datasets and Benchmarks, 2022. paper