Adversarial Diffusion Distillation:教師與判別器(ECCV 2024)
選讀・DLCV 12B・單步生成 的論文導讀(第 5/7 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
挑戰
少步時只模仿擴散預測,可能產生不夠清晰的影像;大型對抗生成器從頭訓練也困難。ADD [14] 結合預訓練初始化、凍結教師與對抗學習。
判別器提供外觀訊號,教師提供去雜訊先驗。兩者的互動須實驗驗證;清晰外觀不保證提示正確或完整保留模態。
方法

圖片來源: [14], arXiv:2311.17042v1, PDF 第 2 頁. 原始 PDF.
學生輸入。 依學生排程對真實圖加雜訊,從少量時間點抽樣。最大時間的訊號為零,涵蓋生成用的純雜訊邊界;中間時間保留來源資訊,支援少步細化。
初始化。 沿用預訓練去雜訊權重、時間與文字條件,使對抗目標不必從隨機參數發現整個生成任務。單步是此參數化的一個訓練操作點。
對抗路徑。 真實與生成圖經預訓練特徵網路,由多層輕量判別頭評分。可輸入文字條件,保留來源資訊的時間點也可輸入來源影像嵌入。判別器用 hinge 目標與正規化,學生提高真實感評分;評分不是獨立驗證的品質機率。
蒸餾路徑。 對學生乾淨預測重新加教師程度的雜訊,再交給凍結教師去雜訊;停止梯度的教師預測作為重建目標。教師期待吵的輸入,不能任意餵乾淨生成圖。原文在潛在系統採像素蒸餾比較取得穩定梯度,因此訓練包含解碼。
合併目標。 , 平衡兩種監督。教師凍結、判別頭更新、學生接收兩路梯度。停止梯度及不同時間權重會影響符合度與多樣性。
推論。 不執行教師或判別器;學生由雜訊預測,可在訓練時間點細化,再依需要解碼。訓練特徵網路不必包含在部署成本。
結果
消融採較小的預訓練教師/學生、COCO 2017 的 5,000 樣本、一步學生;預設 FID 20.6、CLIP 0.319。這不是每個 SDXL-Turbo 模型都適用的頭條成績。
ADD-XL 的一步人類偏好研究,並非兩個面向都優於 SDXL 教師;四步研究才報告優於比較教師設定。不能把四步結論改寫為一步,也不能由偏好直接推論罕見模態覆蓋。
消融實驗
表 1:隨機初始化為 FID/CLIP 293.6/0.065,預訓練為 20.6/0.319;只有蒸餾 315.6/0.076,只有對抗 20.8/0.315,指數加權合併 20.6/0.319。教師作用須連同其他設計解讀,不能假設它主導所有增益。
判別器沒有影像/文字條件時為 21.2/0.302,兩者都有為 20.6/0.319;更大特徵編碼器不必然更好。教師一步/兩步/四步目標的 FID 為 20.6/20.8/20.3,支持該設定的廉價目標,不代表所有長教師軌跡都無用。
結論
預訓練擴散學生可結合對抗外觀與教師去雜訊訊號學短程序;預訓練和條件設計都是系統的一部分。
限制與問題
對抗最佳化可能偏好漂亮卻重複的細節,仍須調整權重、資料與判別器。LADD [15] 改變訓練表示以降低特徵/解碼成本,但最終解碼器並未消失,也不保證不同規模的目標完全等價。
參考文獻
- Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” ICLR, 2021. paper
- Y. Lipman, M. Havasi, P. Holderrieth, et al., “Flow Matching Guide and Code,” arXiv tutorial, 2024. paper
- T. Karras, M. Aittala, T. Aila, S. Laine, “Elucidating the Design Space of Diffusion-Based Generative Models,” NeurIPS, 2022. source
- X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
- J. Song, C. Meng, S. Ermon, “Denoising Diffusion Implicit Models,” ICLR, 2021. paper
- C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, J. Zhu, “DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps,” NeurIPS, 2022. paper
- T. Salimans, J. Ho, “Progressive Distillation for Fast Sampling of Diffusion Models,” ICLR, 2022. paper
- Y. Song, P. Dhariwal, M. Chen, I. Sutskever, “Consistency Models,” ICML, 2023. paper
- Y. Song, P. Dhariwal, “Improved Techniques for Training Consistency Models,” ICLR, 2024. paper
- S. Luo, Y. Tan, L. Huang, J. Li, H. Zhao, “Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference,” arXiv preprint, 2023. paper
- R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
- X. Liu, C. Gong, Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” ICLR, 2023. paper
- X. Liu, X. Zhang, J. Ma, J. Peng, Q. Liu, “InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation,” ICLR, 2024. paper
- A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
- A. Sauer, F. Boesel, T. Dockhorn, A. Blattmann, P. Esser, R. Rombach, “Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation,” SIGGRAPH Asia Conference Papers, 2024. paper · published version
- T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, T. Park, “One-step Diffusion with Distribution Matching Distillation,” CVPR, 2024. paper
- T. H. Nguyen, A. Tran, “SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation,” CVPR, 2024. paper
- M. Zhou, H. Zheng, Z. Wang, M. Yin, H. Huang, “Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation,” ICML, 2024. paper
- T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, “Improved Distribution Matching Distillation for Fast Image Synthesis,” NeurIPS, 2024. paper
- K. Frans, D. Hafner, S. Levine, P. Abbeel, “One Step Diffusion via Shortcut Models,” ICLR, 2025. paper
- Z. Geng, M. Deng, X. Bai, J. Z. Kolter, K. He, “Mean Flows for One-step Generative Modeling,” NeurIPS, 2025. paper
- M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” NeurIPS, 2017. source
- A. Radford, J. W. Kim, C. Hallacy, et al., “Learning Transferable Visual Models From Natural Language Supervision,” ICML, 2021. paper
- D. Ghosh, H. Hajishirzi, L. Schmidt, “GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment,” NeurIPS Datasets and Benchmarks, 2023. paper
- A. Krizhevsky, V. Nair and G. Hinton, “CIFAR-10 and CIFAR-100 datasets,” official dataset page; companion technical report: A. Krizhevsky, “Learning Multiple Layers of Features from Tiny Images,” 2009. dataset and report
- J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
- T. Lin, M. Maire, S. Belongie, et al., “Microsoft COCO: Common Objects in Context,” ECCV, 2014. paper
- L. Yang, Z. Zhang, Y. Song, et al., “Diffusion Models: A Comprehensive Survey of Methods and Applications,” ACM Computing Surveys, 2023. paper
- Z. Yu, J. Yuan, H. Yang, et al., “DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation,” arXiv preprint, 2026. paper
- R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
- C. Schuhmann, R. Beaumont, R. Vencu, et al., “LAION-5B: An open large-scale dataset for training next generation image-text models,” NeurIPS Datasets and Benchmarks, 2022. paper