完整搜尋頁面 →

S3Diff:退化引導的低秩調適

選讀・DLCV 09A.2・單步擴散超解析度 的論文導讀(第 4/8 篇)。引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

本頁內容

補充資料 · 2

  1. 潛在起點不等於加雜訊方程式
  2. 分數正規化與逐圖量測一致性

論文導讀 · 8

  1. ResShift:復原條件的短鏈(NeurIPS 2023)
  2. SinSR:蒸餾確定性的影像條件映射(CVPR 2024)
  3. OSEDiff:LR 潛在起點與變分分數正規化(NeurIPS 2024)
  4. S3Diff:退化引導的低秩調適
  5. TSD-SR:目標感知分數正規化(CVPR 2025)
  6. InvSR:學得的影像條件起點(CVPR 2025)
  7. PiSA-SR:分開像素與語意調適(CVPR 2025)
  8. AddSR:預測細化的對抗復原蒸餾

電腦視覺深度學習

  1. 02 · 深度影像分類
  2. 03 · 物件偵測:從 R-CNN 到 DETR
  3. 09 · 深度影像復原:低階視覺
查看完整課程樹 →

挑戰

預訓練單步生成器不自動適合 SR。S3Diff [4] 依模糊與雜訊估計調整生成器,避免固定修正對不同影像不足或過度,也免用昂貴逐圖語意描述。

退化條件權重與正/負品質提示是兩個問題:前者調適,後者引導。模糊/雜訊向量不保證完整描述真實相機。

方法

退化估計調變低秩調適,正負 prompt 路徑提供引導;須區分訓練支路與部署時的單步路徑。
原論文圖 3 — 退化估計調變低秩調適,正負 prompt 路徑提供引導;須區分訓練支路與部署時的單步路徑。

圖片來源: [4], arXiv:2409.17058v1, PDF 第 5 頁. 原始 PDF.

先驗與影像路徑。 使用 ADD 衍生單步潛在先驗 [14],輸入 LR 表示、只加很少或不加雜訊;調適編碼器及去雜訊器,固定解碼器。編碼器可先抑制缺陷,解碼器保留預訓練表示。

低秩修正。 W∈Rm×nW\in\mathbb R^{m\times n}、A∈Rm×rA\in\mathbb R^{m\times r}、B∈Rr×nB\in\mathbb R^{r\times n}、秩 rr;將一般 W′=W+ABW'=W+AB 改為 W′=W+AC(d)BW'=W+AC(d)B,C(d)∈Rr×rC(d)\in\mathbb R^{r\times r} 依退化描述 dd 調制低秩方向。

退化與區塊識別。 預訓練估計器給兩個正規化模糊/雜訊值,經正弦/Fourier 嵌入與映射。區塊 ID 讓共用映射產生不同 Ci(d)C_i(d),介於所有區塊共用同一矩陣與每塊獨立大型預測器之間。

動態成本。 有效修正依輸入改變,不能像靜態轉接器一次合併;估計與調制仍要執行。手動輸入的退化值也是估計,不是已驗證物理量。

線上負目標。 訓練以 0.05 機率改用退化目標配負品質提示,其餘用正品質提示;重建/感知依選定目標,對抗監督作用於正 HR。如此學負外觀,不只依賴離線生成負例。

損失與引導。 L2、LPIPS、GAN 權重 2、5、0.5;判別器使用預訓練特徵和可訓練頭。正/負引導可能做兩次預測,須報兩次 NFE;原文選小引導以避免過度傷害保真。

訓練。 LSDIR 加 FFHQ 10,000 張 [32]、[33],Real-ESRGAN 式退化 [27];編碼器/去雜訊器秩 16/32。這些值屬開啟稿件的設定,不是所有低秩模型的通則。

結果

原文 DIV2K 驗證:PSNR 23.40、LPIPS 0.2571、DISTS 0.1730、FID 19.35、MUSIQ 68.21;RealSR 的 PSNR/LPIPS/FID 25.03/0.2699/108.88。其中 OSEDiff 重跑列與原論文不同,不能把不同規則拼成唯一標準分數。

改善涉及整組指標與影像,並非每資料集每項皆贏。保真基準可能 PSNR 較高,無參考指標也可能偏好不同外觀。

消融實驗

退化 LoRA 消融:共用修正矩陣在 DIV2K 的 LPIPS/MUSIQ 0.2613/65.52,區塊 ID 條件版 0.2563/66.55;RealSR 對應 0.2626/64.23→0.2573/65.78。

RealSR 線上負目標但無引導為 PSNR/MUSIQ 25.76/66.42,加引導 25.03/67.88,外觀偏好提高而參考保真下降。秩與編解碼消融也不支持容量越大、引導越強越好,成本須一併評估。

結論

退化估計可調制低秩復原器,線上負目標可免語意描述而學品質引導;兩者是不同作用。

限制與問題

估計器可能誤判,兩值描述也漏掉壓縮等效果。動態修正與引導增加工作;通用提示不解決細節歧義,小轉接器也沒移除大型骨幹。比較須固定版本、引導與規則。

參考文獻

  1. R. Wu, L. Sun, Z. Ma, L. Zhang, “One-Step Effective Diffusion Network for Real-World Image Super-Resolution,” NeurIPS, 2024. source
  2. Z. Yue, J. Wang, C. C. Loy, “ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting,” NeurIPS, 2023. source
  3. Y. Wang, W. Yang, X. Chen, et al., “SinSR: Diffusion-Based Image Super-Resolution in a Single Step,” CVPR, 2024. source · supplementary ablations
  4. A. Zhang, Z. Yue, R. Pei, W. Ren, X. Cao, “Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors,” arXiv preprint, 2024. paper
  5. Y. Tai, R. Xie, C. Zhao, K. Zhang, Z. Zhang, J. Zhou, J. Yang, “AddSR: Accelerating diffusion-based blind super-resolution with adversarial diffusion distillation,” Pattern Recognition, 2026. published version · manuscript supplying the discussed tables
  6. L. Dong, Q. Fan, Y. Guo, Z. Wang, Q. Zhang, J. Chen, Y. Luo, C. Zou, “TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution,” CVPR, 2025. paper
  7. Z. Yue, K. Liao, C. C. Loy, “Arbitrary-steps Image Super-resolution via Diffusion Inversion,” CVPR, 2025. paper
  8. L. Sun, R. Wu, Z. Ma, S. Liu, Q. Yi, L. Zhang, “Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach,” CVPR, 2025. paper
  9. A. Liu, Y. Liu, J. Gu, Y. Qiao, C. Dong, “Blind Image Super-Resolution: A Survey and Beyond,” IEEE TPAMI (early access), 2022. source
  10. B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, “Diffusion Models, Image Super-Resolution, and Everything: A Survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 7, pp. 11793–11813, 2025. doi · arXiv
  11. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” ICLR, 2022. source
  12. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
  13. Z. Wang, C. Lu, Y. Wang, F. Bao, C. Li, H. Su, J. Zhu, “ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation,” NeurIPS, 2023. paper
  14. A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
  15. J. Cai, H. Zeng, H. Yong, Z. Cao, L. Zhang, “Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. source
  16. P. Wei, Z. Xie, H. Lu, Z. Zhan, Q. Ye, W. Zuo, L. Lin, “Component Divide-and-Conquer for Real-World Image Super-Resolution,” ECCV, 2020. source
  17. E. Agustsson and R. Timofte, “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study,” in Proc. IEEE CVPR Workshops, 2017, pp. 1122–1131. doi · CVF
  18. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  19. K. Ding, K. Ma, S. Wang, E. P. Simoncelli, “Image Quality Assessment: Unifying Structure and Texture Similarity,” IEEE TPAMI, 2022. source
  20. A. Mittal, R. Soundararajan and A. C. Bovik, “Making a “Completely Blind” Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. doi
  21. J. Ke, Q. Wang, Y. Wang, P. Milanfar, F. Yang, “MUSIQ: Multi-scale Image Quality Transformer,” ICCV, 2021. source
  22. S. Yang, T. Wu, S. Shi, S. Lao, Y. Gong, M. Cao, J. Wang, Y. Yang, “MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment,” CVPR Workshops, 2022. source
  23. J. Wang, K. C. K. Chan, C. C. Loy, “Exploring CLIP for Assessing the Look and Feel of Images,” AAAI, 2023. source
  24. X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
  25. Z. Wu, Y. Dong, X. Wei, “TUDSR: Twice Upsampling-Diffusion for Higher Super-Resolution,” CVPR, 2026. paper
  26. P. Esser, R. Rombach, B. Ommer, “Taming Transformers for High-Resolution Image Synthesis,” CVPR, 2021. paper
  27. X. Wang, L. Xie, C. Dong, Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution With Pure Synthetic Data,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. source
  28. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
  29. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
  30. R. Wu, T. Yang, L. Sun, Z. Zhang, S. Li, L. Zhang, “SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. source · supplementary experiments
  31. H. Liu, C. Li, Q. Wu, Y. J. Lee, “Visual Instruction Tuning,” NeurIPS, 2023. source
  32. Y. Li, K. Zhang, J. Liang, et al., “LSDIR: A Large Scale Dataset for Image Restoration,” CVPR Workshops, 2023. paper
  33. T. Karras, S. Laine, T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” CVPR, 2019. paper
  34. I. Loshchilov, F. Hutter, “Decoupled Weight Decay Regularization,” ICLR, 2019. paper
  35. P. Esser, S. Kulal, A. Blattmann, et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,” ICML, 2024. paper
  36. J. Ho, T. Salimans, “Classifier-Free Diffusion Guidance,” arXiv preprint, 2022. paper
  37. R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang et al., “NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results,” in Proc. IEEE CVPR Workshops, 2017, pp. 1110–1121. doi · CVF