完整搜尋頁面 →

ResShift:復原條件的短鏈(NeurIPS 2023)

選讀・DLCV 09A.2・單步擴散超解析度 的論文導讀(第 1/8 篇)。引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

本頁內容

補充資料 · 2

  1. 潛在起點不等於加雜訊方程式
  2. 分數正規化與逐圖量測一致性

論文導讀 · 8

  1. ResShift:復原條件的短鏈(NeurIPS 2023)
  2. SinSR:蒸餾確定性的影像條件映射(CVPR 2024)
  3. OSEDiff:LR 潛在起點與變分分數正規化(NeurIPS 2024)
  4. S3Diff:退化引導的低秩調適
  5. TSD-SR:目標感知分數正規化(CVPR 2025)
  6. InvSR:學得的影像條件起點(CVPR 2025)
  7. PiSA-SR:分開像素與語意調適(CVPR 2025)
  8. AddSR:預測細化的對抗復原蒸餾

電腦視覺深度學習

  1. 02 · 深度影像分類
  2. 03 · 物件偵測:從 R-CNN 到 DETR
  3. 09 · 深度影像復原:低階視覺
查看完整課程樹 →

挑戰

LR 已提供大量結構,復原不必從純雜訊重建整個場景。ResShift [2] 的前向程序逐漸把 HR 移向放大 LR,反向從觀測附近開始,使短鏈可行。

貢獻是復原專用隨機程序及排程,不是蒸餾大型文字教師。短鏈、任務專用容量、資料與表示須一起評估,跨大型先驗比較不只改變步數。

方法

前向鏈在加入雜訊時,將乾淨內容移向低解析度觀測條件;反轉此鏈不同於從純雜訊無條件生成。
原論文圖 2 — 前向鏈在加入雜訊時,將乾淨內容移向低解析度觀測條件;反轉此鏈不同於從純雜訊無條件生成。

圖片來源: [2], arXiv:2307.12348v3, PDF 第 3 頁. 原始 PDF.

殘差與前向轉移。 x0x_0 為 HR,yy 為放大 LR,e=y−x0e=y-x_0。遞增位移比例 ηt\eta_t 的端點近零與一,δt=ηt−ηt−1\delta_t=\eta_t-\eta_{t-1}。每步加入 δte\delta_te 與共變異數 κ2δtI\kappa^2\delta_tI 的雜訊,κ\kappa 控制強度。此處用 δ\delta 避免混淆其他論文的訊號係數。

封閉形式邊際。 累積位移與高斯增量得到:

q(xt∣x0,y)=N(x0+ηt(y−x0),κ2ηtI).q(x_t\mid x_0,y)=\mathcal N(x_0+\eta_t(y-x_0),\kappa^2\eta_tI).

狀態與影像/潛在張量同形狀,小 ηt\eta_t 接近 HR,大 ηt\eta_t 的平均接近 LR。訓練可直接抽任意時間狀態,不必模擬完整前向鏈。

反向目標。 相鄰前向狀態的後驗可解析計算,把均值裡未知 HR 換成網路 fθ(xt,y,t)f_\theta(x_t,y,t),可得:

μθ=ηt−1ηtxt+δtηtfθ(xt,y,t),\mu_\theta=\frac{\eta_{t-1}}{\eta_t}x_t+\frac{\delta_t}{\eta_t}f_\theta(x_t,y,t),

共變異數為 κ2ηt−1δtI/ηt\kappa^2\eta_{t-1}\delta_tI/\eta_t,fθf_\theta 從狀態、觀測及時間預測 HR。訓練化為加權重建,原實作經比較後省去推導時間權重;這是實務簡化,不等於精確變分式。

潛在表示。 使用與 VQGAN [26] 相關的自動編碼器,原設定空間壓縮四倍,HR 與放大 LR 編碼至相容網格。其任務訓練先驗與大型文字模型的廣泛預訓練先驗不同。

排程。 ηt\sqrt{\eta_t} 的幾何式排程以 pp 控制形狀,κ\kappa 控制雜訊。預設 15 步、p=0.3p=0.3、κ=2\kappa=2。改參數會改訓練狀態,排程掃描不必然是同一模型免費的推論控制。

資料與推論。 用 Real-ESRGAN 式 [27] 合成退化訓練。推論約從 y+κϵy+\kappa\epsilon 開始,經反向更新再解碼。較小雜訊靠近觀測,仍無法識別缺失細節或修正錯誤退化;起點和反向程序保留隨機性。

結果

表 3 的 ImageNet-Test [28]:PSNR 25.01、SSIM 0.677、LPIPS 0.231、CLIPIQA 0.592、MUSIQ 53.660。原文也比較同為 15 步的一般潛在擴散模型,幫助區分程序設計與單純減步,但成績仍依訓練與測試設定。

真實影像的無參考及視覺比較支持該退化範圍的外觀品質,不證明缺失細節精確。應一起閱讀排程研究中的保真取捨。

消融實驗

表 1:10/15/50 步的 PSNR 為 25.20/25.01/24.22,LPIPS 0.2517/0.2312/0.2212。多步改善感知距離,卻降低 PSNR;「品質更好」須指明指標。

15 步、κ=2\kappa=2 時,pp 從 0.3 改 1,LPIPS 0.2312→0.2613,MUSIQ 53.6596→48.4964。固定 p=0.3p=0.3,κ=0.5\kappa=0.5 的 LPIPS 0.2437,κ=8\kappa=8 為 0.2592,預設 2 為 0.2312。排程敏感,不是單調的寫實度旋鈕。

結論

以 LR 為中心的可解析殘差鏈,是從純雜訊開始的替代方案;排程決定保真與外觀取捨。

限制與問題

仍須多次網路評估,不能識別未量測細節;任務先驗、合成退化涵蓋與自動編碼器皆有限制。SinSR [3] 接著把此條件程序的確定性端點蒸餾成一次學生預測。

參考文獻

  1. R. Wu, L. Sun, Z. Ma, L. Zhang, “One-Step Effective Diffusion Network for Real-World Image Super-Resolution,” NeurIPS, 2024. source
  2. Z. Yue, J. Wang, C. C. Loy, “ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting,” NeurIPS, 2023. source
  3. Y. Wang, W. Yang, X. Chen, et al., “SinSR: Diffusion-Based Image Super-Resolution in a Single Step,” CVPR, 2024. source · supplementary ablations
  4. A. Zhang, Z. Yue, R. Pei, W. Ren, X. Cao, “Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors,” arXiv preprint, 2024. paper
  5. Y. Tai, R. Xie, C. Zhao, K. Zhang, Z. Zhang, J. Zhou, J. Yang, “AddSR: Accelerating diffusion-based blind super-resolution with adversarial diffusion distillation,” Pattern Recognition, 2026. published version · manuscript supplying the discussed tables
  6. L. Dong, Q. Fan, Y. Guo, Z. Wang, Q. Zhang, J. Chen, Y. Luo, C. Zou, “TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution,” CVPR, 2025. paper
  7. Z. Yue, K. Liao, C. C. Loy, “Arbitrary-steps Image Super-resolution via Diffusion Inversion,” CVPR, 2025. paper
  8. L. Sun, R. Wu, Z. Ma, S. Liu, Q. Yi, L. Zhang, “Pixel-level and Semantic-level Adjustable Super-resolution: A Dual-LoRA Approach,” CVPR, 2025. paper
  9. A. Liu, Y. Liu, J. Gu, Y. Qiao, C. Dong, “Blind Image Super-Resolution: A Survey and Beyond,” IEEE TPAMI (early access), 2022. source
  10. B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, “Diffusion Models, Image Super-Resolution, and Everything: A Survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 7, pp. 11793–11813, 2025. doi · arXiv
  11. E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” ICLR, 2022. source
  12. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
  13. Z. Wang, C. Lu, Y. Wang, F. Bao, C. Li, H. Su, J. Zhu, “ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation,” NeurIPS, 2023. paper
  14. A. Sauer, D. Lorenz, A. Blattmann, R. Rombach, “Adversarial Diffusion Distillation,” ECCV, 2024. paper
  15. J. Cai, H. Zeng, H. Yong, Z. Cao, L. Zhang, “Toward Real-World Single Image Super-Resolution: A New Benchmark and a New Model,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. source
  16. P. Wei, Z. Xie, H. Lu, Z. Zhan, Q. Ye, W. Zuo, L. Lin, “Component Divide-and-Conquer for Real-World Image Super-Resolution,” ECCV, 2020. source
  17. E. Agustsson and R. Timofte, “NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study,” in Proc. IEEE CVPR Workshops, 2017, pp. 1122–1131. doi · CVF
  18. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  19. K. Ding, K. Ma, S. Wang, E. P. Simoncelli, “Image Quality Assessment: Unifying Structure and Texture Similarity,” IEEE TPAMI, 2022. source
  20. A. Mittal, R. Soundararajan and A. C. Bovik, “Making a “Completely Blind” Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. doi
  21. J. Ke, Q. Wang, Y. Wang, P. Milanfar, F. Yang, “MUSIQ: Multi-scale Image Quality Transformer,” ICCV, 2021. source
  22. S. Yang, T. Wu, S. Shi, S. Lao, Y. Gong, M. Cao, J. Wang, Y. Yang, “MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment,” CVPR Workshops, 2022. source
  23. J. Wang, K. C. K. Chan, C. C. Loy, “Exploring CLIP for Assessing the Look and Feel of Images,” AAAI, 2023. source
  24. X. Fan, Z. Wu, H. Wu, L. Cao, “A Survey on Pre-Trained Diffusion Model Distillations,” arXiv preprint, version 2, 2025. paper
  25. Z. Wu, Y. Dong, X. Wei, “TUDSR: Twice Upsampling-Diffusion for Higher Super-Resolution,” CVPR, 2026. paper
  26. P. Esser, R. Rombach, B. Ommer, “Taming Transformers for High-Resolution Image Synthesis,” CVPR, 2021. paper
  27. X. Wang, L. Xie, C. Dong, Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution With Pure Synthetic Data,” Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. source
  28. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, Li Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
  29. R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” CVPR, 2022. source
  30. R. Wu, T. Yang, L. Sun, Z. Zhang, S. Li, L. Zhang, “SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. source · supplementary experiments
  31. H. Liu, C. Li, Q. Wu, Y. J. Lee, “Visual Instruction Tuning,” NeurIPS, 2023. source
  32. Y. Li, K. Zhang, J. Liang, et al., “LSDIR: A Large Scale Dataset for Image Restoration,” CVPR Workshops, 2023. paper
  33. T. Karras, S. Laine, T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” CVPR, 2019. paper
  34. I. Loshchilov, F. Hutter, “Decoupled Weight Decay Regularization,” ICLR, 2019. paper
  35. P. Esser, S. Kulal, A. Blattmann, et al., “Scaling Rectified Flow Transformers for High-Resolution Image Synthesis,” ICML, 2024. paper
  36. J. Ho, T. Salimans, “Classifier-Free Diffusion Guidance,” arXiv preprint, 2022. paper
  37. R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang et al., “NTIRE 2017 Challenge on Single Image Super-Resolution: Methods and Results,” in Proc. IEEE CVPR Workshops, 2017, pp. 1110–1121. doi · CVF