BlindDPS:共同推論影像與退化
選讀・DLCV 09B・生成式影像復原:取樣出一個合理的答案 的論文導讀(第 4/5 篇)。引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
挑戰
非盲 DPS 假設已知前向算子。BlindDPS,即 CVPR 2023 的「Parallel Diffusion Models of Operator and Image for Blind Inverse Problems」[49],還要估計算子參數。運動去模糊中的清晰影像、模糊核都未知;沒有適當先驗,只降低共同殘差不足以識別兩者。
方法

圖片來源: [49], PDF 第 3 頁. 原始 PDF.
明確列出未知量。 對影像 、模糊核 、卷積 、量測雜訊 ,基本模型為
是模糊觀測,、 是分別訓練的影像與核先驗。兩者獨立是觀察 之前的建模假設;後驗中由概似把它們耦合。原文也把亂流建模為先傾斜/扭曲、再模糊,並為額外算子場建立先驗。
並行跑先驗,再共同修正。 時間 的影像、核分數網路產生乾淨估計 、,共同資料項使用
是各自帶雜訊的擴散狀態, 為量測殘差。影像引導對影像估計反傳,把核狀態視為另一變數;核引導則對核估計反傳。兩組梯度都依賴當前兩個未知量的估計。
實作分別提出反向擴散狀態,再做概似修正。原文的實務演算法使用殘差範數與固定係數,而非要求每一步都套用精確的高斯平方殘差係數。數值選擇與機率動機應分開理解。
限制模糊核並加入有用正規性。 乾淨核估計投影到非負、總和一的集合,修正常見尺度歧義並排除負點擴散函數,但不能證明影像/核解唯一。稀疏項進一步偏好合適運動核;其權重是額外假設,並非所有真實光學模糊都共享的性質。
算子先驗的訓練是方法的一部分。BlindDPS 不是完全不改變的無條件影像模型,在單次推論中就能自動學會任何未知物理退化。
讀圖與選用判斷。 追蹤影像與模糊核兩個乾淨估計如何共同形成預測量測。若更新影像時忽略目前的核,或更新核時假設影像已知,就會失去這個方法的耦合關係。殘差小並不足以辨識正確組合。應同時檢查估計核與修復影像,並區分「訓練算子先驗」和「測試時估計算子」。相較已知算子的修復,這個方法增加不確定性與運算;額外先驗是否有用,取決於它是否能代表實際遇到的退化。
結果
原表 1 的 FFHQ 盲運動去模糊為 FID 29.49/LPIPS 0.281/PSNR 22.24 dB。表 2 另外評估核估計,而非只看影像;FFHQ 運動核的正規化卷積分數為 0.955。
表 3 呈現亂流的感知/失真取捨:FFHQ 的 BlindDPS 為 FID 27.35/PSNR 24.49 dB,監督式 TSR-WGAN 比較列則為 FID 58.30/PSNR 26.29 dB。感知分布分數改善,不代表 PSNR 較高或保證找回真實場景。
消融實驗
圖 7 用均勻先驗替換擴散核先驗,讓核估計只依概似引導。較差的核與影像結果支持學習算子先驗,但未分離所有可能替代正規化方法。
第 4.3 節與補充資料的稀疏研究改變核正規化。省略此項較差,尤其對運動模糊;一段非零強度範圍在測試設定有效。結果支持結合學習與手設先驗,並未建立所有未知相機皆適用的稀疏權重。
結論
BlindDPS 把概似引導擴散擴展到多個未知變數,各有先驗,再以量測模型耦合其取樣。因此必須同時查看重建與算子估計。
限制與問題
不同影像/核配對可能解釋同一觀測。核或影像若超出先驗訓練分布,可能被推向合理卻錯誤的解。空間變動模糊、飽和與未建模感測效應,可能違反前向假設。每增加未知量,都需要建模與計算;可以再加入一個擴散過程,不代表成本或可識別性免費獲得。
參考文獻
- Y. Blau and T. Michaeli, “The Perception-Distortion Tradeoff,” in Proc. IEEE/CVF CVPR, 2018, pp. 6228–6237 (extended version on arXiv). doi · arXiv
- C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang and W. Shi, “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” in Proc. IEEE CVPR, 2017, pp. 105–114. doi · arXiv
- J. Johnson, A. Alahi and L. Fei-Fei, “Perceptual Losses for Real-Time Style Transfer and Super-Resolution,” in Proc. ECCV, 2016, pp. 694–711. doi · arXiv
- X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao and C. C. Loy, “ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks,” in Proc. ECCV Workshops, 2018 (LNCS, 2019), pp. 63–79. doi · arXiv
- X. Wang, L. Xie, C. Dong and Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data,” in Proc. IEEE/CVF ICCV Workshops, 2021, pp. 1905–1914. doi · arXiv
- T. Karras, S. Laine and T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” in Proc. IEEE/CVF CVPR, 2019, pp. 4396–4405. doi · arXiv
- S. Menon, A. Damian, S. Hu, N. Ravi and C. Rudin, “PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models,” in Proc. IEEE/CVF CVPR, 2020, pp. 2434–2442. doi · arXiv
- K. C. K. Chan, X. Wang, X. Xu, J. Gu and C. C. Loy, “GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution,” in Proc. IEEE/CVF CVPR, 2021, pp. 14240–14249. doi · arXiv
- X. Wang, Y. Li, H. Zhang and Y. Shan, “Towards Real-World Blind Face Restoration with Generative Facial Prior,” in Proc. IEEE/CVF CVPR, 2021, pp. 9164–9174. doi · arXiv
- S. Zhou, K. C. K. Chan, C. Li and C. C. Loy, “Towards Robust Blind Face Restoration with Codebook Lookup Transformer,” in Proc. NeurIPS, vol. 35, 2022, pp. 30599–30611. doi · arXiv
- J. Ho, A. Jain and P. Abbeel, “Denoising Diffusion Probabilistic Models,” in Proc. NeurIPS, 2020. proceedings · arXiv
- Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon and B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” in Proc. ICLR, 2021. arXiv
- B. Efron, “Tweedie’s Formula and Selection Bias,” Journal of the American Statistical Association, vol. 106, no. 496, pp. 1602–1614, 2011. doi
- C. Saharia, J. Ho, W. Chan, T. Salimans, D. J. Fleet and M. Norouzi, “Image Super-Resolution via Iterative Refinement,” IEEE TPAMI, 2022 (early access). doi · arXiv
- C. Saharia, W. Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet and M. Norouzi, “Palette: Image-to-Image Diffusion Models,” in Proc. ACM SIGGRAPH Conference, 2022, pp. 1–10. doi · arXiv
- Z. Yue, J. Wang and C. C. Loy, “ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting,” in Proc. NeurIPS, vol. 36, 2023, pp. 13294–13307. doi · arXiv
- H. Chung, J. Kim, M. T. McCann, M. L. Klasky and J. C. Ye, “Diffusion Posterior Sampling for General Noisy Inverse Problems,” in Proc. ICLR, 2023. arXiv
- B. Kawar, M. Elad, S. Ermon and J. Song, “Denoising Diffusion Restoration Models,” in Proc. NeurIPS, vol. 35, 2022, pp. 23593–23606. doi · arXiv
- Y. Wang, J. Yu and J. Zhang, “Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model,” in Proc. ICLR, 2023. arXiv
- J. Song, A. Vahdat, M. Mardani and J. Kautz, “Pseudoinverse-Guided Diffusion Models for Inverse Problems,” in Proc. ICLR, 2023. ICLR page
- Y. Zhu, K. Zhang, J. Liang, J. Cao, B. Wen, R. Timofte and L. Van Gool, “Denoising Diffusion Models for Plug-and-Play Image Restoration,” in Proc. IEEE/CVF CVPR Workshops, 2023, pp. 1219–1229. doi · arXiv
- R. Rombach, A. Blattmann, D. Lorenz, P. Esser and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” in Proc. IEEE/CVF CVPR, 2022, pp. 10674–10685. doi · arXiv
- L. Zhang, A. Rao and M. Agrawala, “Adding Conditional Control to Text-to-Image Diffusion Models,” in Proc. IEEE/CVF ICCV, 2023, pp. 3813–3824. doi · arXiv
- J. Wang, Z. Yue, S. Zhou, K. C. K. Chan and C. C. Loy, “Exploiting Diffusion Prior for Real-World Image Super-Resolution,” International Journal of Computer Vision, vol. 132, no. 12, pp. 5929–5949, 2024. doi · arXiv
- X. Lin, J. He, Z. Chen, Z. Lyu, B. Dai, F. Yu, Y. Qiao, W. Ouyang and C. Dong, “DiffBIR: Toward Blind Image Restoration with Generative Diffusion Prior,” in Proc. ECCV, 2024 (LNCS), pp. 430–448. doi · arXiv
- F. Yu, J. Gu, Z. Li, J. Hu, X. Kong, X. Wang, J. He, Y. Qiao and C. Dong, “Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild,” in Proc. IEEE/CVF CVPR, 2024, pp. 25669–25680. doi · arXiv
- Y. Wang, W. Yang, X. Chen, Y. Wang, L. Guo, L.-P. Chau, Z. Liu, Y. Qiao, A. C. Kot and B. Wen, “SinSR: Diffusion-Based Image Super-Resolution in a Single Step,” in Proc. IEEE/CVF CVPR, 2024, pp. 25796–25805. doi · arXiv
- R. Wu, L. Sun, Z. Ma and L. Zhang, “One-Step Effective Diffusion Network for Real-World Image Super-Resolution,” in Proc. NeurIPS, vol. 37, 2024, pp. 92529–92553. doi · arXiv
- Z. Yue, K. Liao and C. C. Loy, “Arbitrary-steps Image Super-resolution via Diffusion Inversion,” in Proc. IEEE/CVF CVPR, 2025, pp. 23153–23163. doi · arXiv
- L. Dong, Q. Fan, Y. Guo, Z. Wang, Q. Zhang, J. Chen, Y. Luo and C. Zou, “TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution,” in Proc. IEEE/CVF CVPR, 2025, pp. 23174–23184. doi · arXiv
- Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel and M. Le, “Flow Matching for Generative Modeling,” in Proc. ICLR, 2023. arXiv
- X. Liu, C. Gong and Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” in Proc. ICLR, 2023. arXiv
- G. Ohayon, T. Michaeli and M. Elad, “Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration,” in Proc. ICLR, 2025. arXiv
- S. Martin, A. Gagneux, P. Hagemann and G. Steidl, “PnP-Flow: Plug-and-Play Image Restoration with Flow Matching,” in Proc. ICLR, 2025. arXiv
- S. Bhadra, V. A. Kelkar, F. J. Brooks and M. A. Anastasio, “On Hallucinations in Tomographic Image Reconstruction,” IEEE Transactions on Medical Imaging, vol. 40, no. 11, pp. 3249–3260, 2021. doi · arXiv
- W. Ren, R. Goyal, Z. Hu, T. T. Aumentado-Armstrong, I. Mohomed and A. Levinshtein, “Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution,” arXiv:2507.14367, 2025. arXiv
- A. N. Angelopoulos, A. P. Kohli, S. Bates, M. I. Jordan, J. Malik, T. Alshaabi, S. Upadhyayula and Y. Romano, “Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging,” in Proc. ICML, PMLR 162, 2022. proceedings · arXiv
- R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
- Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
- M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler and S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” in Proc. NeurIPS (NIPS), 2017. arXiv
- A. Mittal, R. Soundararajan and A. C. Bovik, “Making a ‘Completely Blind’ Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. doi
- J. Ke, Q. Wang, Y. Wang, P. Milanfar and F. Yang, “MUSIQ: Multi-scale Image Quality Transformer,” in Proc. IEEE/CVF ICCV, 2021, pp. 5128–5137. doi · arXiv
- J. Wang, K. C. K. Chan and C. C. Loy, “Exploring CLIP for Assessing the Look and Feel of Images,” in Proc. AAAI, vol. 37, no. 2, 2023, pp. 2555–2563. doi · arXiv
- X. Li, Y. Ren, X. Jin, C. Lan, X. Wang, W. Zeng, X. Wang and Z. Chen, “Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey,” International Journal of Computer Vision, vol. 133, no. 11, pp. 8078–8108, 2025. doi · arXiv
- G. Daras, H. Chung, C.-H. Lai, Y. Mitsufuji, J. C. Ye, P. Milanfar, A. G. Dimakis and M. Delbracio, “A Survey on Diffusion Models for Inverse Problems,” arXiv:2410.00083, 2024. arXiv
- B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, “Diffusion Models, Image Super-Resolution, and Everything: A Survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 7, pp. 11793–11813, 2025. doi · arXiv
- W. Xia, Y. Zhang, Y. Yang, J.-H. Xue, B. Zhou and M.-H. Yang, “GAN Inversion: A Survey,” IEEE TPAMI, vol. 45, no. 3, pp. 3121–3138, 2023. doi · arXiv
- Y. Song and S. Ermon, “Generative Modeling by Estimating Gradients of the Data Distribution,” in Proc. NeurIPS, 2019. arXiv
- H. Chung, J. Kim, S. Kim and J. C. Ye, “Parallel Diffusion Models of Operator and Image for Blind Inverse Problems,” in Proc. IEEE/CVF CVPR, 2023, pp. 6059–6069. paper