Full search page →

BlindDPS: infer an image and its degradation together

Optional · paper guide 4 of 5 for DLCV 09B · Generative Image Restoration: Sampling a Plausible Answer. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

On this page

Supplementary notes · 4

  1. Why posterior sampling exactly doubles the MSE
  2. GAN-prior restorers in more detail
  3. How the zero-shot diffusion samplers differ
  4. Latent and one-step restorers, method by method

Paper guides · 5

  1. Blau and Michaeli, “The Perception-Distortion Tradeoff” (CVPR 2018)
  2. Chung et al., “Diffusion Posterior Sampling for General Noisy Inverse Problems” (ICLR 2023)
  3. DDNM: preserve measured components and generate the null space
  4. BlindDPS: infer an image and its degradation together
  5. Li et al., “Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey” (IJCV 2025)

Deep Learning for Computer Vision

  1. 02 · Deep Image Classification
View the whole course tree →

Challenge

Non-blind DPS assumes that the forward operator is known. BlindDPS—“Parallel Diffusion Models of Operator and Image for Blind Inverse Problems,” CVPR 2023 [49]—also estimates operator parameters. For motion deblurring, both the sharp image and blur kernel are unknown; reducing their shared residual is not enough to identify them without suitable priors.

Method

Two denoisers estimate an image and an operator; a common residual couples their updates rather than supplying a known blur kernel.
Original paper figure 2 — Two denoisers estimate an image and an operator; a common residual couples their updates rather than supplying a known blur kernel.

Figure source: [49], PDF page 3. Original PDF.

Define the unknowns explicitly. For image xx, kernel kk, convolution ∗* and measurement noise nn, the basic forward model is

y=k∗x+n,p(x,k∣y)∝p(y∣x,k) p(x) p(k).y=k*x+n,\qquad p(x,k\mid y)\propto p(y\mid x,k)\,p(x)\,p(k).

yy is the observed blurry image; p(x)p(x) and p(k)p(k) are separately trained image and kernel priors. Their independence is a modeling assumption before observing yy. The likelihood couples them in the posterior. The paper also models turbulence as a tilt/warp followed by blur, with an additional prior for the operator field.

Run parallel prior processes with coupled corrections. At time tt, image and kernel score networks produce clean estimates x^0(xt)\widehat x_0(x_t) and k^0(kt)\widehat k_0(k_t). A common data term uses

rt=y−k^0∗x^0.r_t=y-\widehat k_0*\widehat x_0.

xt,ktx_t,k_t are their noisy diffusion states and rtr_t is the measurement residual. Image guidance differentiates through the image estimate while treating the kernel state as the other variable; kernel guidance differentiates through the kernel estimate. Both gradients depend on the current estimates of both unknowns.

The implementation makes separate reverse-diffusion proposals and applies likelihood corrections. The paper uses the residual norm with a static coefficient in its practical algorithm, rather than requiring the exact Gaussian squared-residual coefficient at every step. This numerical choice is distinct from the probabilistic motivation.

Constrain the kernel and add useful regularity. Clean kernel estimates are projected to nonnegative, unit-sum kernels. This fixes a common scale ambiguity and excludes negative point-spread functions, but does not prove a unique image/kernel solution. A sparsity term further favors suitable motion kernels. Its weight is an extra assumption, not a property shared by every real optical blur.

Training operator priors is part of the method. BlindDPS is not an unchanged unconditional image model that somehow learns any unknown physical degradation during one inference run.

Read the figure and choose a baseline. Follow both clean estimates into the same predicted measurement. Updating the image while ignoring the current kernel, or updating the kernel as if the image were known, would remove the coupling that defines the method. A small residual alone cannot identify the correct pair. Inspect the estimated kernel as well as the restored image, and distinguish training an operator prior from fitting an operator at test time. Compared with known-operator restoration, this method adds uncertainty and computation; the extra prior is useful only insofar as it represents the degradation actually encountered.

Results

Original Table 1 reports FFHQ 256×256256\times256 blind motion deblurring at FID 29.49 / LPIPS 0.281 / PSNR 22.24 dB. Table 2 also evaluates estimated kernels, rather than judging images alone; its FFHQ motion-kernel normalized-convolution score is 0.955.

Table 3 demonstrates a perceptual/distortion trade-off for turbulence: on FFHQ, BlindDPS has FID 27.35 / PSNR 24.49 dB, while the supervised TSR-WGAN comparison has FID 58.30 / PSNR 26.29 dB. Improved perceptual distribution scores do not mean a higher PSNR or guaranteed recovery of the true scene.

Ablations

Figure 7 replaces the diffusion kernel prior with a uniform prior, leaving kernel estimation to likelihood guidance. Its degraded kernel/image results support learning an operator prior; they do not isolate every possible alternative regularizer.

Section 4.3 and the supplementary sparsity study vary kernel regularization. Omitting the term is worse, particularly for motion blur, while a range of nonzero strengths works in the tested setting. The findings justify the paper’s combination of learned and hand-designed priors, not a universal sparsity weight for unknown cameras.

Conclusion

BlindDPS extends likelihood-guided diffusion to several unknown variables, each with its own prior, and couples their sampling through the measurement model. Jointly inspecting reconstruction and operator estimates is essential.

Problems

Different image/kernel pairs can explain the same observation. A kernel or image outside the trained prior distribution can be forced toward a plausible but wrong solution. Spatially varying blur, saturation and unmodeled sensor effects may violate the assumed forward model. Each additional variable requires modeling and computation, so the ability to add another diffusion process does not make its cost or identifiability free.

References

  1. Y. Blau and T. Michaeli, “The Perception-Distortion Tradeoff,” in Proc. IEEE/CVF CVPR, 2018, pp. 6228–6237 (extended version on arXiv). doi · arXiv
  2. C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang and W. Shi, “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network,” in Proc. IEEE CVPR, 2017, pp. 105–114. doi · arXiv
  3. J. Johnson, A. Alahi and L. Fei-Fei, “Perceptual Losses for Real-Time Style Transfer and Super-Resolution,” in Proc. ECCV, 2016, pp. 694–711. doi · arXiv
  4. X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao and C. C. Loy, “ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks,” in Proc. ECCV Workshops, 2018 (LNCS, 2019), pp. 63–79. doi · arXiv
  5. X. Wang, L. Xie, C. Dong and Y. Shan, “Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data,” in Proc. IEEE/CVF ICCV Workshops, 2021, pp. 1905–1914. doi · arXiv
  6. T. Karras, S. Laine and T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” in Proc. IEEE/CVF CVPR, 2019, pp. 4396–4405. doi · arXiv
  7. S. Menon, A. Damian, S. Hu, N. Ravi and C. Rudin, “PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models,” in Proc. IEEE/CVF CVPR, 2020, pp. 2434–2442. doi · arXiv
  8. K. C. K. Chan, X. Wang, X. Xu, J. Gu and C. C. Loy, “GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution,” in Proc. IEEE/CVF CVPR, 2021, pp. 14240–14249. doi · arXiv
  9. X. Wang, Y. Li, H. Zhang and Y. Shan, “Towards Real-World Blind Face Restoration with Generative Facial Prior,” in Proc. IEEE/CVF CVPR, 2021, pp. 9164–9174. doi · arXiv
  10. S. Zhou, K. C. K. Chan, C. Li and C. C. Loy, “Towards Robust Blind Face Restoration with Codebook Lookup Transformer,” in Proc. NeurIPS, vol. 35, 2022, pp. 30599–30611. doi · arXiv
  11. J. Ho, A. Jain and P. Abbeel, “Denoising Diffusion Probabilistic Models,” in Proc. NeurIPS, 2020. proceedings · arXiv
  12. Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon and B. Poole, “Score-Based Generative Modeling through Stochastic Differential Equations,” in Proc. ICLR, 2021. arXiv
  13. B. Efron, “Tweedie’s Formula and Selection Bias,” Journal of the American Statistical Association, vol. 106, no. 496, pp. 1602–1614, 2011. doi
  14. C. Saharia, J. Ho, W. Chan, T. Salimans, D. J. Fleet and M. Norouzi, “Image Super-Resolution via Iterative Refinement,” IEEE TPAMI, 2022 (early access). doi · arXiv
  15. C. Saharia, W. Chan, H. Chang, C. Lee, J. Ho, T. Salimans, D. Fleet and M. Norouzi, “Palette: Image-to-Image Diffusion Models,” in Proc. ACM SIGGRAPH Conference, 2022, pp. 1–10. doi · arXiv
  16. Z. Yue, J. Wang and C. C. Loy, “ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting,” in Proc. NeurIPS, vol. 36, 2023, pp. 13294–13307. doi · arXiv
  17. H. Chung, J. Kim, M. T. McCann, M. L. Klasky and J. C. Ye, “Diffusion Posterior Sampling for General Noisy Inverse Problems,” in Proc. ICLR, 2023. arXiv
  18. B. Kawar, M. Elad, S. Ermon and J. Song, “Denoising Diffusion Restoration Models,” in Proc. NeurIPS, vol. 35, 2022, pp. 23593–23606. doi · arXiv
  19. Y. Wang, J. Yu and J. Zhang, “Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model,” in Proc. ICLR, 2023. arXiv
  20. J. Song, A. Vahdat, M. Mardani and J. Kautz, “Pseudoinverse-Guided Diffusion Models for Inverse Problems,” in Proc. ICLR, 2023. ICLR page
  21. Y. Zhu, K. Zhang, J. Liang, J. Cao, B. Wen, R. Timofte and L. Van Gool, “Denoising Diffusion Models for Plug-and-Play Image Restoration,” in Proc. IEEE/CVF CVPR Workshops, 2023, pp. 1219–1229. doi · arXiv
  22. R. Rombach, A. Blattmann, D. Lorenz, P. Esser and B. Ommer, “High-Resolution Image Synthesis with Latent Diffusion Models,” in Proc. IEEE/CVF CVPR, 2022, pp. 10674–10685. doi · arXiv
  23. L. Zhang, A. Rao and M. Agrawala, “Adding Conditional Control to Text-to-Image Diffusion Models,” in Proc. IEEE/CVF ICCV, 2023, pp. 3813–3824. doi · arXiv
  24. J. Wang, Z. Yue, S. Zhou, K. C. K. Chan and C. C. Loy, “Exploiting Diffusion Prior for Real-World Image Super-Resolution,” International Journal of Computer Vision, vol. 132, no. 12, pp. 5929–5949, 2024. doi · arXiv
  25. X. Lin, J. He, Z. Chen, Z. Lyu, B. Dai, F. Yu, Y. Qiao, W. Ouyang and C. Dong, “DiffBIR: Toward Blind Image Restoration with Generative Diffusion Prior,” in Proc. ECCV, 2024 (LNCS), pp. 430–448. doi · arXiv
  26. F. Yu, J. Gu, Z. Li, J. Hu, X. Kong, X. Wang, J. He, Y. Qiao and C. Dong, “Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild,” in Proc. IEEE/CVF CVPR, 2024, pp. 25669–25680. doi · arXiv
  27. Y. Wang, W. Yang, X. Chen, Y. Wang, L. Guo, L.-P. Chau, Z. Liu, Y. Qiao, A. C. Kot and B. Wen, “SinSR: Diffusion-Based Image Super-Resolution in a Single Step,” in Proc. IEEE/CVF CVPR, 2024, pp. 25796–25805. doi · arXiv
  28. R. Wu, L. Sun, Z. Ma and L. Zhang, “One-Step Effective Diffusion Network for Real-World Image Super-Resolution,” in Proc. NeurIPS, vol. 37, 2024, pp. 92529–92553. doi · arXiv
  29. Z. Yue, K. Liao and C. C. Loy, “Arbitrary-steps Image Super-resolution via Diffusion Inversion,” in Proc. IEEE/CVF CVPR, 2025, pp. 23153–23163. doi · arXiv
  30. L. Dong, Q. Fan, Y. Guo, Z. Wang, Q. Zhang, J. Chen, Y. Luo and C. Zou, “TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution,” in Proc. IEEE/CVF CVPR, 2025, pp. 23174–23184. doi · arXiv
  31. Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel and M. Le, “Flow Matching for Generative Modeling,” in Proc. ICLR, 2023. arXiv
  32. X. Liu, C. Gong and Q. Liu, “Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow,” in Proc. ICLR, 2023. arXiv
  33. G. Ohayon, T. Michaeli and M. Elad, “Posterior-Mean Rectified Flow: Towards Minimum MSE Photo-Realistic Image Restoration,” in Proc. ICLR, 2025. arXiv
  34. S. Martin, A. Gagneux, P. Hagemann and G. Steidl, “PnP-Flow: Plug-and-Play Image Restoration with Flow Matching,” in Proc. ICLR, 2025. arXiv
  35. S. Bhadra, V. A. Kelkar, F. J. Brooks and M. A. Anastasio, “On Hallucinations in Tomographic Image Reconstruction,” IEEE Transactions on Medical Imaging, vol. 40, no. 11, pp. 3249–3260, 2021. doi · arXiv
  36. W. Ren, R. Goyal, Z. Hu, T. T. Aumentado-Armstrong, I. Mohomed and A. Levinshtein, “Hallucination Score: Towards Mitigating Hallucinations in Generative Image Super-Resolution,” arXiv:2507.14367, 2025. arXiv
  37. A. N. Angelopoulos, A. P. Kohli, S. Bates, M. I. Jordan, J. Malik, T. Alshaabi, S. Upadhyayula and Y. Romano, “Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging,” in Proc. ICML, PMLR 162, 2022. proceedings · arXiv
  38. R. Zhang, P. Isola, A. A. Efros, E. Shechtman and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” in Proc. IEEE/CVF CVPR, 2018, pp. 586–595. doi · arXiv
  39. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  40. M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler and S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” in Proc. NeurIPS (NIPS), 2017. arXiv
  41. A. Mittal, R. Soundararajan and A. C. Bovik, “Making a ‘Completely Blind’ Image Quality Analyzer,” IEEE Signal Processing Letters, vol. 20, no. 3, pp. 209–212, 2013. doi
  42. J. Ke, Q. Wang, Y. Wang, P. Milanfar and F. Yang, “MUSIQ: Multi-scale Image Quality Transformer,” in Proc. IEEE/CVF ICCV, 2021, pp. 5128–5137. doi · arXiv
  43. J. Wang, K. C. K. Chan and C. C. Loy, “Exploring CLIP for Assessing the Look and Feel of Images,” in Proc. AAAI, vol. 37, no. 2, 2023, pp. 2555–2563. doi · arXiv
  44. X. Li, Y. Ren, X. Jin, C. Lan, X. Wang, W. Zeng, X. Wang and Z. Chen, “Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey,” International Journal of Computer Vision, vol. 133, no. 11, pp. 8078–8108, 2025. doi · arXiv
  45. G. Daras, H. Chung, C.-H. Lai, Y. Mitsufuji, J. C. Ye, P. Milanfar, A. G. Dimakis and M. Delbracio, “A Survey on Diffusion Models for Inverse Problems,” arXiv:2410.00083, 2024. arXiv
  46. B. B. Moser, A. S. Shanbhag, F. Raue, S. Frolov, S. Palacio and A. Dengel, “Diffusion Models, Image Super-Resolution, and Everything: A Survey,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 7, pp. 11793–11813, 2025. doi · arXiv
  47. W. Xia, Y. Zhang, Y. Yang, J.-H. Xue, B. Zhou and M.-H. Yang, “GAN Inversion: A Survey,” IEEE TPAMI, vol. 45, no. 3, pp. 3121–3138, 2023. doi · arXiv
  48. Y. Song and S. Ermon, “Generative Modeling by Estimating Gradients of the Data Distribution,” in Proc. NeurIPS, 2019. arXiv
  49. H. Chung, J. Kim, S. Kim and J. C. Ye, “Parallel Diffusion Models of Operator and Image for Blind Inverse Problems,” in Proc. IEEE/CVF CVPR, 2023, pp. 6059–6069. paper