Full search page →

The Modern view surveys in more detail

Optional · paper guide 1 of 3 for IP 05 · Image Restoration and Reconstruction. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

On this page

Supplementary notes · 7

  1. Optimum notch filtering
  2. Deriving the motion-blur transfer function
  3. Deriving the Wiener filter
  4. Choosing the CLS multiplier
  5. Fan-beam filtered backprojection
  6. Non-local means denoising
  7. Total variation as a regularizer

Paper guides · 3

  1. The Modern view surveys in more detail
  2. Non-Local Means: use similar patches to decide which pixels to average
  3. BM3D: denoise groups of similar patches collaboratively

Image Processing

View the whole course tree →

Choose a reading by topic: Noise, Blur, Prior, Evaluation.

Choose an inverse-problem survey by assumptions
Teaching reading map — the cards compare questions addressed by different papers. They are complementary categories, not a performance ranking or a diagram copied from one source.

Denoising

Tian et al., “Deep learning on image denoising: An overview” (2020)

[10]. Organises CNN denoisers into four groups: additive white Gaussian noise, real (camera) noise, blind denoising (unknown noise), and hybrid cases with blur or low resolution. Takeaway (our reading): the taxonomy itself is organised by noise type, with real camera noise and unknown noise levels as separate categories from synthetic Gaussian noise. The noise model from this chapter, and how realistic it is, stays central in deep denoising.

Elad, Kawar and Vaksman, “Image Denoising: The Deep Learning Revolution and Beyond” (2023)

[11]. Traces the history from classical priors to deep denoisers, then argues that denoisers matter beyond denoising. A good denoiser can serve as the prior inside a solver for other inverse problems, and it is the engine of diffusion-based image generation. Takeaway: the classic “noise only” problem of this chapter has become a building block for nearly everything else.

Deblurring and general inverse problems

Zhang et al., “Deep Image Deblurring: A Survey” (IJCV 2022)

[12]. Covers causes of blur, datasets and metrics, then a taxonomy of CNN-based non-blind and blind deblurring by architecture and loss, plus specialised domains (faces, text, stereo). Takeaway: non-blind deblurring (known hh, as in this chapter) is largely about suppressing the noise amplification and ringing we saw with the inverse filter, while blind deblurring must estimate hh too, and that remains the harder problem.

Ongie et al., “Deep Learning Techniques for Inverse Problems in Imaging” (2020)

[13]. Gives a taxonomy for any g=Hf+η\mathbf g = \mathbf{H}\mathbf f + \boldsymbol\eta problem: is the forward model H\mathbf H known at training time, only at test time, or not at all, and is training supervised or unsupervised? Takeaway (our reading): how much you know about H\mathbf H is the first design decision, and the survey also discusses failure modes of the resulting methods. Knowing H\mathbf H, as in this chapter, opens options that a purely end-to-end network ignores.

Kamilov, Bouman, Buzzard and Wohlberg, “Plug-and-Play Methods for Integrating Physical and Learned Models in Computational Imaging” (2023)

[14]. Reviews plug-and-play (PnP) priors: iterative algorithms that alternate between a step enforcing the physics (g≈Hf\mathbf g \approx \mathbf H \mathbf f) and a step that calls a denoiser, which stands in for the prior. Takeaway: the forward model and the learned prior can be developed separately and combined at run time. DPIR [15] is a widely used example that plugs a deep CNN denoiser into half-quadratic splitting for deblurring, super-resolution and demosaicing.

CT reconstruction

Post-processing:

run FBP, then a CNN removes the artifacts. FBPConvNet [16] does exactly this with a CNN and reports better quality than total-variation reconstruction on sparse-view CT at a fraction of the run time.

Unrolled (learned iterative) reconstruction:

take an iterative algorithm, fix its number of iterations, and replace parts of each iteration with small networks, trained end to end. Learned Primal-Dual [17] unrolls a primal-dual algorithm, keeping the forward projector and backprojector inside each iteration. Monga, Li and Eldar, “Algorithm Unrolling” (2019) [18] surveys this design pattern across signal and image processing. Takeaway: because each layer mirrors an algorithm step, unrolled networks are more interpretable and more efficient than generic networks.

Wang, Ye and De Man, “Deep learning for tomographic image reconstruction” (2020)

[19] reviews the whole landscape for CT and other tomographic modalities: image-domain, sensor-domain and hybrid networks, physics-informed designs, and the open issues of data availability, robustness and clinical validation.

Reading these papers critically

The most useful comparison is between assumptions, not publication dates. A denoising taxonomy may be organized by noise type; a deblurring taxonomy may organize architectures and blur formation; an inverse-problem review may distinguish explicit operators from learned mappings; and a diffusion review must also distinguish conditional training from test-time likelihood guidance. These axes cross: one method can occupy several categories without the surveys contradicting each other.

For each reported result, record what was known at inference and how the degraded/clean pair was constructed. A known-kernel experiment does not establish blind restoration, and synthetic Gaussian noise does not represent every sensor pipeline. Generative samples can express ambiguity but still depend on model and likelihood assumptions. Use a survey to locate competing formulations, then read the original methods and protocols before choosing a baseline. Historical tables provide context; they are not a controlled common-data experiment simply because the scores appear side by side.

References

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 5. publisher page
  2. H. Hwang and R. A. Haddad, “Adaptive median filters: new algorithms and results,” IEEE Transactions on Image Processing, vol. 4, no. 4, pp. 499–502, 1995. DOI
  3. B. R. Hunt, “The application of constrained least squares estimation to image restoration by digital computer,” IEEE Transactions on Computers, vol. C-22, no. 9, pp. 805–812, 1973. DOI
  4. A. C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, IEEE Press, 1988 (SIAM reprint 2001). free online edition
  5. L. A. Shepp and B. F. Logan, “The Fourier reconstruction of a head section,” IEEE Transactions on Nuclear Science, vol. 21, no. 3, pp. 21–43, 1974. DOI
  6. scikit-image developers, “Radon transform,” scikit-image documentation (gallery example). docs
  7. A. Buades, B. Coll and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, vol. 2, pp. 60–65, 2005. DOI
  8. K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007. DOI
  9. K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, “Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017. doi · arXiv
  10. C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo and C.-W. Lin, “Deep learning on image denoising: An overview,” Neural Networks, vol. 131, pp. 251–275, 2020. arXiv
  11. M. Elad, B. Kawar and G. Vaksman, “Image denoising: The deep learning revolution and beyond — A survey paper,” SIAM Journal on Imaging Sciences, vol. 16, no. 3, pp. 1594–1654, 2023. arXiv
  12. K. Zhang, W. Ren, W. Luo, W.-S. Lai, B. Stenger, M.-H. Yang and H. Li, “Deep image deblurring: A survey,” International Journal of Computer Vision, 2022. arXiv
  13. G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis and R. Willett, “Deep learning techniques for inverse problems in imaging,” arXiv:2005.06001, 2020. arXiv
  14. U. S. Kamilov, C. A. Bouman, G. T. Buzzard and B. Wohlberg, “Plug-and-play methods for integrating physical and learned models in computational imaging: Theory, algorithms, and applications,” IEEE Signal Processing Magazine, vol. 40, no. 1, pp. 85–97, 2023. arXiv
  15. K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool and R. Timofte, “Plug-and-play image restoration with deep denoiser prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6360–6376, 2022. arXiv
  16. K. H. Jin, M. T. McCann, E. Froustey and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4509–4522, 2017. arXiv
  17. J. Adler and O. Öktem, “Learned primal-dual reconstruction,” IEEE Transactions on Medical Imaging, 2018. arXiv
  18. V. Monga, Y. Li and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” arXiv:1912.10557, 2019. arXiv
  19. G. Wang, J. C. Ye and B. De Man, “Deep learning for tomographic image reconstruction,” Nature Machine Intelligence, vol. 2, no. 12, pp. 737–748, 2020. DOI
  20. A. Buades, B. Coll and J.-M. Morel, “A review of image denoising algorithms, with a new one,” Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 490–530, 2005. DOI
  21. L. I. Rudin, S. Osher and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60, no. 1–4, pp. 259–268, 1992. DOI
  22. A. Chambolle, “An algorithm for total variation minimization and applications,” Journal of Mathematical Imaging and Vision, vol. 20, no. 1–2, pp. 89–97, 2004. DOI
  23. A. Buades, B. Coll and J.-M. Morel, “Non-Local Means Denoising,” Image Processing On Line, vol. 1, pp. 208–212, 2011. DOI · article and implementation