Full search page →

BM3D: denoise groups of similar patches collaboratively

Optional · paper guide 3 of 3 for IP 05 · Image Restoration and Reconstruction. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

Challenge

A small patch can be difficult to denoise alone, yet several similar patches may share a compact representation. Dabov et al.’s TIP 2007 paper [8] makes that observation operational: group matching blocks, transform them together, suppress noise in the transform domain, and aggregate their overlapping estimates. The “3D” is a stack of two-dimensional image blocks, not a requirement for a volumetric image. Read NLM first to understand why distant patches can supply useful evidence.

Method

BM3D's first stage produces a basic estimate; the second uses it for grouping and Wiener gains while filtering blocks from the original noisy image.
Original figure 3 — Follow both inputs to step 2. The basic estimate guides matching and Wiener weights; the original noisy image still supplies the coefficients being filtered. Each stage returns overlapping block estimates to their image locations.

Source: Dabov et al. [8], TIP 2007 accepted author manuscript, PDF page 5, figure 3. Original PDF.

First pass: build a usable estimate. For each reference block, find similar blocks in a search region and stack them into a group YY. Apply a separable transform TT across each block and across the group dimension, threshold the coefficients, then invert:

Y^=T−1 ⁣(Hλσ(TY)).\widehat Y=T^{-1}\!\left(\mathcal H_{\lambda\sigma}(TY)\right).

YY is a group of noisy blocks, Y^\widehat Y its filtered estimate, TT the three-dimensional transform, σ\sigma the assumed Gaussian noise standard deviation, and λ\lambda a threshold multiplier. Hτ\mathcal H_\tau sets coefficients with magnitude below τ\tau to zero. Similar blocks concentrate their common signal into relatively few coefficients; the remaining coefficients are more likely to be noise. Return the filtered blocks to their original positions and combine overlapping estimates with reliability weights to form a basic image estimate.

Second pass: use the basic estimate without treating it as truth. Match blocks in the basic estimate. At those same locations, form groups from both the basic and original noisy images. In an orthonormal-transform idealization, a coefficient gain is

gj=∣bj∣2∣bj∣2+σ2,c^j=gjcj.g_j=\frac{|b_j|^2}{|b_j|^2+\sigma^2},\qquad \widehat c_j=g_j c_j.

bjb_j is coefficient jj from the basic group, cjc_j the corresponding original noisy coefficient, gjg_j the Wiener gain, and c^j\widehat c_j the filtered coefficient. The implementation must account for the selected transform’s noise variance/normalization; the expression explains the shrinkage principle. Invert the transform and aggregate again. Filtering only the basic image would incorrectly discard the original measurement path shown in the figure.

Results

Table III compares grayscale denoising PSNR across test images and Gaussian noise levels; figures 6–8 show visual comparisons. Table IV evaluates the color extension separately. The evidence is for the stated additive-noise models and image sets, not arbitrary raw-camera noise, blur or compression. The paper also reports computation on contemporary hardware; those times should not be transferred to a modern CPU or library implementation without measurement.

Ablations and parameter evidence

Table II changes the transforms while retaining the Normal Profile settings. Its “DC-only” group-axis transform performs poorly relative to richer group transforms: matching blocks are similar, not identical, so retaining only their average loses useful differences. The study also finds that the precise within-block transform matters less than one might expect. Figure 9 compares parameter profiles, including settings for stronger noise. These are controlled design comparisons; a faster profile changes the quality–cost trade-off rather than proving the full method unnecessary.

Conclusion

BM3D combines self-similarity with sparse transform-domain estimation. NLM mainly averages compatible observations; BM3D can retain variation within a group through transform coefficients. It is a useful training-free Gaussian-denoising baseline and a way to understand why grouping, shrinkage and aggregation are separate design decisions.

Problems

An incorrect noise level changes both matching and shrinkage. Severe noise can make unrelated blocks appear similar, while unique structures may have too few matches. Aggregated groups overlap and share observations, so their errors are not independent. Color handling and signal-dependent noise require explicit treatment; the grayscale formula is not automatically a complete camera-noise pipeline. The second pass improves an estimate but cannot guarantee recovery of details unsupported by the observation.

References

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 5. publisher page
  2. H. Hwang and R. A. Haddad, “Adaptive median filters: new algorithms and results,” IEEE Transactions on Image Processing, vol. 4, no. 4, pp. 499–502, 1995. DOI
  3. B. R. Hunt, “The application of constrained least squares estimation to image restoration by digital computer,” IEEE Transactions on Computers, vol. C-22, no. 9, pp. 805–812, 1973. DOI
  4. A. C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, IEEE Press, 1988 (SIAM reprint 2001). free online edition
  5. L. A. Shepp and B. F. Logan, “The Fourier reconstruction of a head section,” IEEE Transactions on Nuclear Science, vol. 21, no. 3, pp. 21–43, 1974. DOI
  6. scikit-image developers, “Radon transform,” scikit-image documentation (gallery example). docs
  7. A. Buades, B. Coll and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, vol. 2, pp. 60–65, 2005. DOI
  8. K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007. DOI
  9. K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, “Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017. doi · arXiv
  10. C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo and C.-W. Lin, “Deep learning on image denoising: An overview,” Neural Networks, vol. 131, pp. 251–275, 2020. arXiv
  11. M. Elad, B. Kawar and G. Vaksman, “Image denoising: The deep learning revolution and beyond — A survey paper,” SIAM Journal on Imaging Sciences, vol. 16, no. 3, pp. 1594–1654, 2023. arXiv
  12. K. Zhang, W. Ren, W. Luo, W.-S. Lai, B. Stenger, M.-H. Yang and H. Li, “Deep image deblurring: A survey,” International Journal of Computer Vision, 2022. arXiv
  13. G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis and R. Willett, “Deep learning techniques for inverse problems in imaging,” arXiv:2005.06001, 2020. arXiv
  14. U. S. Kamilov, C. A. Bouman, G. T. Buzzard and B. Wohlberg, “Plug-and-play methods for integrating physical and learned models in computational imaging: Theory, algorithms, and applications,” IEEE Signal Processing Magazine, vol. 40, no. 1, pp. 85–97, 2023. arXiv
  15. K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool and R. Timofte, “Plug-and-play image restoration with deep denoiser prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6360–6376, 2022. arXiv
  16. K. H. Jin, M. T. McCann, E. Froustey and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4509–4522, 2017. arXiv
  17. J. Adler and O. Öktem, “Learned primal-dual reconstruction,” IEEE Transactions on Medical Imaging, 2018. arXiv
  18. V. Monga, Y. Li and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” arXiv:1912.10557, 2019. arXiv
  19. G. Wang, J. C. Ye and B. De Man, “Deep learning for tomographic image reconstruction,” Nature Machine Intelligence, vol. 2, no. 12, pp. 737–748, 2020. DOI
  20. A. Buades, B. Coll and J.-M. Morel, “A review of image denoising algorithms, with a new one,” Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 490–530, 2005. DOI
  21. L. I. Rudin, S. Osher and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60, no. 1–4, pp. 259–268, 1992. DOI
  22. A. Chambolle, “An algorithm for total variation minimization and applications,” Journal of Mathematical Imaging and Vision, vol. 20, no. 1–2, pp. 89–97, 2004. DOI
  23. A. Buades, B. Coll and J.-M. Morel, “Non-Local Means Denoising,” Image Processing On Line, vol. 1, pp. 208–212, 2011. DOI · article and implementation