完整搜尋頁面 →

非局部平均去雜訊

選讀・IP 05・影像復原與重建 的補充資料(第 6/7 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

白話說:要猜一個有雜訊的像素原本是什麼值,不要只問它的鄰居,而是去問附近所有「長得像它」的地方。校對一個被污漬蓋住的字也是這樣:把它和別處出現的同一個字比對,再綜合讀到的結果。

高斯濾波器平均的是距離近的像素;在邊緣附近它會把兩側混在一起,所以會模糊。非局部平均(non-local means, NLM)[7] 則平均「周圍區塊(patch)相似」的像素,只要在搜尋視窗內,離多遠都可以。精確的寫法是

u^(p)=1Z(p)∑q∈Ω(p)w(p,q) f(q),w(p,q)=exp⁡ ⁣(−∥Pp−Pq∥2h2),Z(p)=∑q∈Ω(p)w(p,q),\hat u(p) = \frac{1}{Z(p)} \sum_{q \in \Omega(p)} w(p, q)\, f(q), \qquad w(p, q) = \exp\!\left(-\frac{\lVert P_p - P_q \rVert^2}{h^2}\right), \qquad Z(p) = \sum_{q \in \Omega(p)} w(p, q),

其中 ff 是有雜訊的影像,u^(p)\hat u(p) 是像素 pp 的估計值,Ω(p)\Omega(p) 是搜尋視窗(有資格為 pp 投票的像素),PpP_p 是以 pp 為中心的小區塊(例如 5×55 \times 5 像素)的灰階值所組成的向量,∥⋅∥\lVert\cdot\rVert 是歐氏範數,h>0h > 0 是強度參數,Z(p)Z(p) 是使權重總和為 1 的正規化常數。兩個區塊一致時權重接近 1,差距遠大於 hh 時權重趨近 0。hh 決定了「相似」的標準:hh 太小,符合條件的區塊太少,雜訊去不乾淨;hh 太大,不相像的區塊也來投票,影像就變模糊。常見的經驗法則是讓 hh 接近雜訊標準差 σ\sigma。scikit-image 的實作也可以傳入 σ\sigma,用它在計算區塊權重時考慮雜訊變異數。

為什麼有效?如果影像中有 MM 個真正相像的區塊,平均它們的中心像素可以把雜訊變異數降低約 MM 倍;而且邊緣不會變模糊,因為只有含有同一條邊緣的區塊會投票。所以 NLM 在紋理與重複結構(窗戶、磚牆、毛皮)上表現很好,這些地方有大量相像的區塊。至於獨一無二的細節,找不到相像區塊,雜訊會留下來,或是被不夠相像的區塊平均而失真。

計算成本。 設影像有 NN 個像素、搜尋視窗為 S×SS \times S、區塊為 k×kk \times k,直接計算約需 NS2k2N S^2 k^2 次運算。比較快的版本用累計和(積分影像)一次算出某個位移下所有區塊的距離,免去 k2k^2 這個因子;另一個辦法是縮小 SS。BM3D [8] 等區塊匹配方法把同樣的想法再推進一步。同一組作者的期刊論文 [20] 把 NLM 與其他去雜訊方法放在一起比較。

以下程式在 camera 測試影像的 256×256256 \times 256 裁切上加入固定種子的高斯雜訊(在 0–1 範圍內 σ=0.08\sigma = 0.08),再比較一組小範圍掃描中最佳的高斯濾波器與 NLM(fast_mode=True,13×1313 \times 13 搜尋視窗,5×55 \times 5 區塊)。

程式碼 · Python · 20 行
import numpy as np
from skimage import data, img_as_float
from skimage.filters import gaussian
from skimage.metrics import peak_signal_noise_ratio as psnr
from skimage.restoration import denoise_nl_means

clean = img_as_float(data.camera())[:256, :256]           # 256 x 256 crop
sigma = 0.08                                              # noise standard deviation
rng = np.random.default_rng(0)
noisy = np.clip(clean + rng.normal(0, sigma, clean.shape), 0, 1)

# Gaussian smoothing, best of a small sweep over its width s
g = max((psnr(clean, gaussian(noisy, s)), s) for s in (0.8, 1.0, 1.2, 1.5))

# Non-local means: 5x5 patches, search window 13x13 (patch_distance=6), h near sigma
nlm = {hf: denoise_nl_means(noisy, patch_size=5, patch_distance=6, h=hf * sigma,
                            sigma=sigma, fast_mode=True) for hf in (0.6, 0.8, 1.0)}
for hf, out in nlm.items():
    print(f"NLM h = {hf} sigma: {psnr(clean, out):.2f} dB")
print(f"noisy {psnr(clean, noisy):.2f} dB | Gaussian s={g[1]}: {g[0]:.2f} dB")

輸出:

程式碼 · 4 行
NLM h = 0.6 sigma: 32.80 dB
NLM h = 0.8 sigma: 32.85 dB
NLM h = 1.0 sigma: 32.52 dB
noisy 22.30 dB | Gaussian s=1.0: 29.91 dB

在這個裁切與這組種子上,NLM 比最佳的高斯濾波器高出約 3 dB,三個 hh 值之間的差距在 0.35 dB 以內。這只是一張影像、一次雜訊抽樣,請當成示範,不要當成方法的排名。常見的錯誤是為了「多平滑一點」把 hh 設得遠大於 σ\sigma:結果只是多繞一圈的模糊。

參考文獻

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 5. publisher page
  2. H. Hwang and R. A. Haddad, “Adaptive median filters: new algorithms and results,” IEEE Transactions on Image Processing, vol. 4, no. 4, pp. 499–502, 1995. DOI
  3. B. R. Hunt, “The application of constrained least squares estimation to image restoration by digital computer,” IEEE Transactions on Computers, vol. C-22, no. 9, pp. 805–812, 1973. DOI
  4. A. C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, IEEE Press, 1988 (SIAM reprint 2001). free online edition
  5. L. A. Shepp and B. F. Logan, “The Fourier reconstruction of a head section,” IEEE Transactions on Nuclear Science, vol. 21, no. 3, pp. 21–43, 1974. DOI
  6. scikit-image developers, “Radon transform,” scikit-image documentation (gallery example). docs
  7. A. Buades, B. Coll and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, vol. 2, pp. 60–65, 2005. DOI
  8. K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007. DOI
  9. K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, “Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017. doi · arXiv
  10. C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo and C.-W. Lin, “Deep learning on image denoising: An overview,” Neural Networks, vol. 131, pp. 251–275, 2020. arXiv
  11. M. Elad, B. Kawar and G. Vaksman, “Image denoising: The deep learning revolution and beyond — A survey paper,” SIAM Journal on Imaging Sciences, vol. 16, no. 3, pp. 1594–1654, 2023. arXiv
  12. K. Zhang, W. Ren, W. Luo, W.-S. Lai, B. Stenger, M.-H. Yang and H. Li, “Deep image deblurring: A survey,” International Journal of Computer Vision, 2022. arXiv
  13. G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis and R. Willett, “Deep learning techniques for inverse problems in imaging,” arXiv:2005.06001, 2020. arXiv
  14. U. S. Kamilov, C. A. Bouman, G. T. Buzzard and B. Wohlberg, “Plug-and-play methods for integrating physical and learned models in computational imaging: Theory, algorithms, and applications,” IEEE Signal Processing Magazine, vol. 40, no. 1, pp. 85–97, 2023. arXiv
  15. K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool and R. Timofte, “Plug-and-play image restoration with deep denoiser prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6360–6376, 2022. arXiv
  16. K. H. Jin, M. T. McCann, E. Froustey and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4509–4522, 2017. arXiv
  17. J. Adler and O. Öktem, “Learned primal-dual reconstruction,” IEEE Transactions on Medical Imaging, 2018. arXiv
  18. V. Monga, Y. Li and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” arXiv:1912.10557, 2019. arXiv
  19. G. Wang, J. C. Ye and B. De Man, “Deep learning for tomographic image reconstruction,” Nature Machine Intelligence, vol. 2, no. 12, pp. 737–748, 2020. DOI
  20. A. Buades, B. Coll and J.-M. Morel, “A review of image denoising algorithms, with a new one,” Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 490–530, 2005. DOI
  21. L. I. Rudin, S. Osher and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60, no. 1–4, pp. 259–268, 1992. DOI
  22. A. Chambolle, “An algorithm for total variation minimization and applications,” Journal of Mathematical Imaging and Vision, vol. 20, no. 1–2, pp. 89–97, 2004. DOI
  23. A. Buades, B. Coll and J.-M. Morel, “Non-Local Means Denoising,” Image Processing On Line, vol. 1, pp. 208–212, 2011. DOI · article and implementation