完整搜尋頁面 →

全變分正則化

選讀・IP 05・影像復原與重建 的補充資料(第 7/7 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

白話說:正則化項(regularizer)是一個懲罰項,用來表達「好的答案長什麼樣子」。Wiener 與 CLS 偏好平滑的答案,而全變分(total variation, TV)偏好分段常數的答案:一塊塊平坦的區域,中間隔著銳利的跳變。對許多照片來說,這比「處處平滑」貼切得多。

Rudin、Osher 與 Fatemi 的 ROF 模型 [21] 以下列最佳化問題對影像 ff 去雜訊:

min⁡u 12∥u−f∥22+λ TV(u),TV(u)=∑x,y(∂xu)2+(∂yu)2,\min_{u}\ \tfrac12 \lVert u - f \rVert_2^2 + \lambda\, \mathrm{TV}(u), \qquad \mathrm{TV}(u) = \sum_{x, y} \sqrt{(\partial_x u)^2 + (\partial_y u)^2},

其中 uu 是去雜訊後的影像,ff 是有雜訊的輸入,∂xu\partial_x u 與 ∂yu\partial_y u 是 uu 在相鄰像素間的差分,λ>0\lambda > 0 是正則化強度:λ\lambda 越大,結果越平坦。第一項讓 uu 貼近資料,第二項把梯度的大小加總(L1L_1 型懲罰,也就是大小的總和)。把它和本章前面平滑濾波器背後的懲罰項比較:CLS [3] 懲罰的是拉普拉斯的平方,Tikhonov 型則懲罰梯度的平方 ∥∇u∥22\lVert\nabla u\rVert_2^2。平方型懲罰導出線性濾波器,可以寫成頻域公式,如 Wiener 與 CLS 的公式。L1L_1 型的 TV 懲罰則導出非線性濾波器,沒有這樣的公式,需要 Chambolle 演算法 [22] 這類迭代求解器。同一個目標函數的一維版本,以分裂法求解,在 OPT 08・幾何應用、分裂法與現代方法中有詳細的演算法與實驗;該頁處理的是訊號,不是影像。

為什麼邊緣能保留。 拿一個高度為 1 的跳變,和一條在 100 個取樣點上爬升同樣高度的斜坡比較。TV 加總的是大小,所以兩者的代價都是 1,由資料項來決定取捨。梯度平方的懲罰對跳變收費 1,對斜坡只收約 0.01,因此把每條邊緣變成斜坡比較划算。為什麼會出現階梯。 L1L_1 懲罰偏好剛好為零的梯度,所以即使真實訊號是平緩的斜坡,TV 也傾向輸出平坦的階梯。階梯效應(staircasing)是已知的副作用:紋理與細微的明暗會被壓平。

程式碼 · Python · 27 行
import numpy as np
from skimage import data, img_as_float
from skimage.filters import gaussian
from skimage.metrics import peak_signal_noise_ratio as psnr
from skimage.restoration import denoise_tv_chambolle

clean = img_as_float(data.camera())[:256, :256]
rng = np.random.default_rng(0)
noisy = np.clip(clean + rng.normal(0, 0.08, clean.shape), 0, 1)

# 1) A step and a 100-sample ramp with the same end values: total variation vs squared gradient
step, ramp = np.r_[np.zeros(50), np.ones(50)], np.linspace(0, 1, 100)
tv1d = lambda u: np.abs(np.diff(u)).sum()
l2 = lambda u: (np.diff(u) ** 2).sum()
print(f"TV: step {tv1d(step):.2f}, ramp {tv1d(ramp):.2f} | squared gradient: step {l2(step):.2f}, ramp {l2(ramp):.3f}")

# 2) Image: Gaussian vs TV, each at its best setting from a small sweep
g = max((psnr(clean, gaussian(noisy, s)), s) for s in (0.8, 1.0, 1.2, 1.5))
t = max((psnr(clean, denoise_tv_chambolle(noisy, weight=w)), w) for w in (0.05, 0.1, 0.15))
print(f"noisy {psnr(clean, noisy):.2f} dB | Gaussian s={g[1]}: {g[0]:.2f} dB | TV weight={t[1]}: {t[0]:.2f} dB")

# 3) Staircasing: TV on a noisy smooth sine (the true signal has no flat parts)
x = np.linspace(0, 1, 200)
f = 0.5 + 0.4 * np.sin(2 * np.pi * x) + rng.normal(0, 0.05, 200)
u = denoise_tv_chambolle(f, weight=0.1, eps=1e-8, max_num_iter=5000)
flat = lambda v: np.mean(np.abs(np.diff(v)) < 1e-4)
print(f"share of neighbouring pairs that are flat: noisy {flat(f):.2f}, TV {flat(u):.2f}")

輸出:

程式碼 · 3 行
TV: step 1.00, ramp 1.00 | squared gradient: step 1.00, ramp 0.010
noisy 22.30 dB | Gaussian s=1.0: 29.91 dB | TV weight=0.1: 32.20 dB
share of neighbouring pairs that are flat: noisy 0.00, TV 0.69

這裡 denoise_tv_chambolle 的 weight 參數扮演上式的 λ\lambda。在與 NLM 補充資料相同的雜訊裁切上,TV 比最佳的高斯濾波器高約 2.3 dB(NLM 又比 TV 高約 0.65 dB);而在真實訊號沒有平坦區段的正弦波上,TV 輸出的相鄰點對中有 69% 是平的,這就是階梯效應。常見的錯誤是只用 PSNR 來評斷 λ\lambda:TV 的結果可能分數不錯,看起來卻像卡通,所以也要用眼睛看影像。

參考文獻

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 5. publisher page
  2. H. Hwang and R. A. Haddad, “Adaptive median filters: new algorithms and results,” IEEE Transactions on Image Processing, vol. 4, no. 4, pp. 499–502, 1995. DOI
  3. B. R. Hunt, “The application of constrained least squares estimation to image restoration by digital computer,” IEEE Transactions on Computers, vol. C-22, no. 9, pp. 805–812, 1973. DOI
  4. A. C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging, IEEE Press, 1988 (SIAM reprint 2001). free online edition
  5. L. A. Shepp and B. F. Logan, “The Fourier reconstruction of a head section,” IEEE Transactions on Nuclear Science, vol. 21, no. 3, pp. 21–43, 1974. DOI
  6. scikit-image developers, “Radon transform,” scikit-image documentation (gallery example). docs
  7. A. Buades, B. Coll and J.-M. Morel, “A non-local algorithm for image denoising,” IEEE CVPR, vol. 2, pp. 60–65, 2005. DOI
  8. K. Dabov, A. Foi, V. Katkovnik and K. Egiazarian, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080–2095, 2007. DOI
  9. K. Zhang, W. Zuo, Y. Chen, D. Meng and L. Zhang, “Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3142–3155, 2017. doi · arXiv
  10. C. Tian, L. Fei, W. Zheng, Y. Xu, W. Zuo and C.-W. Lin, “Deep learning on image denoising: An overview,” Neural Networks, vol. 131, pp. 251–275, 2020. arXiv
  11. M. Elad, B. Kawar and G. Vaksman, “Image denoising: The deep learning revolution and beyond — A survey paper,” SIAM Journal on Imaging Sciences, vol. 16, no. 3, pp. 1594–1654, 2023. arXiv
  12. K. Zhang, W. Ren, W. Luo, W.-S. Lai, B. Stenger, M.-H. Yang and H. Li, “Deep image deblurring: A survey,” International Journal of Computer Vision, 2022. arXiv
  13. G. Ongie, A. Jalal, C. A. Metzler, R. G. Baraniuk, A. G. Dimakis and R. Willett, “Deep learning techniques for inverse problems in imaging,” arXiv:2005.06001, 2020. arXiv
  14. U. S. Kamilov, C. A. Bouman, G. T. Buzzard and B. Wohlberg, “Plug-and-play methods for integrating physical and learned models in computational imaging: Theory, algorithms, and applications,” IEEE Signal Processing Magazine, vol. 40, no. 1, pp. 85–97, 2023. arXiv
  15. K. Zhang, Y. Li, W. Zuo, L. Zhang, L. Van Gool and R. Timofte, “Plug-and-play image restoration with deep denoiser prior,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 10, pp. 6360–6376, 2022. arXiv
  16. K. H. Jin, M. T. McCann, E. Froustey and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Transactions on Image Processing, vol. 26, no. 9, pp. 4509–4522, 2017. arXiv
  17. J. Adler and O. Öktem, “Learned primal-dual reconstruction,” IEEE Transactions on Medical Imaging, 2018. arXiv
  18. V. Monga, Y. Li and Y. C. Eldar, “Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing,” arXiv:1912.10557, 2019. arXiv
  19. G. Wang, J. C. Ye and B. De Man, “Deep learning for tomographic image reconstruction,” Nature Machine Intelligence, vol. 2, no. 12, pp. 737–748, 2020. DOI
  20. A. Buades, B. Coll and J.-M. Morel, “A review of image denoising algorithms, with a new one,” Multiscale Modeling & Simulation, vol. 4, no. 2, pp. 490–530, 2005. DOI
  21. L. I. Rudin, S. Osher and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D: Nonlinear Phenomena, vol. 60, no. 1–4, pp. 259–268, 1992. DOI
  22. A. Chambolle, “An algorithm for total variation minimization and applications,” Journal of Mathematical Imaging and Vision, vol. 20, no. 1–2, pp. 89–97, 2004. DOI
  23. A. Buades, B. Coll and J.-M. Morel, “Non-Local Means Denoising,” Image Processing On Line, vol. 1, pp. 208–212, 2011. DOI · article and implementation