完整搜尋頁面 →

導引濾波器:沿著導引影像平滑

選讀・IP 03・強度轉換與空間濾波 的補充資料(第 4/5 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

方框與高斯濾波器只看像素的位置,所以會跨越邊緣一起平均。導引濾波器(guided filter)同樣會平滑影像,但邊緣由另一張影像,也就是導引影像(guide)來決定,因此輸出保留了導引影像的邊緣,雜訊則被平均掉。在自我導引(self-guided)的情況下,導引影像就是輸入影像本身。

He、Sun 與 Tang [8] 假設在每個小視窗 wkw_k 之中,輸出都是導引影像的線性函數:

qi=akIi+bk,i∈wk.q_i = a_k I_i + b_k, \qquad i \in w_k .

其中 II 是導引影像,qq 是輸出,wkw_k 是以像素 kk 為中心、半徑為 rr 的方形視窗(共 (2r+1)2(2r+1)^2 個像素)。視窗內 ∇q=ak∇I\nabla q = a_k \nabla I,所以只有 II 有邊緣的地方,qq 才可能有邊緣。係數 ak,bka_k, b_k 的選法是讓 qq 盡量接近輸入 pp,並以 ϵak2\epsilon a_k^2 懲罰過大的斜率:

min⁡ak, bk∑i∈wk((akIi+bk−pi)2+ϵ ak2)  ⇒  ak=(Ip)‾k−μkpˉkσk2+ϵ,bk=pˉk−akμk.\min_{a_k,\, b_k} \sum_{i \in w_k} \Big( (a_k I_i + b_k - p_i)^2 + \epsilon\, a_k^2 \Big) \;\Rightarrow\; a_k = \frac{\overline{(Ip)}_k - \mu_k \bar p_k}{\sigma_k^2 + \epsilon}, \qquad b_k = \bar p_k - a_k \mu_k .

其中 μk\mu_k 與 σk2\sigma_k^2 是 II 在 wkw_k 內的平均與變異數,pˉk\bar p_k 是 pp 在 wkw_k 內的平均,(Ip)‾k\overline{(Ip)}_k 是乘積 IipiI_i p_i 在 wkw_k 內的平均,ϵ>0\epsilon > 0 是正則化(regularization)參數。一個像素 ii 同時落在 (2r+1)2(2r+1)^2 個視窗內,每個視窗給出的 (ak,bk)(a_k, b_k) 略有不同,所以最後的輸出取它們的平均:

qi=aˉiIi+bˉi,aˉi=1∣w∣∑k: i∈wkak,bˉi=1∣w∣∑k: i∈wkbk,q_i = \bar a_i I_i + \bar b_i, \qquad \bar a_i = \frac{1}{|w|} \sum_{k:\, i \in w_k} a_k, \quad \bar b_i = \frac{1}{|w|} \sum_{k:\, i \in w_k} b_k ,

其中 ∣w∣=(2r+1)2|w| = (2r+1)^2,求和範圍是包含像素 ii 的所有視窗。這個平均本身也是一次方框濾波。

ϵ\epsilon 的作用。 在自我導引(p=Ip = I)時,ak=σk2/(σk2+ϵ)a_k = \sigma_k^2 / (\sigma_k^2 + \epsilon)。在平坦區域,σk2≪ϵ\sigma_k^2 \ll \epsilon,所以 ak≈0a_k \approx 0、bk≈pˉkb_k \approx \bar p_k,輸出就是單純的方框平均。在邊緣上,σk2≫ϵ\sigma_k^2 \gg \epsilon,所以 ak≈1a_k \approx 1、bk≈0b_k \approx 0,像素幾乎原樣通過。因此 ϵ\epsilon 越大,平滑越強;ϵ\epsilon 越小,越能保留邊緣。由於 ϵ\epsilon 是和變異數比較,它的尺度是強度範圍的平方:強度在 [0,1][0, 1] 時,ϵ=0.01\epsilon = 0.01 表示約 0.10.1 的起伏會被視為邊緣。

計算量。 上面所有量(μk\mu_k、pˉk\bar p_k、(Ip)‾k\overline{(Ip)}_k、I2‾k\overline{I^2}_k、aˉi\bar a_i、bˉi\bar b_i)都是方框平均,而方框平均藉由滾動累加,每個像素只需固定的運算量,所以整個濾波器對 NN 個像素是 O(N)O(N),且與 rr 無關。它不必對每個像素搜尋強度相近的鄰居,這正是雙邊濾波器計算昂貴的原因。

程式碼 · Python · 24 行
import numpy as np
from scipy.ndimage import uniform_filter, sobel
from skimage import data, img_as_float
from skimage.metrics import peak_signal_noise_ratio as psnr

def guided_filter(I, p, r, eps):
    box = lambda x: uniform_filter(x, size=2 * r + 1, mode="reflect")
    mu, pbar = box(I), box(p)
    var = box(I * I) - mu**2
    a = (box(I * p) - mu * pbar) / (var + eps)
    b = pbar - a * mu
    return box(a) * I + box(b)

clean = img_as_float(data.camera())
noisy = clean + np.random.default_rng(0).normal(0, 0.08, clean.shape)
r, eps = 4, 0.1**2
box_out = uniform_filter(noisy, size=2 * r + 1, mode="reflect")
gf_out = guided_filter(noisy, noisy, r, eps)          # self-guided

grad = lambda x: np.hypot(sobel(x, 0), sobel(x, 1))
edges = grad(clean) > np.percentile(grad(clean), 90)  # strongest 10% of edge pixels
for name, out in [("noisy", noisy), ("box", box_out), ("guided", gf_out)]:
    print(f"{name:7s} PSNR {psnr(clean, out, data_range=1):5.2f} dB   edge gradient {grad(out)[edges].mean():.3f}")
print(f"clean   edge gradient {grad(clean)[edges].mean():.3f}")

輸出

noisy   PSNR 21.93 dB   edge gradient 1.054
box     PSNR 23.88 dB   edge gradient 0.221
guided  PSNR 27.03 dB   edge gradient 0.810
clean   edge gradient 1.005

兩種濾波器使用相同的視窗(9×9),處理同一張帶雜訊的影像(雜訊標準差 0.08)。導引濾波器的輸出比方框濾波器的輸出更接近乾淨影像,PSNR 高出 3.15 dB。第二欄以 Sobel 梯度強度的平均值衡量邊緣銳利度,取乾淨影像中邊緣最強的 10% 像素(Sobel 單位,未正規化):方框濾波器只保留乾淨值的約五分之一(0.221 對 1.005),導引濾波器則保留約五分之四(0.810)。帶雜訊影像的數值略高於乾淨影像,只是因為雜訊本身也增加了梯度。作為驗證,閉式解的程式碼與逐一視窗計算 ak,bka_k, b_k 的直接迴圈,在影像一小塊裁切區域的非邊界處相符,誤差約 10−1510^{-15}。

參考文獻

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 3. publisher page
  2. S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. ter Haar Romeny, J. B. Zimmerman and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Computer Vision, Graphics, and Image Processing, 1987. doi
  3. K. Zuiderveld, “Contrast limited adaptive histogram equalization,” in P. S. Heckbert (ed.), Graphics Gems IV, Morgan Kaufmann, 1994, pp. 474–485. entry
  4. A. M. Reza, “Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement,” Journal of VLSI Signal Processing, 2004. doi
  5. D. Coltuc, P. Bolon and J.-M. Chassery, “Exact histogram specification,” IEEE Transactions on Image Processing, 2006. doi
  6. C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. IEEE International Conference on Computer Vision (ICCV), 1998. pdf
  7. S. Paris, P. Kornprobst, J. Tumblin and F. Durand, “Bilateral filtering: Theory and applications,” Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1–73, 2009. doi
  8. K. He, J. Sun and X. Tang, “Guided image filtering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6):1397–1409, 2013 (first presented at ECCV 2010). doi
  9. P. Milanfar, “A tour of modern image filtering,” IEEE Signal Processing Magazine, vol. 30, no. 1, pp. 106–128, 2013. IEEE Xplore
  10. S. Gingichashvili and D. Lischinski, “Evaluation and comparison of edge-preserving filters,” arXiv:2012.13778, 2020. arXiv
  11. Y. Qi, Z. Yang, W. Sun et al., “A comprehensive overview of image enhancement techniques,” Archives of Computational Methods in Engineering, 2021. doi
  12. C. Li, C. Guo, L. Han, J. Jiang, M.-M. Cheng, J. Gu and C. C. Loy, “Low-light image and video enhancement using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (arXiv:2104.10729). arXiv
  13. C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong and R. Cong, “Zero-reference deep curve estimation for low-light image enhancement,” in Proc. IEEE/CVF CVPR, 2020. arXiv
  14. H. Zeng, J. Cai, L. Li, Z. Cao and L. Zhang, “Learning image-adaptive 3D lookup tables for high performance photo enhancement in real-time,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
  15. M. Gharbi, J. Chen, J. T. Barron, S. W. Hasinoff and F. Durand, “Deep bilateral learning for real-time image enhancement,” ACM Transactions on Graphics (SIGGRAPH), 2017. arXiv
  16. H. Wu, S. Zheng, J. Zhang and K. Huang, “Fast end-to-end trainable guided filter,” in Proc. IEEE/CVF CVPR, 2018. arXiv
  17. Y. Cai, H. Bian, J. Lin, H. Wang, R. Timofte and Y. Zhang, “Retinexformer: One-stage Retinex-based transformer for low-light image enhancement,” in Proc. IEEE/CVF ICCV, 2023. arXiv
  18. S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, vol. 81, no. 1, pp. 24–52, 2009 (first presented at ECCV 2006). doi