完整搜尋頁面 →

雙邊濾波器:只和長得像的像素平均

選讀・IP 03・強度轉換與空間濾波 的補充資料(第 5/5 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。

導覽:本頁與本系列: 導覽:本頁與本系列

高斯濾波器對鄰居只問一個問題:它有多近?雙邊濾波器(bilateral filter)問兩個問題:它有多近,以及它長得像我嗎?邊緣另一側的鄰居看起來差很多,幾乎會被忽略,所以邊緣不會被抹糊。

Tomasi 與 Manduchi [6] 把每個像素 pp 換成視窗 N(p)\mathcal N(p)(半徑為 rr)內各像素 qq 的加權平均:

I^p=1Wp∑q∈N(p)exp⁡ ⁣(−∥p−q∥22σs2)⏟空間權重  exp⁡ ⁣(−(Ip−Iq)22σr2)⏟值域權重 Iq,Wp=∑q∈N(p)exp⁡ ⁣(−∥p−q∥22σs2)exp⁡ ⁣(−(Ip−Iq)22σr2).\hat I_p = \frac{1}{W_p} \sum_{q \in \mathcal N(p)} \underbrace{\exp\!\Big(-\frac{\lVert p - q \rVert^2}{2\sigma_s^2}\Big)}_{\text{空間權重}} \; \underbrace{\exp\!\Big(-\frac{(I_p - I_q)^2}{2\sigma_r^2}\Big)}_{\text{值域權重}} \, I_q, \qquad W_p = \sum_{q \in \mathcal N(p)} \exp\!\Big(-\frac{\lVert p - q \rVert^2}{2\sigma_s^2}\Big) \exp\!\Big(-\frac{(I_p - I_q)^2}{2\sigma_r^2}\Big).

其中 IpI_p 與 IqI_q 是像素 pp 與 qq 的輸入強度,∥p−q∥\lVert p - q \rVert 是兩個像素位置之間的距離,I^p\hat I_p 是輸出,WpW_p 是權重總和,讓權重加起來為 1。空間尺度 σs\sigma_s(單位為像素)決定鄰居可以多遠仍算數;當 σr→∞\sigma_r \to \infty 時,值域因子等於 1,濾波器就恰好是高斯濾波器。值域尺度 σr\sigma_r(單位為強度)決定鄰居可以差多少仍算「像我」。它的角色類似導引濾波器的 ϵ\epsilon:遠大於 σr\sigma_r 的差異會被當成邊緣。

邊緣為什麼能保留。 假設某個像素強度為 0.2,緊鄰一道升到 0.7 的階梯。階梯另一側的鄰居與它差 0.5,若 σr=0.15\sigma_r = 0.15,它們的值域權重是 exp⁡(−0.52/(2⋅0.152))≈0.004\exp(-0.5^2 / (2 \cdot 0.15^2)) \approx 0.004。平均幾乎完全取自同一側的像素,所以階梯依然銳利,而兩側各自的雜訊被平均掉。σr\sigma_r 要選得比雜訊強度略大,且小於想保留的邊緣對比。同一條規則也說明了限制:孤立的雜訊像素(脈衝雜訊)與所有鄰居都不同,因此大多只保留它自己;對比高於 σr\sigma_r 的細緻紋理會被保留,而不是被平滑。

計算量。 權重取決於影像的值,所以沒有固定的卷積核可以預先算好,也沒有可分離或 FFT 的捷徑。直接求和時每個像素要走訪 (2r+1)2(2r+1)^2 個鄰居,對 NN 個像素是 O(Nr2)O(N r^2);可分離的高斯濾波器是 O(Nr)O(Nr),導引濾波器是 O(N)O(N)。Paris 與 Durand [18] 證明這個濾波器可以寫成三維空間(兩個位置軸加上強度軸)中的高斯卷積再做一次除法,而且這個卷積可以在空間與強度都粗略降取樣的網格上計算;綜述 [7] 涵蓋了這個方法與其他加速法。

基底層與細節層。 以雙邊濾波器處理影像會得到保留強邊緣的平滑基底層(base layer),把它減掉則剩下細節層(detail layer)。色調映射與細節增強會分別編輯這兩層再加回去 [7],所以需要保邊濾波器:若基底層用高斯濾波器,編輯後邊緣周圍會出現光暈。

瑕疵。 輸出是輸入像素的加權平均,而權重隨像素而變,所以沒有任何條件迫使它對輸入保持區域線性。在有雜訊且緩慢變化的區域,它可能把平滑的斜坡切成平台與跳階(下方有量測),在細節增強時,強邊緣附近也可能產生梯度反轉(gradient reversal)[8]。導引濾波器的視窗模型 qi=akIi+bkq_i = a_k I_i + b_k 在視窗內給出 ∇q=ak∇I\nabla q = a_k \nabla I,而在自我導引時 ak=σk2/(σk2+ϵ)a_k = \sigma_k^2 / (\sigma_k^2 + \epsilon) 介於 0 與 1 之間,所以在一個視窗內,輸出的梯度不會與導引影像的梯度反向。這是模型本身的性質,也與 [8] 所報告的「導引濾波器在強邊緣附近表現較好」一致;至於雜訊斜坡上的平台效應則是另一回事,本篇並未對導引濾波器做這項測試。

程式碼 · Python · 37 行
import numpy as np
from scipy.ndimage import gaussian_filter, sobel
from skimage import data, img_as_float
from skimage.metrics import peak_signal_noise_ratio as psnr

def bilateral(img, r, sigma_s, sigma_r):
    H, W = img.shape
    pad = np.pad(img, r, mode="reflect")
    num, den = np.zeros_like(img), np.zeros_like(img)
    for dy in range(-r, r + 1):              # (2r+1)^2 shifted copies of the image
        for dx in range(-r, r + 1):
            q = pad[r + dy : r + dy + H, r + dx : r + dx + W]   # neighbor value I_q
            w = np.exp(-(dx * dx + dy * dy) / (2 * sigma_s**2)) \
                * np.exp(-((q - img) ** 2) / (2 * sigma_r**2))
            num += w * q
            den += w                          # den is W_p
    return num / den

clean = img_as_float(data.camera())
noisy = clean + np.random.default_rng(0).normal(0, 0.08, clean.shape)
grad = lambda x: np.hypot(sobel(x, 0), sobel(x, 1))
edges = grad(clean) > np.percentile(grad(clean), 90)    # strongest 10% of edge pixels

gauss = gaussian_filter(noisy, 2, truncate=3, mode="reflect")   # same sigma_s, same 13x13 support
bil = bilateral(noisy, r=6, sigma_s=2, sigma_r=0.15)
for name, out in [("noisy", noisy), ("gaussian", gauss), ("bilateral", bil)]:
    print(f"{name:9s} PSNR {psnr(clean, out, data_range=1):5.2f} dB   edge gradient {grad(out)[edges].mean():.3f}")
print(f"clean     edge gradient {grad(clean)[edges].mean():.3f}")

# Noisy linear ramp: how erratic are the step-to-step differences? (1.0 = the true slope)
n = 200
ramp = np.tile(np.linspace(0.2, 0.8, n), (n, 1))
nr = ramp + np.random.default_rng(1).normal(0, 0.02, ramp.shape)
step = 0.6 / (n - 1)
for name, out in [("gaussian", gaussian_filter(nr, 2, truncate=3)), ("bilateral", bilateral(nr, 6, 2, 0.05))]:
    d = np.diff(out[:, 20:-20], axis=1)
    print(f"ramp {name:9s} std of step-to-step difference / true slope = {d.std() / step:.2f}")

輸出

noisy     PSNR 21.93 dB   edge gradient 1.054
gaussian  PSNR 25.72 dB   edge gradient 0.346
bilateral PSNR 28.81 dB   edge gradient 0.787
clean     edge gradient 1.005
ramp gaussian  std of step-to-step difference / true slope = 0.33
ramp bilateral std of step-to-step difference / true slope = 1.56

兩個濾波器都使用 σs=2\sigma_s = 2 與相同的 13×13 支撐範圍,並在與導引濾波器範例相同的雜訊影像上(雜訊標準差 0.08)以相同的兩項指標比較。雙邊濾波器的輸出比高斯濾波器的輸出更接近乾淨影像 3.09 dB,並且保留了乾淨影像約 78% 的邊緣梯度(0.787 對 1.005),高斯濾波器只保留約 34%(0.346)。邊緣銳利度的定義同前:乾淨影像中最強的 10% 邊緣像素上,Sobel 梯度強度的平均。作為對照,上一則補充的導引濾波器(9×9 視窗、ϵ=0.12\epsilon = 0.1^2)在同一張影像上得到 27.03 dB 與 0.810,所以兩種保邊濾波器的表現在同一範圍,且數字取決於各自選用的參數。斜坡測試(雜訊標準差 0.02、σr=0.05\sigma_r = 0.05)量化了平台與跳階的瑕疵:理想斜坡的比值是 0,而雙邊輸出的比值接近高斯輸出的五倍。

參考文獻

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 3. publisher page
  2. S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. ter Haar Romeny, J. B. Zimmerman and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Computer Vision, Graphics, and Image Processing, 1987. doi
  3. K. Zuiderveld, “Contrast limited adaptive histogram equalization,” in P. S. Heckbert (ed.), Graphics Gems IV, Morgan Kaufmann, 1994, pp. 474–485. entry
  4. A. M. Reza, “Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement,” Journal of VLSI Signal Processing, 2004. doi
  5. D. Coltuc, P. Bolon and J.-M. Chassery, “Exact histogram specification,” IEEE Transactions on Image Processing, 2006. doi
  6. C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. IEEE International Conference on Computer Vision (ICCV), 1998. pdf
  7. S. Paris, P. Kornprobst, J. Tumblin and F. Durand, “Bilateral filtering: Theory and applications,” Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1–73, 2009. doi
  8. K. He, J. Sun and X. Tang, “Guided image filtering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6):1397–1409, 2013 (first presented at ECCV 2010). doi
  9. P. Milanfar, “A tour of modern image filtering,” IEEE Signal Processing Magazine, vol. 30, no. 1, pp. 106–128, 2013. IEEE Xplore
  10. S. Gingichashvili and D. Lischinski, “Evaluation and comparison of edge-preserving filters,” arXiv:2012.13778, 2020. arXiv
  11. Y. Qi, Z. Yang, W. Sun et al., “A comprehensive overview of image enhancement techniques,” Archives of Computational Methods in Engineering, 2021. doi
  12. C. Li, C. Guo, L. Han, J. Jiang, M.-M. Cheng, J. Gu and C. C. Loy, “Low-light image and video enhancement using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (arXiv:2104.10729). arXiv
  13. C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong and R. Cong, “Zero-reference deep curve estimation for low-light image enhancement,” in Proc. IEEE/CVF CVPR, 2020. arXiv
  14. H. Zeng, J. Cai, L. Li, Z. Cao and L. Zhang, “Learning image-adaptive 3D lookup tables for high performance photo enhancement in real-time,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
  15. M. Gharbi, J. Chen, J. T. Barron, S. W. Hasinoff and F. Durand, “Deep bilateral learning for real-time image enhancement,” ACM Transactions on Graphics (SIGGRAPH), 2017. arXiv
  16. H. Wu, S. Zheng, J. Zhang and K. Huang, “Fast end-to-end trainable guided filter,” in Proc. IEEE/CVF CVPR, 2018. arXiv
  17. Y. Cai, H. Bian, J. Lin, H. Wang, R. Timofte and Y. Zhang, “Retinexformer: One-stage Retinex-based transformer for low-light image enhancement,” in Proc. IEEE/CVF ICCV, 2023. arXiv
  18. S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, vol. 81, no. 1, pp. 24–52, 2009 (first presented at ECCV 2006). doi