雙邊濾波器:只和長得像的像素平均
選讀・IP 03・強度轉換與空間濾波 的補充資料(第 5/5 則)。主線不依賴這則補充;引用編號對應本頁底部的參考文獻。
導覽:本頁與本系列: 導覽:本頁與本系列
高斯濾波器對鄰居只問一個問題:它有多近?雙邊濾波器(bilateral filter)問兩個問題:它有多近,以及它長得像我嗎?邊緣另一側的鄰居看起來差很多,幾乎會被忽略,所以邊緣不會被抹糊。
Tomasi 與 Manduchi [6] 把每個像素 換成視窗 (半徑為 )內各像素 的加權平均:
其中 與 是像素 與 的輸入強度, 是兩個像素位置之間的距離, 是輸出, 是權重總和,讓權重加起來為 1。空間尺度 (單位為像素)決定鄰居可以多遠仍算數;當 時,值域因子等於 1,濾波器就恰好是高斯濾波器。值域尺度 (單位為強度)決定鄰居可以差多少仍算「像我」。它的角色類似導引濾波器的 :遠大於 的差異會被當成邊緣。
邊緣為什麼能保留。 假設某個像素強度為 0.2,緊鄰一道升到 0.7 的階梯。階梯另一側的鄰居與它差 0.5,若 ,它們的值域權重是 。平均幾乎完全取自同一側的像素,所以階梯依然銳利,而兩側各自的雜訊被平均掉。 要選得比雜訊強度略大,且小於想保留的邊緣對比。同一條規則也說明了限制:孤立的雜訊像素(脈衝雜訊)與所有鄰居都不同,因此大多只保留它自己;對比高於 的細緻紋理會被保留,而不是被平滑。
計算量。 權重取決於影像的值,所以沒有固定的卷積核可以預先算好,也沒有可分離或 FFT 的捷徑。直接求和時每個像素要走訪 個鄰居,對 個像素是 ;可分離的高斯濾波器是 ,導引濾波器是 。Paris 與 Durand [18] 證明這個濾波器可以寫成三維空間(兩個位置軸加上強度軸)中的高斯卷積再做一次除法,而且這個卷積可以在空間與強度都粗略降取樣的網格上計算;綜述 [7] 涵蓋了這個方法與其他加速法。
基底層與細節層。 以雙邊濾波器處理影像會得到保留強邊緣的平滑基底層(base layer),把它減掉則剩下細節層(detail layer)。色調映射與細節增強會分別編輯這兩層再加回去 [7],所以需要保邊濾波器:若基底層用高斯濾波器,編輯後邊緣周圍會出現光暈。
瑕疵。 輸出是輸入像素的加權平均,而權重隨像素而變,所以沒有任何條件迫使它對輸入保持區域線性。在有雜訊且緩慢變化的區域,它可能把平滑的斜坡切成平台與跳階(下方有量測),在細節增強時,強邊緣附近也可能產生梯度反轉(gradient reversal)[8]。導引濾波器的視窗模型 在視窗內給出 ,而在自我導引時 介於 0 與 1 之間,所以在一個視窗內,輸出的梯度不會與導引影像的梯度反向。這是模型本身的性質,也與 [8] 所報告的「導引濾波器在強邊緣附近表現較好」一致;至於雜訊斜坡上的平台效應則是另一回事,本篇並未對導引濾波器做這項測試。
程式碼 · Python · 37 行
import numpy as np
from scipy.ndimage import gaussian_filter, sobel
from skimage import data, img_as_float
from skimage.metrics import peak_signal_noise_ratio as psnr
def bilateral(img, r, sigma_s, sigma_r):
H, W = img.shape
pad = np.pad(img, r, mode="reflect")
num, den = np.zeros_like(img), np.zeros_like(img)
for dy in range(-r, r + 1): # (2r+1)^2 shifted copies of the image
for dx in range(-r, r + 1):
q = pad[r + dy : r + dy + H, r + dx : r + dx + W] # neighbor value I_q
w = np.exp(-(dx * dx + dy * dy) / (2 * sigma_s**2)) \
* np.exp(-((q - img) ** 2) / (2 * sigma_r**2))
num += w * q
den += w # den is W_p
return num / den
clean = img_as_float(data.camera())
noisy = clean + np.random.default_rng(0).normal(0, 0.08, clean.shape)
grad = lambda x: np.hypot(sobel(x, 0), sobel(x, 1))
edges = grad(clean) > np.percentile(grad(clean), 90) # strongest 10% of edge pixels
gauss = gaussian_filter(noisy, 2, truncate=3, mode="reflect") # same sigma_s, same 13x13 support
bil = bilateral(noisy, r=6, sigma_s=2, sigma_r=0.15)
for name, out in [("noisy", noisy), ("gaussian", gauss), ("bilateral", bil)]:
print(f"{name:9s} PSNR {psnr(clean, out, data_range=1):5.2f} dB edge gradient {grad(out)[edges].mean():.3f}")
print(f"clean edge gradient {grad(clean)[edges].mean():.3f}")
# Noisy linear ramp: how erratic are the step-to-step differences? (1.0 = the true slope)
n = 200
ramp = np.tile(np.linspace(0.2, 0.8, n), (n, 1))
nr = ramp + np.random.default_rng(1).normal(0, 0.02, ramp.shape)
step = 0.6 / (n - 1)
for name, out in [("gaussian", gaussian_filter(nr, 2, truncate=3)), ("bilateral", bilateral(nr, 6, 2, 0.05))]:
d = np.diff(out[:, 20:-20], axis=1)
print(f"ramp {name:9s} std of step-to-step difference / true slope = {d.std() / step:.2f}")輸出
noisy PSNR 21.93 dB edge gradient 1.054
gaussian PSNR 25.72 dB edge gradient 0.346
bilateral PSNR 28.81 dB edge gradient 0.787
clean edge gradient 1.005
ramp gaussian std of step-to-step difference / true slope = 0.33
ramp bilateral std of step-to-step difference / true slope = 1.56兩個濾波器都使用 與相同的 13×13 支撐範圍,並在與導引濾波器範例相同的雜訊影像上(雜訊標準差 0.08)以相同的兩項指標比較。雙邊濾波器的輸出比高斯濾波器的輸出更接近乾淨影像 3.09 dB,並且保留了乾淨影像約 78% 的邊緣梯度(0.787 對 1.005),高斯濾波器只保留約 34%(0.346)。邊緣銳利度的定義同前:乾淨影像中最強的 10% 邊緣像素上,Sobel 梯度強度的平均。作為對照,上一則補充的導引濾波器(9×9 視窗、)在同一張影像上得到 27.03 dB 與 0.810,所以兩種保邊濾波器的表現在同一範圍,且數字取決於各自選用的參數。斜坡測試(雜訊標準差 0.02、)量化了平台與跳階的瑕疵:理想斜坡的比值是 0,而雙邊輸出的比值接近高斯輸出的五倍。
參考文獻
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 3. publisher page
- S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. ter Haar Romeny, J. B. Zimmerman and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Computer Vision, Graphics, and Image Processing, 1987. doi
- K. Zuiderveld, “Contrast limited adaptive histogram equalization,” in P. S. Heckbert (ed.), Graphics Gems IV, Morgan Kaufmann, 1994, pp. 474–485. entry
- A. M. Reza, “Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement,” Journal of VLSI Signal Processing, 2004. doi
- D. Coltuc, P. Bolon and J.-M. Chassery, “Exact histogram specification,” IEEE Transactions on Image Processing, 2006. doi
- C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. IEEE International Conference on Computer Vision (ICCV), 1998. pdf
- S. Paris, P. Kornprobst, J. Tumblin and F. Durand, “Bilateral filtering: Theory and applications,” Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1–73, 2009. doi
- K. He, J. Sun and X. Tang, “Guided image filtering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6):1397–1409, 2013 (first presented at ECCV 2010). doi
- P. Milanfar, “A tour of modern image filtering,” IEEE Signal Processing Magazine, vol. 30, no. 1, pp. 106–128, 2013. IEEE Xplore
- S. Gingichashvili and D. Lischinski, “Evaluation and comparison of edge-preserving filters,” arXiv:2012.13778, 2020. arXiv
- Y. Qi, Z. Yang, W. Sun et al., “A comprehensive overview of image enhancement techniques,” Archives of Computational Methods in Engineering, 2021. doi
- C. Li, C. Guo, L. Han, J. Jiang, M.-M. Cheng, J. Gu and C. C. Loy, “Low-light image and video enhancement using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (arXiv:2104.10729). arXiv
- C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong and R. Cong, “Zero-reference deep curve estimation for low-light image enhancement,” in Proc. IEEE/CVF CVPR, 2020. arXiv
- H. Zeng, J. Cai, L. Li, Z. Cao and L. Zhang, “Learning image-adaptive 3D lookup tables for high performance photo enhancement in real-time,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
- M. Gharbi, J. Chen, J. T. Barron, S. W. Hasinoff and F. Durand, “Deep bilateral learning for real-time image enhancement,” ACM Transactions on Graphics (SIGGRAPH), 2017. arXiv
- H. Wu, S. Zheng, J. Zhang and K. Huang, “Fast end-to-end trainable guided filter,” in Proc. IEEE/CVF CVPR, 2018. arXiv
- Y. Cai, H. Bian, J. Lin, H. Wang, R. Timofte and Y. Zhang, “Retinexformer: One-stage Retinex-based transformer for low-light image enhancement,” in Proc. IEEE/CVF ICCV, 2023. arXiv
- S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, vol. 81, no. 1, pp. 24–52, 2009 (first presented at ECCV 2006). doi