The bilateral filter: averaging only with pixels that look alike
Optional · supplementary note 5 of 5 for IP 03 · Intensity Transformations and Spatial Filtering. Nothing in the main text depends on it; citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
A Gaussian filter asks one question about a neighbor: how close is it? The bilateral filter asks two: how close is it, and does it look like me? A neighbor on the other side of an edge looks very different, so it is nearly ignored, and the edge is not blurred.
Tomasi and Manduchi [6] replace each pixel by a weighted mean of the pixels in a window of radius :
Here and are the input intensities at pixels and , is the distance between the two pixel positions, is the output, and is the sum of the weights, which makes them add up to one. The spatial scale (in pixels) says how far a neighbor may be and still count; with the range factor is 1 and the filter is exactly a Gaussian filter. The range scale (in intensity units) says how different a neighbor may look. It plays the role the guided filter’s plays: differences well above are treated as edges.
Why edges survive. Take a pixel with intensity 0.2 right beside a step up to 0.7. Its neighbors across the step differ by 0.5, and with their range weight is . The average is taken almost entirely over the pixels on its own side, so the step stays sharp while the noise on each side is averaged out. Choose a little above the noise level and below the contrast of the edges you want to keep. The same rule explains the limits: an isolated noisy pixel (impulse noise) differs from all of its neighbors, so it mostly keeps itself, and fine texture with contrast above is preserved instead of smoothed.
Cost. The weights depend on the image values, so there is no fixed kernel to precompute and no separable or FFT shortcut. Evaluating the sum directly visits neighbors per pixel, i.e. for pixels, against for a separable Gaussian and for the guided filter. Paris and Durand [18] showed that the filter can be written as a Gaussian convolution in a three-dimensional space (two position axes plus intensity) followed by a division, and that this convolution can be computed on a coarsely downsampled grid in space and intensity; the survey [7] covers this and other accelerations.
Base and detail. Filtering an image with the bilateral filter gives a smooth base layer that keeps the strong edges, and subtracting it leaves a detail layer. Tone mapping and detail enhancement edit the two layers separately and add them back [7], which is why an edge-preserving filter is needed: with a Gaussian base the edit would leave halos around edges.
Artifacts. The output is a weighted mean of input pixels with weights that change from pixel to pixel, so nothing forces it to stay locally linear in the input. On a noisy, slowly varying region it can break a smooth ramp into plateaus and jumps (measured below), and in detail enhancement it can produce gradient reversal near strong edges [8]. The guided filter’s window model gives inside the window, and in the self-guided case lies between 0 and 1, so within a window the output gradient never points against the guide’s gradient. That is a property of the model, and it is consistent with the report in [8] that the guided filter behaves better near strong edges; the plateau effect on noisy ramps is a separate matter that this note does not test for the guided filter.
Code · Python · 37 lines
import numpy as np
from scipy.ndimage import gaussian_filter, sobel
from skimage import data, img_as_float
from skimage.metrics import peak_signal_noise_ratio as psnr
def bilateral(img, r, sigma_s, sigma_r):
H, W = img.shape
pad = np.pad(img, r, mode="reflect")
num, den = np.zeros_like(img), np.zeros_like(img)
for dy in range(-r, r + 1): # (2r+1)^2 shifted copies of the image
for dx in range(-r, r + 1):
q = pad[r + dy : r + dy + H, r + dx : r + dx + W] # neighbor value I_q
w = np.exp(-(dx * dx + dy * dy) / (2 * sigma_s**2)) \
* np.exp(-((q - img) ** 2) / (2 * sigma_r**2))
num += w * q
den += w # den is W_p
return num / den
clean = img_as_float(data.camera())
noisy = clean + np.random.default_rng(0).normal(0, 0.08, clean.shape)
grad = lambda x: np.hypot(sobel(x, 0), sobel(x, 1))
edges = grad(clean) > np.percentile(grad(clean), 90) # strongest 10% of edge pixels
gauss = gaussian_filter(noisy, 2, truncate=3, mode="reflect") # same sigma_s, same 13x13 support
bil = bilateral(noisy, r=6, sigma_s=2, sigma_r=0.15)
for name, out in [("noisy", noisy), ("gaussian", gauss), ("bilateral", bil)]:
print(f"{name:9s} PSNR {psnr(clean, out, data_range=1):5.2f} dB edge gradient {grad(out)[edges].mean():.3f}")
print(f"clean edge gradient {grad(clean)[edges].mean():.3f}")
# Noisy linear ramp: how erratic are the step-to-step differences? (1.0 = the true slope)
n = 200
ramp = np.tile(np.linspace(0.2, 0.8, n), (n, 1))
nr = ramp + np.random.default_rng(1).normal(0, 0.02, ramp.shape)
step = 0.6 / (n - 1)
for name, out in [("gaussian", gaussian_filter(nr, 2, truncate=3)), ("bilateral", bilateral(nr, 6, 2, 0.05))]:
d = np.diff(out[:, 20:-20], axis=1)
print(f"ramp {name:9s} std of step-to-step difference / true slope = {d.std() / step:.2f}")Output
noisy PSNR 21.93 dB edge gradient 1.054
gaussian PSNR 25.72 dB edge gradient 0.346
bilateral PSNR 28.81 dB edge gradient 0.787
clean edge gradient 1.005
ramp gaussian std of step-to-step difference / true slope = 0.33
ramp bilateral std of step-to-step difference / true slope = 1.56Both filters have and the same 13×13 support, and run on the same noisy image as the guided-filter example (noise standard deviation 0.08), with the same two measures. The bilateral output is 3.09 dB closer to the clean image than the Gaussian one, and it keeps about 78% of the clean edge gradient (0.787 against 1.005) where the Gaussian keeps about 34% (0.346). Edge sharpness is the mean Sobel gradient over the strongest 10% of clean-image edge pixels, as before. For comparison, the guided filter of the previous note (9×9 window, ) reached 27.03 dB and 0.810 on this image, so the two edge-preserving filters are in the same range, and the numbers depend on the parameters chosen for each. The ramp test (noise standard deviation 0.02, ) quantifies the plateau-and-jump artifact: for a perfect ramp the ratio would be 0, and the bilateral output’s ratio is nearly five times the Gaussian’s.
References
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 3. publisher page
- S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. ter Haar Romeny, J. B. Zimmerman and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Computer Vision, Graphics, and Image Processing, 1987. doi
- K. Zuiderveld, “Contrast limited adaptive histogram equalization,” in P. S. Heckbert (ed.), Graphics Gems IV, Morgan Kaufmann, 1994, pp. 474–485. entry
- A. M. Reza, “Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement,” Journal of VLSI Signal Processing, 2004. doi
- D. Coltuc, P. Bolon and J.-M. Chassery, “Exact histogram specification,” IEEE Transactions on Image Processing, 2006. doi
- C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. IEEE International Conference on Computer Vision (ICCV), 1998. pdf
- S. Paris, P. Kornprobst, J. Tumblin and F. Durand, “Bilateral filtering: Theory and applications,” Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1–73, 2009. doi
- K. He, J. Sun and X. Tang, “Guided image filtering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6):1397–1409, 2013 (first presented at ECCV 2010). doi
- P. Milanfar, “A tour of modern image filtering,” IEEE Signal Processing Magazine, vol. 30, no. 1, pp. 106–128, 2013. IEEE Xplore
- S. Gingichashvili and D. Lischinski, “Evaluation and comparison of edge-preserving filters,” arXiv:2012.13778, 2020. arXiv
- Y. Qi, Z. Yang, W. Sun et al., “A comprehensive overview of image enhancement techniques,” Archives of Computational Methods in Engineering, 2021. doi
- C. Li, C. Guo, L. Han, J. Jiang, M.-M. Cheng, J. Gu and C. C. Loy, “Low-light image and video enhancement using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (arXiv:2104.10729). arXiv
- C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong and R. Cong, “Zero-reference deep curve estimation for low-light image enhancement,” in Proc. IEEE/CVF CVPR, 2020. arXiv
- H. Zeng, J. Cai, L. Li, Z. Cao and L. Zhang, “Learning image-adaptive 3D lookup tables for high performance photo enhancement in real-time,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
- M. Gharbi, J. Chen, J. T. Barron, S. W. Hasinoff and F. Durand, “Deep bilateral learning for real-time image enhancement,” ACM Transactions on Graphics (SIGGRAPH), 2017. arXiv
- H. Wu, S. Zheng, J. Zhang and K. Huang, “Fast end-to-end trainable guided filter,” in Proc. IEEE/CVF CVPR, 2018. arXiv
- Y. Cai, H. Bian, J. Lin, H. Wang, R. Timofte and Y. Zhang, “Retinexformer: One-stage Retinex-based transformer for low-light image enhancement,” in Proc. IEEE/CVF ICCV, 2023. arXiv
- S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, vol. 81, no. 1, pp. 24–52, 2009 (first presented at ECCV 2006). doi