Full search page →

The guided filter: smoothing that follows a guide image

Optional · supplementary note 4 of 5 for IP 03 · Intensity Transformations and Spatial Filtering. Nothing in the main text depends on it; citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

The box and Gaussian filters smooth across edges because they only look at pixel positions. The guided filter smooths an image but lets a second image, the guide, decide where the edges are, so the output keeps the guide’s edges while the noise is averaged away. In the self-guided case the guide is the input itself.

He, Sun and Tang [8] assume that, inside every small window wkw_k, the output is a linear function of the guide:

qi=akIi+bk,i∈wk.q_i = a_k I_i + b_k, \qquad i \in w_k .

Here II is the guide image, qq is the output, and wkw_k is a square window of radius rr (so (2r+1)2(2r+1)^2 pixels) centered at pixel kk. Because ∇q=ak∇I\nabla q = a_k \nabla I inside the window, qq can only have an edge where II has one. The coefficients ak,bka_k, b_k are chosen so that qq stays close to the input pp, with a penalty ϵak2\epsilon a_k^2 on large slopes:

min⁡ak, bk∑i∈wk((akIi+bk−pi)2+ϵ ak2)  ⇒  ak=(Ip)‾k−μkpˉkσk2+ϵ,bk=pˉk−akμk.\min_{a_k,\, b_k} \sum_{i \in w_k} \Big( (a_k I_i + b_k - p_i)^2 + \epsilon\, a_k^2 \Big) \;\Rightarrow\; a_k = \frac{\overline{(Ip)}_k - \mu_k \bar p_k}{\sigma_k^2 + \epsilon}, \qquad b_k = \bar p_k - a_k \mu_k .

Here μk\mu_k and σk2\sigma_k^2 are the mean and variance of II in wkw_k, pˉk\bar p_k is the mean of pp in wkw_k, (Ip)‾k\overline{(Ip)}_k is the mean of the product IipiI_i p_i in wkw_k, and ϵ>0\epsilon > 0 is the regularizer. A pixel ii lies in (2r+1)2(2r+1)^2 different windows, each giving a slightly different (ak,bk)(a_k, b_k), so the final output averages them:

qi=aˉiIi+bˉi,aˉi=1∣w∣∑k: i∈wkak,bˉi=1∣w∣∑k: i∈wkbk,q_i = \bar a_i I_i + \bar b_i, \qquad \bar a_i = \frac{1}{|w|} \sum_{k:\, i \in w_k} a_k, \quad \bar b_i = \frac{1}{|w|} \sum_{k:\, i \in w_k} b_k ,

where ∣w∣=(2r+1)2|w| = (2r+1)^2 and the sums run over the windows that contain pixel ii. That last average is itself a box filter.

What ϵ\epsilon does. In the self-guided case (p=Ip = I), ak=σk2/(σk2+ϵ)a_k = \sigma_k^2 / (\sigma_k^2 + \epsilon). In a flat patch, σk2≪ϵ\sigma_k^2 \ll \epsilon, so ak≈0a_k \approx 0 and bk≈pˉkb_k \approx \bar p_k: the output is the plain box mean. Across an edge, σk2≫ϵ\sigma_k^2 \gg \epsilon, so ak≈1a_k \approx 1, bk≈0b_k \approx 0, and the pixel is passed through. A larger ϵ\epsilon therefore smooths more, and a smaller one preserves more edges. Since ϵ\epsilon is compared with a variance, its scale is the squared intensity range: with intensities in [0,1][0, 1], ϵ=0.01\epsilon = 0.01 treats variations of about 0.10.1 as edges.

Cost. Every quantity above (μk\mu_k, pˉk\bar p_k, (Ip)‾k\overline{(Ip)}_k, I2‾k\overline{I^2}_k, aˉi\bar a_i, bˉi\bar b_i) is a box mean, and a box mean needs constant work per pixel with running sums, so the whole filter is O(N)O(N) for NN pixels, independent of rr. There is no per-pixel search over intensities, which is what makes the bilateral filter expensive.

Code · Python · 24 lines
import numpy as np
from scipy.ndimage import uniform_filter, sobel
from skimage import data, img_as_float
from skimage.metrics import peak_signal_noise_ratio as psnr

def guided_filter(I, p, r, eps):
    box = lambda x: uniform_filter(x, size=2 * r + 1, mode="reflect")
    mu, pbar = box(I), box(p)
    var = box(I * I) - mu**2
    a = (box(I * p) - mu * pbar) / (var + eps)
    b = pbar - a * mu
    return box(a) * I + box(b)

clean = img_as_float(data.camera())
noisy = clean + np.random.default_rng(0).normal(0, 0.08, clean.shape)
r, eps = 4, 0.1**2
box_out = uniform_filter(noisy, size=2 * r + 1, mode="reflect")
gf_out = guided_filter(noisy, noisy, r, eps)          # self-guided

grad = lambda x: np.hypot(sobel(x, 0), sobel(x, 1))
edges = grad(clean) > np.percentile(grad(clean), 90)  # strongest 10% of edge pixels
for name, out in [("noisy", noisy), ("box", box_out), ("guided", gf_out)]:
    print(f"{name:7s} PSNR {psnr(clean, out, data_range=1):5.2f} dB   edge gradient {grad(out)[edges].mean():.3f}")
print(f"clean   edge gradient {grad(clean)[edges].mean():.3f}")

Output

noisy   PSNR 21.93 dB   edge gradient 1.054
box     PSNR 23.88 dB   edge gradient 0.221
guided  PSNR 27.03 dB   edge gradient 0.810
clean   edge gradient 1.005

Both filters use the same window (9×9) on the same noisy image (noise standard deviation 0.08). The guided filter’s output is 3.15 dB closer to the clean image than the box filter’s. The second column measures edge sharpness as the mean Sobel gradient magnitude over the 10% of pixels with the strongest edges in the clean image (Sobel units, unnormalized): the box filter keeps about a fifth of the clean value (0.221 vs 1.005), the guided filter about four fifths (0.810). The noisy value is slightly above the clean one only because noise adds gradient. As a check, the closed-form code agrees with a direct window-by-window loop over ak,bka_k, b_k on a crop, away from the border, to about 10−1510^{-15}.

References

  1. R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 3. publisher page
  2. S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz, T. Greer, B. ter Haar Romeny, J. B. Zimmerman and K. Zuiderveld, “Adaptive histogram equalization and its variations,” Computer Vision, Graphics, and Image Processing, 1987. doi
  3. K. Zuiderveld, “Contrast limited adaptive histogram equalization,” in P. S. Heckbert (ed.), Graphics Gems IV, Morgan Kaufmann, 1994, pp. 474–485. entry
  4. A. M. Reza, “Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement,” Journal of VLSI Signal Processing, 2004. doi
  5. D. Coltuc, P. Bolon and J.-M. Chassery, “Exact histogram specification,” IEEE Transactions on Image Processing, 2006. doi
  6. C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Proc. IEEE International Conference on Computer Vision (ICCV), 1998. pdf
  7. S. Paris, P. Kornprobst, J. Tumblin and F. Durand, “Bilateral filtering: Theory and applications,” Foundations and Trends in Computer Graphics and Vision, vol. 4, no. 1, pp. 1–73, 2009. doi
  8. K. He, J. Sun and X. Tang, “Guided image filtering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(6):1397–1409, 2013 (first presented at ECCV 2010). doi
  9. P. Milanfar, “A tour of modern image filtering,” IEEE Signal Processing Magazine, vol. 30, no. 1, pp. 106–128, 2013. IEEE Xplore
  10. S. Gingichashvili and D. Lischinski, “Evaluation and comparison of edge-preserving filters,” arXiv:2012.13778, 2020. arXiv
  11. Y. Qi, Z. Yang, W. Sun et al., “A comprehensive overview of image enhancement techniques,” Archives of Computational Methods in Engineering, 2021. doi
  12. C. Li, C. Guo, L. Han, J. Jiang, M.-M. Cheng, J. Gu and C. C. Loy, “Low-light image and video enhancement using deep learning: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021 (arXiv:2104.10729). arXiv
  13. C. Guo, C. Li, J. Guo, C. C. Loy, J. Hou, S. Kwong and R. Cong, “Zero-reference deep curve estimation for low-light image enhancement,” in Proc. IEEE/CVF CVPR, 2020. arXiv
  14. H. Zeng, J. Cai, L. Li, Z. Cao and L. Zhang, “Learning image-adaptive 3D lookup tables for high performance photo enhancement in real-time,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. arXiv
  15. M. Gharbi, J. Chen, J. T. Barron, S. W. Hasinoff and F. Durand, “Deep bilateral learning for real-time image enhancement,” ACM Transactions on Graphics (SIGGRAPH), 2017. arXiv
  16. H. Wu, S. Zheng, J. Zhang and K. Huang, “Fast end-to-end trainable guided filter,” in Proc. IEEE/CVF CVPR, 2018. arXiv
  17. Y. Cai, H. Bian, J. Lin, H. Wang, R. Timofte and Y. Zhang, “Retinexformer: One-stage Retinex-based transformer for low-light image enhancement,” in Proc. IEEE/CVF ICCV, 2023. arXiv
  18. S. Paris and F. Durand, “A fast approximation of the bilateral filter using a signal processing approach,” International Journal of Computer Vision, vol. 81, no. 1, pp. 24–52, 2009 (first presented at ECCV 2006). doi