Guided Image Filtering

Kaiming He The Chinese University of Hong Kong

Jian Sun Microsoft Research Asia
Xiaoou Tang The Chinese University of Hong Kong
Introduction
• Edge-preserving filtering
– An important topic in computer vision
• Denoising, image smoothing/sharpening, texture decomposition, HDR
compression, image abstraction, optical flow estimation, image super-
resolution, feature smoothing…

– Existing methods
• Weighted Least Square [Lagendijk et al. 1988]
• Anisotropic diffusion [Perona and Malik 1990]
• Bilateral filter [Aurich and Weule 95], [Tomasi and Manduchi 98]
• Digital TV (Total Variation) filter [Chan et al. 2001]
Introduction
• Bilateral filter qi  W ( p) p
jN ( i )
ij j

spatial Gs(xi-xj)

input p bilateral output q

W=GsGr
range Gr(pi-pj)
Introduction
• Joint bilateral filter [Petschnigg et al. 2004] qi  W ( I ) p
jN ( i )
ij j

bilateral filter: I=p

spatial Gs(xi-xj)

input p bilateral output q

W=GsGr
range Gr(Ii-Ij)

E.g. p: noisy / chrominance channel

I: flash / luminance channel
guide I
Introduction
• Advantages of bilateral filtering
– Preserve edges in the smoothing process
– Simple and intuitive
– Non-iterative
Introduction
• Problems in bilateral filtering
– Complexity
• Brute-force: O(r2)
• Distributive histogram: O(logr) [Weiss 06]
• Bilateral grid: band-dependent [Paris and Durand 06], [Chen et al. 07] Approximate
(quantized)
• Integral histogram: O(1) [Porikli 08], [Yang et al. 09]
Introduction
• Problems in bilateral filtering
– Complexity Example: detail enhancement

– Gradient distortion
gradient
reversal

• Preserves edges,
but not gradients

gradient
reversal

input enhanced
Introduction
• Our target - to design a new filter
– Edge-preserving filtering
Advantages of bilateral filter
– Non-iterative
– O(1) time, fast and non-approximate
Overcome bilateral filter’s
– No gradient distortion problems
Guided filter
qi  pi  ni min  (aI i  b  pi ) 2  a 2
( a ,b )
i

ni - noise / texture
Linear regression
input p

output q
qi  aI i cov( I , p )
a
var( I )  
qi  aI i  b
b  p  aI
guide I
Bilateral/joint bilateral filter does
not have this linear model
 Definition
Guided filter
cov k ( I , p )
• Extend to the entire image ak 
vark ( I )  
– In all local windows ωk ,compute
the linear coefficients bk  pk  aI k
– Compute the average of akIi+bk in
1
all ωk that covers pixel qi
qi 

 (a I
k |i
k i  bk )
k

 ai I i  bi
qi
ω2
ω1
ω3
 Definition
Guided filter
cov k ( I , p )
• Parameters ak 
vark ( I )  
– Window radius r
bk  pk  aI k
– Regularization ε
1
qi 

 (a I
k |i
k i  bk )
k

 ai I i  bi
qi
ω2 2r
ω1
ω3
Guided filter: smoothing
a cascade of
mean filters

cov( I , p )
a var(I )   a0
var( I )   cov( I , p )   qi  a I i  b  p
b p
b  p  aI

input p output q

var(I )   r : determines
band-width
guide I (like σs in BF)
Guided filter: edge-preserving

qi  a I i  b qi  a I i  I i a  b
ε : degree of
cov( I , p ) edge-preserving
  a  (like σr in BF)
var(I )  

I i qi

guide I output q
 input &
Example – edge-preserving smoothing guide

guided
filter
(let I=p)

r=4, ε=0.12 r=4, ε=0.22 r=4, ε=0.42

bilateral
filter

σs=4, σr=0.1 σs=4, σr=0.2 σs=4, σr=0.4

• Our target - to design a new filter
– Edge-preserving filtering
Advantages of bilateral filter
– Non-iterative
– O(1) time, fast and non-approximate
Overcome bilateral filter’s
– No gradient distortion problems
 Definition
Complexity
cov k ( I , p )
• mean, var, cov in all local windows ak 
vark ( I )  
• Integral images [Franklin 1984] bk  pk  aI k
– O(1) time – independent of r
– Non-approximate qi  ai I i  bi

O(1) bilateral O(1) bilateral O(1) guided

(32-bin, 40ms/M) (64-bin, 80ms/M) (exact, 80ms/M)
[Porikli 08]
 Gradient Preserving
bilateral filter guided filter

input filtered
q  a I
large
fluctuation
detail
(input - filtered)

enhanced
(detail * 5 + input) gradient
reversal
 gradient
reversal
Example – detail enhancement

bilateral filter guided filter

input (I=p) bilateral filter guided filter

σs=16, σr=0.1 r=16, ε=0.12
 gradient
reversal
Example – detail enhancement

bilateral filter guided filter

input (I=p) bilateral filter guided filter

σs=16, σr=0.1 r=16, ε=0.12
Example – HDR compression

gradient keep
reversal anti-aliased

bilateral filter guided filter

input HDR

bilateral filter guided filter

σs=15, σr=0.12 r=15, ε=0.122
Example – flash/no-flash denoising

gradient
reversal

input p joint bilateral filter

(no-flash) σs=8, σr=0.02

joint bilateral guided filter

guide I guided filter

(flash) r=8, ε=0.022
Beyond smoothing
• Applications: feathering/matting, haze removal

very small ε
guide I

preserve most
gradients output q

q  a I
input p
Example – feathering

guide I
(size 3000x2000)
Example – feathering

filter input p (binary segmentation)

Example – feathering

filter output q (alpha matte)

Example – feathering

filter output q matting Laplacian

guide I filter input p [Levin et al. 06]
0.3s
image size 6M 2 min
Example – haze removal

filter input p
guide I (dark channel prior filter output q
[He et al. 09])
Example – haze removal

guided filter global optimization

guide I (<0.1s, 600x400p) (10s)
Limitation
• “What is an edge” – inherently ambiguous, context-dependent
weaker halo halo
edge

stronger
texture

Input Bilateral filter Guided filter

σs=16, σr=0.4 r=16, ε=0.42
Conclusion
• We go from “BF” to “GF”
– Edge-preserving filtering
– Non-iterative
– O(1) time, fast, accurate
– Gradient preserving
– More generic than “smoothing”

Thank you!