1

of
31
Image Restoration
– What is image restoration?
– Noise and images
– Noise models
– Noise removal using spatial domain filtering
– Periodic noise
– Noise removal using frequency domain
filtering
2
of
31
What is Image Restoration?
Image restoration attempts to restore
images that have been degraded
– Identify the degradation process and attempt
to reverse it
– Similar to image enhancement, but more
objective
3
of
31
Noise and Images
The sources of noise in digital
images arise during image
acquisition (digitization) and
transmission
– Imaging sensors can be
affected by ambient
conditions
– Interference can be added
to an image during
transmission
4
of
31
Degradation Model

f(x,y) h(x,y) g(x,y)



n(x,y)

Degradation Model: g = h*f + n

5
of Restoration Model
31

Degradation Restoration
f(x,y) f(x,y)
Model Filter

Unconstrained Constrained
• Inverse Filter • Wiener Filter
• Pseudo-inverse Filter
6
of
31
Noise Model
We can consider a noisy image to be
modelled as follows:
g ( x , y )  f ( x, y )   ( x, y )
where f(x, y) is the original image pixel,
η(x, y) is the noise term and g(x, y) is the
resulting noisy pixel
If we can estimate the model the noise in an
image is based on this will help us to figure
out how to restore the image
7
of
31
Noise Models
There are many different Gaussian Rayleigh
models for the image
noise term η(x, y):
– Gaussian
• Most common model Erlang Exponential

– Rayleigh
– Erlang
– Exponential Uniform
– Uniform Impulse

– Impulse
• Salt and pepper noise
8
of
31
Noise Example
The test pattern to the right is
ideal for demonstrating the
addition of noise
The following slides will show
the result of adding noise Image
based on various models to
this image
Histogram to go here

Histogram
9
of
31
Noise Example (cont…)

Gaussian Rayleigh Erlang

10
of
31
Noise Example (cont…)

Histogram to go here

Exponential Uniform Impulse

11
of
31
Filtering to Remove Noise
We can use spatial filters of different kinds
to remove different kinds of noise
The arithmetic mean filter is a very simple
one and is calculated as follows:
ˆf ( x, y )  1
 g ( s, t )
mn ( s ,t )S xy
1/ 1/ 1/ This is implemented as the
9 9 9
1/ 1/ 1/
simple smoothing filter
9 9 9
Blurs the image to remove
1/ 1/ 1/
9 9 9 noise
12
of
31
Other Means
There are different kinds of mean filters all of
which exhibit slightly different behaviour:
– Geometric Mean
– Harmonic Mean
– Contraharmonic Mean
13
of
31
Other Means (cont…)
There are other variants on the mean which
can give different performance
Geometric Mean:
1
  mn
fˆ ( x, y )    g ( s, t )
( s ,t )S xy 
Achieves similar smoothing to the arithmetic
mean, but tends to lose less image detail
14
of
31
Other Means (cont…)
Harmonic Mean:
mn
fˆ ( x, y ) 
1

( s ,t )S xy g ( s, t )
Works well for salt noise, but fails for pepper
noise
Also does well for other kinds of noise such
as Gaussian noise
15
of
31
Other Means (cont…)
Contraharmonic Mean:
 g ( s, t )
( s ,t )S xy
Q 1

fˆ ( x, y ) 
 g ( s, t )
( s ,t )S xy
Q

Q is the order of the filter and adjusting its

value changes the filter’s behaviour
Positive values of Q eliminate pepper noise
Negative values of Q eliminate salt noise
16
of
31
Noise Removal Examples

Image
Original Corrupted
Image By Gaussian
Noise

After A 3*3 After A 3*3

Arithmetic Geometric
Mean Filter Mean Filter
17
of
31
Noise Removal Examples (cont…)

Image
Corrupted
By Pepper
Noise

Result of
Filtering Above
With 3*3
Contraharmonic
Q=1.5
18
of
31
Noise Removal Examples (cont…)

Image
Corrupted
By Salt
Noise

Result of
Filtering Above
With 3*3
Contraharmonic
Q=-1.5
19
of
31
Contraharmonic Filter: Here Be Dragons

Choosing the wrong value for Q when using

the contraharmonic filter can have drastic
results
20
of
31
Order Statistics Filters
Spatial filters that are based on ordering the
pixel values that make up the
neighbourhood operated on by the filter
Useful spatial filters include
– Median filter
– Max and min filter
– Midpoint filter
– Alpha trimmed mean filter
21
of
31
Median Filter
Median Filter:
fˆ ( x, y )  median{g ( s, t )}
( s ,t )S xy

Excellent at noise removal, without the

smoothing effects that can occur with other
smoothing filters
Particularly good when salt and pepper
noise is present
22
of
31
Max and Min Filter
Max Filter:
fˆ ( x, y )  max {g ( s, t )}
( s ,t )S xy

Min Filter:
fˆ ( x, y )  min {g ( s, t )}
( s ,t )S xy

Max filter is good for pepper noise and min

is good for salt noise
23
of
31
Midpoint Filter
Midpoint Filter:
1
ˆf ( x, y )   max {g ( s, t )}  min {g ( s, t )}
2 ( s ,t )S xy ( s ,t )S xy 
Good for random Gaussian and uniform
noise
24
of
31
Alpha-Trimmed Mean Filter
Alpha-Trimmed Mean Filter:
1
fˆ ( x, y ) 
mn  d
 g ( s, t )
( s ,t )S xy
r

We can delete the d/2 lowest and d/2 highest

grey levels
So gr(s, t) represents the remaining mn – d
pixels
25
of
31
Noise Removal Examples

Image Result of 1
Corrupted Pass With A
By Salt And 3*3 Median
Pepper Noise Filter

Result of 2 Result of 3
Passes With Passes With
A 3*3 Median A 3*3 Median
Filter Filter
26
of
31
Noise Removal Examples (cont…)

Image Image
Corrupted Corrupted
By Pepper By Salt
Noise Noise

Result Of Result Of
Filtering Filtering
Above Above
With A 3*3 With A 3*3
Max Filter Min Filter
27
of
31
Noise Removal Examples (cont…)
Image Image Further
Corrupted Corrupted
By Uniform By Salt and
Noise Pepper Noise

Filtered By Filtered By
5*5 Arithmetic 5*5 Geometric
Mean Filter Mean Filter

Filtered By Filtered By
5*5 Median 5*5 Alpha-Trimmed
Filter Mean Filter
28
of
31
Periodic Noise
Typically arises due to
electrical or electromagnetic
interference
Gives rise to regular noise
patterns in an image
Frequency domain
techniques in the Fourier
domain are most effective
at removing periodic noise
29
of
31
Band Reject Filters
Removing periodic noise form an image
involves removing a particular range of
frequencies from that image
Band reject filters can be used for this purpose
An ideal band reject filter is given as follows:
 W
1 if D(u, v)  D0  2
 W W
H (u, v)  0 if D0   D(u, v)  D0 
 2 2
1 if D(u, v)  D0  W
 2
30
of
31
Band Reject Filters (cont…)
The ideal band reject filter is shown below,
along with Butterworth and Gaussian
versions of the filter

Ideal Band Butterworth Gaussian

Reject Filter Band Reject Band Reject
Filter (of order 1) Filter
31
of
31
Band Reject Filter Example
Image corrupted by Fourier spectrum of
sinusoidal noise corrupted image

Butterworth band Filtered image

reject filter
32
of Periodic Noise Reduction by Frequency
31
Domain Filtering

Bandreject Filters
HBP(u,v)=1-HBR(u,v)
Notch Filters: HNP(u,v)=1-HNR(u,v)
•Most useful of the selective filters.
•Rejects or passes frequencies in a predefined
neighbourhood about the center of the frequency
rectangle
•It helps to isolate the noise pattern.
33
of Zero-Phase-Shift filters must be symmetric about
31
the origin. Notch with center (u0,v0) and (-u0,-v0).

0 if D1u, v  D0 or D2 u, v  D0
Hu, v  
Notch Filters 1 otherwise
1
Ideal :
 M
2
  N 
2 2
D1u, v   u   u 0    v   v0  
 2   2  
1
 M
2
  N 
2 2
D1u, v   u   u 0    v   v0  
 2   2  

The center of the frequency rectangle has been shifted to the point
(M/2,N/2)
34
of
31 Periodic Noise Reduction by Frequency
Domain Filtering Notch Filters

1
3-2) Butterworth of order n : Hu, v 
n
 D0 2 
1  
 D1u, v D2 u, v 

1  D1 u,v D2 u,v  
3-3) Gaussian :   2 
2 
Hu, v  1  e 
D0
35
of
31
36
of
31
37
of
31 Periodic Noise Reduction by Frequency
Domain Filtering
4) Optimum Notch Filters

f x , y   g x , y   w x , y   x , y 
 x , y   F H u , v G u , v 
1

g x , y  x , y   g x , y  x , y 
w x , y  
 x , y    x , y 
2 2

To obtain w(x,y) the goal is to minimize the variance in the

neighborhood of x,y in the image.
38
of
31
39
of
31
40
of
31
41
of
31
42
of
31
Adaptive Filters
The filters discussed so far are applied to an
entire image without any regard for how
image characteristics vary from one point to
another
The behaviour of adaptive filters changes
depending on the characteristics of the
image inside the filter region
We will take a look at the adaptive median
filter
43
of
31
Adaptive Median Filtering
The median filter performs relatively well on
impulse noise as long as the spatial density
of the impulse noise is not large
The adaptive median filter can handle much
more spatially dense impulse noise, and
also performs some smoothing for non-
impulse noise
The key insight in the adaptive median filter
is that the filter size changes depending on
the characteristics of the image
44
of
31
Adaptive Median Filtering (cont…)
Remember that filtering looks at each
original pixel image in turn and generates a
new filtered pixel
First examine the following notation:
– zmin = minimum grey level in Sxy
– zmax = maximum grey level in Sxy
– zmed = median of grey levels in Sxy
– zxy = grey level at coordinates (x, y)
– Smax =maximum allowed size of Sxy
45
of
31
Adaptive Median Filtering (cont…)
Level A: A1 = zmed – zmin
A2 = zmed – zmax
If A1 > 0 and A2 < 0, Go to level B
Else increase the window size
If window size ≤ repeat Smax level A
Else output zmed
Level B: B1 = zxy – zmin
B2 = zxy – zmax
If B1 > 0 and B2 < 0, output zxy
Else output zmed
46
of
31
Adaptive Median Filtering (cont…)
The key to understanding the algorithm is to
remember that the adaptive median filter
has three purposes:
– Remove impulse noise
– Provide smoothing of other noise
– Reduce distortion
47
of
31
Adaptive Filtering Example

Image corrupted by salt Result of filtering with a 7 Result of adaptive

and pepper noise with * 7 median filter median filtering with i = 7
probabilities Pa = Pb=0.25
48
of Linear, Position-Invariant Degradations
31

gx, y   h x, y   f x, y   x, y 

G u, v   Hu, v  Fu, v   Nu, v 

Estimation of the Degradation Function

1) Estimation by Image Observation
G s u , v 
H s u, v  
F̂s u, v 

Assuming that the effect of noise is negligible.

49
of
31
Linear, Position-Invariant Degradations

gx, y   h x, y   f x, y   x, y 

G u, v   Hu, v  Fu, v   Nu, v 
Estimation the Degradation Function
2) Estimation by Experimentation
G u , v 
Hu, v  
A

A : impulse Fourier transform which is constant.

50
of
31
51
of
31 Linear, Position-Invariant Degradations

Estimation the Degradation Function

3) Estimation by modeling
Turbulence model :
Hu, v   e  2
K u  v 2

5
6

Mathematical model :

T  jua vb
Hu, v  Sinua  vb e
ua  vb
52
of
31
53
of
31
54
of
31
Degradation Model

• In noise-free cases, a blurred image can

be modeled as
g  f h
h : linear space - invariant blur function
f : original image
n In the DFT domain, G(u,v) = F(u,v) H(u,v)
55
of
31
Inverse Filtering

• Assume h is known (low-pass filter)

n Inverse filter H’(u,v) = 1 / H(u,v)

n
F (u, v)  G(u, v) H (u, v)
56
of
31
Lost Information
57
of
31 How to get F^(u,v) from degraded image G(u,v)
1) Inverse Filtering

Gu, v Nu, v
F̂u, v   Fu, v 
Hu, v Hu, v

We should know N(u,v) to use this method.

We should use this method near origin
because H(u,v) is near zero in other areas.
58
of
31
59
of
31
Wiener filter

• It removes the additive noise and inverts the

blurring simultaneously.
• The Wiener filtering is optimal in terms of the
mean square error.
60
of How to get F^(u,v) from degraded image G(u,v)
31
2) Wiener (Minimum mean square Error)
Filtering

2 
 
e  E  f  f̂   0

2

 
H u , v 
2
 1 
F̂ u , v      G u , v 
 H u , v  H u , v  2  S  u , v  
 S f u , v  

H(u,v) : degradation function

Sn(u,v)=|N(u,v)|2 power spectrum of the noise
Sf(u,v)=|F(u,v)|2 power spectrum of the un degraded image
61
of
31
Wiener filter

Where Sf (u,v), Sη(u,v) are respectively power spectra of

the original image and the additive noise, and H(u,v) is the
blurring filter.
Wiener filter has two separate parts, an inverse filtering part
and a noise smoothing part.
 It performs the deconvolution by inverse filtering (highpass
filtering) and removes the noise with a compression operation
(lowpass filtering).
62
of
31
Power Spectrum

• The most common way of generating a power

spectrum is by using a Fourier transform and
taking the square of the magnitude of the
complex coefficients.
• Other techniques such as the maximum entropy
method can also be used to estimate the power
spectrum.
63
of Implementation
31

• To implement the Wiener filter in practice we have to estimate

the power spectra of the original image and the additive noise.
• For white additive noise the power spectrum is equal to the
variance of the noise.
• To estimate the power spectrum of the original image many
methods can be used. A direct estimate is the estimate of the
power spectrum computed from the observation:
64
of
31

If Sf(u,v) is not known :

2


e  E  f  f̂   0

2

 H u , v  
2
1
F̂ u , v      G u , v 
 H u , v  H u , v  2  K 
 
65
of
31
66
of
31
67
of
31 How to get F^(u,v) from degraded image
G(u,v) :

3) Constrained Least Squares Filtering

 H u , v 
* 
F̂ u , v     G u , v 
 H u , v  2   P u , v  2 
 
 0 1 0 

p x , y     1 4  1
 0 1 0 
68
of
31
69
of
31
70
of
31 Maximum-Likelihood (ML)
Estimation
• h is unknown
n Assume parametric models for the blur
function, original image, and/or noise
n Parametric set  is estimated by
 ml  arg{max p(y |  )}

n Solution is difficult
71
of
31 Expectation-Maximization (EM)
Algorithm
n Find complete set : for z  , f(z)=y
n Choose an initial guess of 
• Expectation-step
g ( |  )  E [p(z |  ) | y, ]
k k

n Maximization-step
 k 1
 arg max g ( |  )
k

72
of
31
Conclusions

• Noise-free case: inverse filtering

n Noisy case: Weiner filter
n Blind case: Maximum-Likelihood
approach using the Expectation-
Maximization algorithm