3 Contents

In this lecture we will look at image

restoration techniques used for noise
removal
– What is image restoration?
– Noise and images
– Noise models
– Noise removal using spatial domain filtering
– Noise removal using frequency domain
filtering
4 What is Image Restoration?
Image restoration attempts to
restore images that have been
degraded
– Identify the degradation process and attempt
to reverse it
– Similar to image enhancement, but more
objective
5 Noise and Images
The sources of noise in
digital images arise during
image acquisition
(digitization) and
transmission
– Imaging sensors can be
affected by ambient
conditions
– Interference can be added
to an image during
transmission
6 Noise Model
We can consider a noisy image to be
modelled as follows:
g(x, y) = f (x, y) +η(x, y)
where f (x, y) is the original image pixel,
η(x, y) is the noise term and g(x, y) is the
resulting noisy pixel
If we can estimate the noise model we
can figure out how to restore the image
 7 Noise Models (cont...)
There are many different
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Gaussian Rayleigh
models for the image
noise term η(x, y):
– Gaussian
• Most common model Erlang Exponential
– Rayleigh
– Erlang (Gamma)
– Exponential
Uniform
– Uniform Impuls
– Impulse
• Salt and pepper noise
 8 Noise Example
• The test pattern to the right is
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

ideal for demonstrating the

addition of noise
• The following slides will show
the result of adding noise Image
based on various models to
this image
Histogram to go
here

Histogram
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

9
Noise Example (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

10
Noise Example (cont…)
11
Restoration in the presence of noise
only
• We can use spatial filters of different kinds
to remove different kinds of noise
• The arithmetic mean filter is a very simple
one and is calculated as follows:
ˆ 1
f (x, y) mn
=
∑
g(s, t)
x
( s,t ) y

1 1 1
This is implemented
∈S as the
/9 /9 /9 simple smoothing filter
1 1 1
/9 /9 /9 It blurs the image.
1 1 1
/9 /9 /9
12
Restoration in the presence of noise
only (cont.)
• There are different kinds of mean filters all
of which exhibit slightly different
behaviour:
– Geometric Mean
– Harmonic Mean
– Contraharmonic Mean
13
Restoration in the presence of noise
only (cont.)
Geometric Mean:
1
⎡
⎤
mn
ˆ
⎢∏
f (x, y) = g(s, t) ⎥
⎥
⎢⎣
( s,t )
∈Sxy
• Achieves similar smoothing to ⎦ the
arithmetic mean, but tends to lose less
image detail.
14
Restoration in the presence of noise
only (cont.)
Harmonic Mean:
mn
ˆ
f (x, y) =
∑
1
x
g(s, t)
( s,t ) y
∈S
• Works well for salt noise, but fails for
pepper noise.
• Also does well for other kinds of noise
such as Gaussian noise.
15
Restoration in the presence of noise
only (cont.)
Contraharmonic Mean:

∑
g(s, t)Q+1
( s,t )
fˆ(x,
∈S
y) =
xy
( s,t )∈Sxy

∑
g(s, t)Q
• Q is the order of the filter.
• Positive values of Q eliminate pepper noise.
• Negative values of Q eliminate salt noise.
• It cannot eliminate both simultaneously.
 16 Noise Removal Examples
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image
Original image corrupted
by Gaussian
noise

3x3
Geometric
3x3 Mean Filter
Arithmetic (less blurring
Mean Filter than AMF, the
image is
sharper)
 17 Noise Removal Examples (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image corrupted by
pepper noise at 0.1

Filtering with a 3x3

Contraharmonic Filter
with Q=1.5
 18 Noise Removal Examples (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image corrupted by
salt noise at 0.1

Filtering with a 3x3

Contraharmonic Filter
with Q=-1.5
 19 Contraharmonic Filter: Here Be Dragons

• Choosing the wrong value for Q when using the

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

contraharmonic filter can have drastic results

Pepper noise filtered by Salt noise filtered by a

a 3x3 CF with Q=-1.5 3x3 CF with Q=1.5
 20 Order Statistics Filters
• Spatial filters based on ordering the
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

pixel values that make up the

neighbourhood defined by the filter
support.

• Useful spatial filters include

– Median filter
– Max and min filter
– Midpoint filter
– Alpha trimmed mean filter
21 Median Filter
Median Filter:
ˆ
f (x, y) = median{g(s,t)}
(s,t )∈Sxy

• Excellent at noise removal, without the

smoothing effects that can occur with other
smoothing filters.
• Particularly good when salt and pepper
noise is present.
22 Max and Min Filter
Max Filter:
ˆ
f (x, y) = max {g(s, t)}
(s,t )∈Sxy

Min Filter:
ˆ
f (x, y) = min {g(s,t)}
(s,t )∈Sxy

• Max filter is good for pepper noise and

Min filter is good for salt noise.
23 Midpoint Filter
Midpoint Filter:

ˆ 1⎡ ⎤
f (x, y) = ⎢ maxxy{g(s,
xy )∈S {g(s, t)}⎥
t)}+ (s,tmin ⎦
2 (s,t )∈S
⎣
• Good for random Gaussian and
uniform noise.
24 Alpha-Trimmed Mean Filter
Alpha-Trimmed Mean Filter:

ˆ 1
f (x, y) = mn − r

∑
( s,t ) g (s, t)
d ∈Sxy

• We can delete the d/2 lowest and d/2

highest grey levels.
• So gr(s, t) represents the remaining mn – d
pixels.
 25 Noise Removal Examples

1 pass with a
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Salt And
Pepper at 0.2 3x3 median

2 passes with 3 passes with

a 3x3 median a 3x3 median

Repeated passes remove the noise better but also blur the image
 26 Noise Removal Examples (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image Image
corrupted corrupted
by Pepper by Salt
noise noise

Filtering Filtering
above above
with a 3x3 with a 3x3
Max Filter Min Filter
 27 Noise Removal Examples (cont…)
Image corrupted Image further corrupted
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

by uniform noise by Salt and Pepper

noise

Filtering by a 5x5 Filtering by a 5x5

Arithmetic Mean Filter Geometric Mean Filter

Filtering by a 5x5 Filtering by a 5x5

Median Filter Alpha- Trimmed Mean
Filter (d=5)
28 Adaptive Filters
• The filters discussed so far are applied to
an entire image without any regard for
how image characteristics vary from one
point to another.
• The behaviour of adaptive filters
changes depending on the characteristics
of the image inside the filter region.
• We will take a look at the adaptive
median filter.
29 Adaptive Median Filtering
• The median filter performs relatively
well on impulse noise as long as the
spatial density of the impulse noise is
not large.

• The adaptive median filter can handle

much more spatially dense impulse
noise, and also performs some
smoothing for non-impulse noise.
30 Adaptive Median Filtering (cont…)
• The key to understanding the algorithm is
to remember that the adaptive median
filter has three purposes:
– Remove impulse noise
– Provide smoothing of other noise
– Reduce distortion (excessive thinning or
thickenning of object boundaries).
31 Adaptive Median Filtering (cont…)
• In the adaptive median filter, the filter size
changes depending on the characteristics
of the image.
• Notation:
– Sxy = the support of the filter centerd at (x, y)
z = minimum grey level in Sxy
– min = maximum grey level in Sxy
– zmax = median of grey levels in Sxy
z
– med = grey level at coordinates (x, y)
– zxy =maximum allowed size of Sxy
32 Adaptive Median Filtering (cont…)
Stage A: A1 = zmed – zmin
A2 = zmed – zmax
If A1 > 0 and A2 < 0, Go to stage B
Else increase the window size
If window size ≤ Smax repeat stage A
Else output zmed

Stage B: B1 = zxy – zmin

B2 = zxy – zmax
If B1 > 0 and B2 < 0, output zxy
Else output zmed
33 Adaptive Median Filtering (cont…)
Stage A: A1 = zmed – zmin
A2 = zmed – zmax
If A1 > 0 and A2 < 0, Go to stage B
Else increase the window size
If window size ≤ Smax repeat stage A
Else output zmed
• Stage A determines if the output of the median filter zmed
is an impulse or not (black or white).
• If it is not an impulse, we go to stage B.
• If it is an impulse the window size is increased until it
reaches Smax or zmed is not an impulse.
• Note that there is no guarantee that zmed will not be an
impulse. The smaller the the density of the noise is, and,
the larger the support Smax, we expect not to have an
34 Adaptive Median Filtering (cont…)
Stage B: B1 = zxy – zmin
B2 = zxy – zmax
If B1 > 0 and B2 < 0, output zxy
Else output zmed

Stage B determines if the pixel value at (x, y), that

is zxy, is an impulse or not (black or white).
If it is not an impulse, the algorithm outputs the
unchanged pixel value zxy .
If it is an impulse the algorithm outputs the median
z .
med
 35 Adaptive Filtering Example
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Image corrupted by salt Result of filtering with a Result of adaptive

and pepper noise with 7x7 median filter median filtering
probabilities Pa = Pb=0.25 with Smax = 7

AMF preserves sharpness and details, e.g. the connector fingers.

 35 Adaptive, local noise reduction filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

• The simplest statistical measures of a random

variable are its mean and variance.
• These are reasonable parameters on which to
base an adaptive filter be cause they are
quantities closely related to the appearance of
an image.
• The mean gives a measure of average
intensity in the region over which the mean is
computed, and the variance gives a measure of
contrast in that region.
 35 Adaptive, local noise reduction filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

• Our filter is to operate on a local region, Sxy.

• The response of the filter at any point (x, y) on
which the region is centered is to be based on
four quantities:
(a) g(x, y) the value of the noisy image at (x, y);
(b) the variance ofthe noise corrupting f(x, y) to
form g(x, y)
(c) mL the local mean of the pixels in Sxy;
(d) the local variance of the pixels in Sxy
 35 Adaptive, local noise reduction filter
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

• We want the behavior of the filter to be as

follows:
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

35
Adaptive, local noise reduction filter
 Periodic Noise reduction by
36
frequency domain filtering

• Periodic noise can be analyzed and filtered

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

quite effectively using frequency domain

techniques.
• The basic idea is that periodic noise appears
as concentrated bursts of energy in the
Fourier transform, at locations corresponding
to the frequencies of the periodic interference.
• The approach is to use a selective filter to
isolate the noise.
• The three types of selective filters
(bandreject, bandpass, and notch) for basic
periodic noise reduction.
 36 Periodic Noise
• Typically arises due to
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

electrical or electromagnetic
interference.
• Gives rise to regular noise
patterns in an image.
• Frequency domain techniques
in the Fourier domain are
most effective at removing
periodic noise.
37 Band Reject Filters
• One of the principal applications of bandreject
filtering is for noise removal in applications where
the general location of the noise component(s) in
the frequency domain is approximately known.
• A good example is an image corrupted by additive
periodic noise that can be approximated as
two-dimensional sinusoidal functions.
 38 Band Reject Filters (cont…)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

Ideal Band Butterworth Gaussian

Reject Filter Band Reject Band Reject
Filter (of order 1) Filter
37 Band Reject Filters
• Removing periodic noise form an image involves
removing a particular range of frequencies from that
image.
• Figure 5.16(a), which is the same as Fig. 5.5(a), shows an
image heavily corrupted by sinusoidal noise of various
frequencies.The noise components are easily seen as
symmetric pairs of bright dots in the Fourier spectrum shown
in Fig. 5.16(b). In this example, the components lie on an
approximate circle about the origin of the transform, so a
circularly symmetric bandreject filter is a good choice. Figure
5.16(c) shows a Butterworth bandreject filter of order 4, with
the appropriate radius and width to enclose completely the
noise impulses. Since it is desirable in general to remove as
little as possible from the transform, sharp, narrow filters are
common in bandreject filtering. The result of filtering Fig.
 39 Band Reject Filter Example
Image corrupted by Fourier spectrum of
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

sinusoidal noise corrupted image

Butterworth band Filtered image

reject filter
37 Band Pass Filters
• A bandpass filter performs the opposite operation of a
bandreject filter.

• Performing straight bandpass filtering on an image is

not a common procedure because it generally
removes too much image detail. However,bandpass
filtering is quite useful in isolating the effects on an
image caused by selected frequency bands
 39 Band Pass Filter Example
Image corrupted by Fourier spectrum of
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

sinusoidal noise corrupted image

Butterworth band Filtered image

reject filter
• The image was generated by (1) using Eq.to obtain the
bandpass filter corresponding to the bandreject filter used
in Fig. 5.16; and (2) taking the inverse transform of the
bandpass-filtered transform.
• Most image detail was lost, but the information that
remains is most useful, as it is clear that the noise pattern
recovered using this method is quite close to the noise
that corrupted the image in Fig. 5.16(a).
• In other words, bandpass filtering helped isolate the
noise pattern. This is a useful result because it simplifies
analysis of the noise, reasonably independently of image
content.
Noise pattern of the image in Fig. 5.16(a) obtained
by bandpass filtering.
 40 Notch Filters
• A notch filter rejects (or passes) requencies
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

in predefined neighborhoods about a center

frequency
• Due to the symmetry of the Fourier transform,
notch filters must appear in symmetric pairs
about the origin in order to obtain meaningful
results.
• Just by looking at the nearly horizontal lines of the noise
pattern in Fig. 5.19(a), we expect its contribution in the
frequency domain to be concentrated along the vertical axis.
However, the noise is not dominant enough to have a clear
pattern along this axis, as is evident from the spectrum shown
in Fig. 5.19(b). We can get an idea of what the noise
contribution looks like by constructing a simple ideal notch
pass filter along the vertical axis of the Fourier transform, as
shown in Fig. 5.19(c).
• The spatial representation of the noise pattern (inverse
transform of the notch-pass–filtered result) is shown in Fig.
5.19(d).This noise pattern corresponds closely to the pattern
in Fig. 5.19(a). Having thus constructed a suitable notch pass
filter that isolates the noise to a reasonable degree, we can
obtain the corresponding notch reject filter from Eq. The result
of processing the image with the notch reject filter is shown in
Fig. 5.19(e). This image contains significantly fewer visible
noise scan lines than Fig. 5.19(a).
 41 Optimum Notch Filtering
• Several interference components (not a single
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

burst).
• Removing completely the star-like components
may also remove image information.
 42 Optimum Notch Filtering (cont.)

• Apply the notch filter to isolate the bursts.

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

• Remove a portion of the burst.

• The procedure consists of first isolating the
principal contributions of the interference
pattern and then subtracting a variable,
weighted portion of the pattern from the
corrupted image.
• Although we develop the procedure in the
context of a specific application, the basic
approach is quite general and can be applied to
other restoration tasks in which multiple
periodic interference is a problem
• The first step is to extract the principal frequency
components of the interference pattern - can be done
by placing a notch pass filter,at the location of each
spike.
• If the filter is constructed to pass only components
associated with the interference pattern, then the
Fourier transform of the interference noise pattern is
given by the expression

• G(u, v) denotes FT of corrupted image

• After a particular filter has been selected, the
corresponding pattern in the spatial domain is obtained
from the expression

• Because the corrupted image is assumed to be formed

by the addition of the uncorrupted image f(x, y) and the
interference (x, y), if were known completely,
subtracting the pattern from g(x, y) to obtain f(x, y)
would be a simple matter. The problem, of course, is
that this filtering procedure usually yields only an
approximation of the true pattern. The effect of
components not present in the estimate of interference
can be minimized instead by subtracting g(x, y) from a
weighted portion of it to obtain an estimate of :
 43 Optimum Notch Filtering (cont.)

• A noise estimate in the DFT domain:

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

N(k,l) = H (k,l)G(k,l)

• In the spatial domain:

η(m, n) = ℑ−1 H (k,l)G(k,l)

{ }
• Image estimate:
ˆ
f (m, n) = g(m, n) − w(m, n)η(m,
 Consider a neighborhood of size by about a point (x, y).The “local”
44
variance of at coordinates (x, y) can be estimated from the samples,
as follows:

ˆ
f (m, n) = g(m, n) − w(m, n)η(m, n)
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

• Compute the weight minimizing the variance over a local

neighbourhood of the estimated image centered at (m,n):
a b
1
∑ ∑ ⎡ f (m + k, n + l) − f (m, ⎤
σ (m, n) = ˆ 2

(2a +1)(2b k =−a l =−b ⎢⎣ˆ

+1) n) a
⎥⎦
∑ ∑ fˆ(m + k, n +
1
with ˆ b
f (m, n) (2a +1)(2b k =−a l =−b
= +1) l)
• Substituting the estimate in σ(m,n): yields:
 45 Optimum Notch Filtering (cont.)
a b
Images taken from Gonzalez & Woods, Digital Image Processing (2002)

∑+1)
∑ k =−a l
σ (m, n) 1
(2a +1)(2b g(m + k, n + l) − w(m + k, n + l)η(m + k, n +
= {[ 2
− ⎡⎣l) g−⎦(m, n) − w(m,η (m, ⎤ }
=−b

n) ] n)
• A simplification is to assume that the weight remains
constant over the neighbourhood:
w(m + k, n + l) = w(m, n), − a ≤ k ≤ a, − b ≤ l ≤ b
a b

∑+1)
∑ k =−a l
σ (m, n) 1
(2a +1)(2b g(m + k, n + l) − w(m, n)η(m + k, n +
=
=−b
{[
− g l)
(m,−n) − w(m, n)η (m,
[ ]
2
n)
]
 46 Optimum Notch Filtering (cont.)

• To minimize the variance:

Images taken from Gonzalez & Woods, Digital Image Processing (2002)

∂σ (m, n)
=0
∂w(m, n)

yielding the closed-form solution:

g(m, n)η(m, n) − g (m, n)η (m, n)

w(m, n) =
η 2 (m, n) −η 2 (m, n)

• More elaborated result is obtained for non-

constant weight w(m,n) at each pixel.