IMAGE RESTORATION

Image Restoration

A process which tries to recover or restore an image which has been

degraded by some degradation method is called image restoration

We have to fnd what is the degradation model that is degraded the

image

Once we got the model, then apply reverse operation to recover

original image

The degradation process is modelled as a degradation function that

together with an additive noise term operates on an input image
f(x,y) to produce degraded image g(x,y)

Given g(x,y), some knowledge about H, and some knowledge about

the noise term, the objective is to produce an estimate of the
original image.

The more that is known about H and the noise term the closer the
estimate can be

Goal of restoration is

f ( x, y ) f ( x, y )
If H is a linear, position invariant process, then the degraded image
can be described as the convolution of h and f with an added noise
term

g(x,y)=h(x,y)*f(x,y)+(x,y)

h(x,y) is the spatial domain representation of the degradation

function

In the frequency domain, the representation is:

G(u,v)=H(u,v)F(u,v)+N(u,v)

Each term in this expression is the Fourier transform of the of the

corresponding terms in the equation above.

Noise models

Common sources of noise

(1) during image acquisition : Environmental conditions (heat,

light), imaging sensor quality
(2) during image transmission : due to interference in the channel

Spatial and frequency properties of noise

Frequency properties of noise refer to the frequency content of

noise in the Fourier sense

For example, if the Fourier spectrum of the noise is constant, the

noise is usually called white noise

Except spatially periodic noise, we will assume that noise is

independent of spatial coordinates and uncorrelated to the image

Noise probability density functions

With respect to the spatial noise term, we will be concerned with the
statistical behavior of the intensity values.

May be treated as random variables characterized by a probability

density function (PDF)

Common PDFs used will describe:

Gaussian noise

Rayleigh noise

Erlang (Gamma) noise

Exponential noise

Uniform noise

Impulse (salt-and-pepper) noise

Gaussian noise

Gaussian (normal) noise models are simple to consider.

The PDF of a Gaussian random variable, z, is given to the right

as:

In this case, approximately 70% of the values of z will be within

with in one standard deviation

Approximately 95% of the values of z will be within two

standard deviations
Rayleigh noise

The PDF of Rayleigh noise is

The basic shape of this PDF is skewed to the right

Can be useful in approximating skewed histograms

Erlang (Gamma) noise

The PDF of Erlang noise is given as:

a > 0, b is a positive integer

Exponential noise

The PDF of exponential noise is given as:

a>0

This PDF is a special case of the Erlang PDF with b=1

Uniform noise

The PDF of uniform noise is given as

Impulse (salt-and-pepper) noise

The PDF of (bipolar) impulse noise is given as:

If b>a then any pixel with intensity b will appear as a light dot in the
image

Pixels with intensity a will appear as a dark dot

This test pattern is well-suited for illustrating the noise models,
because it is composed of simple, constant areas that span the grey
scale from black to white in only three increments. This facilitates
visual analysis of the characteristics of the various noise components
added to the image
Periodic noise

Periodic noise typically arises from interference during image

acquisition

Spatially dependent noise type

Can be effectively reduced by frequency domain filtering

Figure 1 (a) image corrupted by periodic noise ( b) spectrum corresponding to image

The above figure (a) shows the image corrupted by periodic noise
(sinusoidal noise of various frequencies). The fourier spectrum of a
pure sinusoid is a pair of conjugate impulses located at the conjugate
frequencies of the sine wave. Thus if the amplitude of a sine wave is
strong enough, we would expect to see in the spectrum of image a
pair of impulses for each sine wave in the image.The figure (b) shows
the impulses appearing in an approximate circle represents the
frequency of noise.

Estimation of noise parameters

Noise parameters can often be estimated by observing the Fourier

spectrum of the image

Periodic noise tends to produce frequency spikes

Parameters of noise PDFs may be known (partially) from sensor

specification

Capture a set of flat images from a known setup (i.e. a

uniform gray surface under uniform illumination)

Study characteristics of resulting image(s) to develop an

indicator of system noise

If only a set of images already generated by a sensor are available,

estimate the PDF function of the noise from small strips of
reasonably constant background intensity

Consider a subimage (S) and let

ps(zi), i=0,1,2,L-1

denote the probability estimates of the intensities of the pixels

in S.

L is the number of possible intensities in the image

The mean and the variance of the pixels in S are given by:

The shape of the noise histogram identifies the closest PDF match.

If the shape is Gaussian, then the mean and variance are all that is
needed to construct a model for the noise (i.e. the mean and the
variance completely define the Gaussian PDF)

If the noise is impulse, then a constant (with the exception of the

noise) area of the image is needed to calculate Pa and Pb probabilities
for the impulse PDF

Restoration in the presence of noise only spatial filtering

When only additive random noise is present, spatial filtering is

commonly used to restore images.

i.e. H[f(x,y)] = f(x,y)

Or g(x,y) = f(x,y) + (x,y)

Mean filters

Order-Statistic filters

Adaptive filters

Mean filters

Arithmetic mean filter

Computes the average value of a corrupted image g(x,y) in the area

defined by a window (neighborhood)
The operation is generally implemented using a spatial filter of size
m*n in which all coefficients have value 1/mn

A mean filter smoothes local variations in an image

Noise is reduced as a result of blurring

Geometric mean filter

A restored pixel is given by the product of the pixels in an area

defined by a window (neighborhood), raised to the power 1/mn

Achieves smoothing comparable to the arithmetic mean filter, but

tends to loose less details in the process

Harmonic mean filter

A restored pixel is given by the expression

Works well for salt noise (fails for pepper noise)

Works well for Gaussian noise also

Contraharmonic mean filter

A restored pixel is given by the expression

Q is the order of the filter

Works well for salt and pepper noise (cannot do both

simultaneously)

+Q eliminates pepper noise, -Q eliminates salt noise

Q=0 arithmetic mean filter

Q=-1 harmonic mean filter

Order-Statistic filters

Median filter

Max and min filters

Midpoint filter

Alpha-trimmed mean filter

Median filter

Replaces the value of a pixel by the median of the pixel values in the
neighborhood of that pixel

The pixel at (x,y) is included in the calculation

Works well for various noise types, with less blurring than linear
filters of similar size

Odd sized neighborhoods and efficient sorts yield a computationally

efficient implementation

Most commonly used order-statistic filter

Max and min filters

The 100th percentile filter (or max filter) is given by

Useful for finding the brightest points in an image

Tends to reduce pepper noise (i.e. dark pixel values)

The 0th percentile filter (or min filter) is given by

Midpoint filter

Replaces the value of a pixel by the midpoint between the maximum

and minimum pixels in a neighborhood

Combines order statistics and averaging

Works best for randomly distributed noise (e.g. Gaussian or uniform)

Alpha-trimmed mean filter

If we delete the d/2 lowest and the d/2 highest intensity values from
a neighborhood g(s,t) of size m*n and let gr(s,t) represent the
remaining mn-d pixels, the average of the remaining pixels is called
an alpha-trimmed mean filter and is given by:

d can vary from 0 to mn-1

If d=0 the filter becomes the arithmetic mean filter

If d=mn-1, the filter reduces to a median filter

Adaptive filters

All filters considered thus far are applied to an image without regard
for how image characteristics may vary from one point to another in
the image

An adaptive filter is one whose behavior can change based on

statistical characteristics of an area within the image

This is typically the m*n filter region in the Sx,y window

Generally provides superior performance at the cost of

increased filter complexity

Adaptive, local noise reduction filter

The mean and variance are reasonable parameters upon which to

base a simple adaptive filter

They are closely related to image properties

The mean gives the average intensity over a region

The variance gives a measure of the contrast in a region

A simple filter will operate on a local region Sx,y with the response at
any point (x,y) base on four quantities:

The value of the noisy image at (x,y): g(x,y)

The variance of the noise corrupting f(x,y) to form g(x,y):

2

The local mean of the pixels in Sx,y: mL

The local variance of the pixels in Sx,y: 2L

If 2 =0, return the value g(x,y)

This is the zero-noise case where g(x,y)= f(x,y)

If the local variance (2L) is high relative to 2, return a value close to

g(x,y)

A high local variance is generally associated with image

features (i.e. an edge, etc.) and should be preserved

If 2L = 2, return the arithmetic mean of the pixels in Sx,y

This occurs if the local area has the same properties as

the overall image. Local noise is reduced by averaging.

An adaptive expression may be written as:

The only quantity that must be known is 2

Everything else can be computed from Sx,y

An assumption here is that 2 2L

 This is generally reasonable given that the noise we are
considering is additive and position independent

If this is not true then a simple test could set the ratio of
the variances to one if 2 > 2L L

Adaptive median filter

A median filter works well in the spectral density of the impulse

noise is not large

A Pa and Pb less than 0.2 is a good general rule of thumb

An adaptive median filter can handle noise with probabilities greater

than these

An additional benefit is that the adaptive median filter attempts to

preserve detail while smoothing the impulse noise

The adaptive median filter works in a rectangular window area Sx,y

The size of Sx,y is not fixed

The output of the filter is a single value that will be used to replace
the center value of Sx,y

Consider the following notation.

The algorithm works in two stages (denoted A and B)

Periodic noise reduction by frequency domain filtering

Suppose image is contaminated with periodic noise. Then what is the

procedure to remove this periodic noise

If taking the Fourier transform of periodic noise and display that the
corresponding uv locations in fourier transformation plane very
bright dots will get

That dot indicates what is the frequency of periodic noise present in

the image

Once we know the frequency components then remove it by proper

filter and take inverse fourier transform

Bandreject

Bandpass

Notch Filter

Bandreject filters
 Assume the following:

D(u,v) is the distance from the center of the frequency

rectangle

D0 is the radial center of the band of interest

W is the width of the band of interest

Ideal bandreject filter

Butterworth bandreject filter

Gaussian bandreject filter

Bandreject filtering is used for noise removal in applications where
the general location of nosie componenets in the frequency domain
is approximately known

The above figure shows an image heavily corrupted by sinusoidal

noise of various frequencies. The noise components are easily seen
as symmetric pairs of bright dots in the fourier spectrum. In this the
components lie on the approximate circle about the origin of
transform. So a circularly symmetric bandreject filter is used to
remove noise.

After that inverse fourier transform is taken and image is displayed in

spatial domain

Bandpass filters

Bandpass filter performs opposite operation of bandreject filter

HBP(u,v)=1- HBR(u,v)

HBR(u,v) is the corresponding bandreject filter

Performing straight bandpass filtering on an image cannot do as it

remove too much image details
It helps isolate the noise pattern.

Notch filters

A notch filter rejects (or passes depending on its construction)

frequencies in a pre-defined area (neighborhood) about the center of
the frequency rectangle

We desire that the filters be zero-phase-shift

Must be symmetric about the origin

A notch with center at (u0,v0) must have a corresponding

notch at (-u0,-v0)

Notch reject filters are easily constructed as products of highpass

filters whose centers have been translated to the center of the
notches

A general form for a Butterworth notch reject filter of order n and

containing three notch pairs is

The constant D0k is the same for each pair of notches, but can be
different for different pairs
A notch pass filter can be expressed as

Optimum Notch Filters

f x , y g x , y w x , y x , y
x, y F 1 H u, v G u, v
g x, y x, y g x, y x, y
w x, y
2 x, y 2 x, y
To obtain w(x,y) the goal is to minimize the variance in the
neighborhood of x,y in the image

Estimating the degradation function

There are three principal ways to estimate the degradation function

to be used in the restoration:

1. Estimation by Observation;

2. Estimation by Experimentation;

3. Estimation by Mathematical modeling.

The process of image restoration by use of the estimated

degradation function is sometimes called blind deconvolution

Estimating the degradation function: Observation

Suppose that we observe an image g(x,y) degraded with an unknown

linear H.

we will try to obtain some information about the degradation

function from the image itself.

For example, if the image is blurred, we can look at a small

rectangular selection of the image containing a part of an object and
background.

To reduce the effects of noise, we look for an area of strong signal

(area of high contrast) and try to process that subimage to un-blurr it
as much as possible (for instance, by sharpening subimage with a
sharpening filter).

Let denote the original subimage by gs(x,y) and the its restored
version by , we can assume

Gs (u, v)
H s (u, v)
Fs (u, v)
From the characteristics of this function, we then deduce the
complete degradation function H(u,v) based on the assumption of
position invariance.

For example, if Hs(u,v) has a Gaussian shape, we can construct H(u,v)

on a larger scale with the same (Gaussian) shape

Clearly, this is a quite involved process and used in very specific

situations.

Estimating the degradation function: Experimentation

If equipment similar to the equipment used to acquire the degraded

image is available, images similar to the degraded images can be
acquired with various system settings until they degraded as closely
as possible to the image that needs to be restored.

Then the idea is to obtain the impulse response of the degradation

by imaging an impulse (small dot of light) using the same system
settings, since a linear space-invariant system is characterized
completely by its impulse response.

An impulse is simulated by a maximally bright (to reduce the effect of

noise) dot of light.
 G (u, v)
H (u, v)
A
Estimat ing the degradation function: Modeling

Atmospheric Turbulence Blur

modeled by a Gaussian pdf given by

k (u2 v2 )5/ 6
H (u, v) e
k is a constant depending on the nature of turbulence.

Common in remote sensing and aerial imaging.

K=6/5, it is like Gaussian low pass filter.

 Motion Blur

Assuming an ideal imaging system with the duration of exposure T

and that an image f(x,y) undergoes planar motion with time-
varying components x0(t) and y0(t), the blurred image is

In the frequency domain:

Which is by shifting property of F.T

The degradation is

If the motion variables are known, the degradation transfer function

can be obtained.

For instance, assuming that the image motion is at a rate x0(t) = at/T
and y0(t) = bt/T, the degradation function will be:
Note: very important topics
(1) Weiner(minimum mean square error) filtering (Refer
Gonzalez page no: 374-377)
(2) Direct inverse filtering (Refer Gonzalez page no: 373-374)
(3) Estimating the degradation function ( by observation,
experimentation and modelling)
(4) Periodic noise reduction by frequency domain filtering (
make understand what is periodic noise also)
(5) Noise models ( Gaussian, Rayleigh, exponential, erlang,
uniform,impulse)
(6) Restoration in the presence of noise-only spatial filtering
(Mean filters, order statistic filters, adaptive filters)
(7) Image degradation/restoration model

It is better to study complete 3rd unit.(easy to study)