Digital Image Processing

7 December 2006

Dr. ir. Aleksandra Pizurica

Prof. Dr. Ir. Wilfried Philips

Aleksandra.Pizurica @telin.UGent.be
Tel: 09/264.3415

UNIVERSITEIT
GENT
Telecommunicatie en
Informatieverwerking

Image Restoration
version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Objectives of Image Restoration

Image restoration likewise image enhancement attemts at improving

the image quality

Some overlap exists between image enhancement and restoration

Important differences: image enhancement is largely subjective, while

image restoration is mainly objective process

Restoration attempts to recover an image that has been degraded by

using a priori knowledge about degradation process

Restoration techniques involve modelling of degradation and applying

the inverse process in order to recover the image

The restoration approach usually involves a criterion of goodness (e.g.,

mean squared error, smoothness, minimal desription length,) that
will yield an optimal estimate of the desired result

04.a3

version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Overview of restoration techniques

A categorization according to the degradation model (noise, blur or both)

Another possible categorization:

Spatial domain techniques
Frequency domain techniques
Other transform domain (e.g., wavelet) techniques

Model based approaches:

Bayesian techniques make use of a priori knowledge about the
unknown, undegraded image statistical image modeling
Total variation involves regularization penalization of not-desired
local image structures

Statistical image modeling

Modeling marginal statistics (image histograms)
Context models modeling interactions among pixel intensities
Powerful contextual models: Markov Random Field (MRF) models
04.a4
 Degradation model

version: 7/12/2006 W.Philips,

A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

Reminder: camera model

x, k
f0(x,y)
h(x,y)
f ( xk , yl )
fi(x,y) = f kl

y, l
Camera: CCD (Charged-
Optical system coupled device) pixel matrix

A pixel sensor measures the image intensity in the neighborhood of (xk,yl)

f kl = f ( xk , yl ) = f o ( xk x' , yl y ' ) w( x' , y ' )dx' dy ' = ( f o w)( xk , yl )
weighting function, e.g. w(x,y)=1
for |x|< and |y|< and 0 otherwise

Remark: f kl = f ( xk , yl ) where f ( x, y ) = ( f i h w)( x, y ) = ( f h' )( x, y )

Mathematical model: linear filter followed by ideal sampling 04.a6
version: 7/12/2006 W.Philips,
A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

Reminder: camera model

averaging ideal
lens
fi(x,y) over pixels f(x,y) sampling fkl=f(xk,yl)
linear filter

The linear filter here is low-pass (attenuates high frequencies)

The image becomes blurred, fine details are lost

The sampling keeps only the values of f(x,y) at discrete positions (xk, yl)
Aliasing appears if sampling frequency is not high enough

Remarks
Uniform sampling: xk=k, yl=l
More general xk k
= V (sampling matrix V)
(sampling lattice)
yl l
! For compactness we shall write f(x,y) instead of f(xk,yl) !
04.a7

version: 7/12/2006 W.Philips,

A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

Degradation model
Scene Not-ideal Ideal
anti-alias filter sampling
+ Gk ,l (DFT-coefficients)

H( fx, fy) noise

fx
Equivalent model Gk ,l =
sampling frequency
H k ,l Fk ,l + N k ,l
Scene Lens with ideal Equivalent
Ideal
frequency
sampling
(digital) PSF + B ' k ,l
characteristic Bk ,l (linear filter)
noise
Hideaal( fx, fy) Hk,l
fx k

If there is no aliasing we can model the analogue PSF by a digital filter

04.a8
version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Degradation model
Scene Lens with ideal
Ideal Equivalent
frequency + Gk ,l
characteristic
sampling Fk ,l (digital) PSF

noise
Hk,l N k ,l
Gk ,l = H k ,l Fk ,l + N k ,l
k
equivalent
equivalent
g ( xk , yk ) = h( xk , yl ) f ( xk , yl ) + n( xk , yl )

Degradation
For compactness we
function h(x,y)
+ g ( x, y )
write x,y instead of xk,yl f ( x, y )
noise
n(x,y)

Digital filter: models imperfections in the lens, form of the pixels,

(if aliasing appears equivalent with analogue PSF may not hold) 04.a9

Noise reduction
version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Why is denoising important

original denoised
Not only visual
enhancement, but
also: automatic
processing is
facilitated!

Example:
edge detection

04.a11

version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Noise models
Noise models can be categorized according to

marginal statistics (first-order statistics, marginal probability density function):

Gaussian, Rayleigh, Poisson, impulsive,

higher-order statistics
white noise (uncorrelated)
colored (correlated)

type of mixing with the signal

additive
multiplicative
other (more complex)

dependence on the signal

statistically independent of the signal
statisticaly dependent of the signal

Many techniques assume additive white Gaussian noise (AWGN) model

04.a12
version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Noise models: marginal statistics

Some common probability densitu functions (pdfs)of noise:
* Gaussian e.g., thermal noise and a variety of noise sources
* Rayleigh e.g. amplitude of random complex numbers whose real and
imaginary components are normally and independently
distributed. Examples: ultrasound imaging

Rayleigh Rice Impulsive

* Rice e.g., MRI image magnitude (Gaussian and Rayleigh are

special cases of this distribution)
* Poisson models photon noise in the sensor (an average number of
photons within a given observation window)
* Bipolar impulsive (e.g., salt and pepper) noise
04.a13

version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Noise in MRI
f = [ f1 ,..., f N ] - ideal, noise-free data;
d = [d1 ,..., d N ] - observed noisy image (complex-valued);

d l = ( f l cos + nl ,Re ) + j ( f l sin + nl ,Im )

ml = | d l |= ( f l cos + nl ,Re ) 2 + ( f l sin + nl ,Im ) 2

low SNR
p(m) ( f = 0, n2 = 1)
The magnitude ml
is Rician distributed high SNR
( f = f1 , n2 = 1)

f1 m
04.a14
version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Noise in MRI

p(m) low SNR noisy m

(f=0)

magnitude

contrast
high SNR
( f =f1 )
noise-free f

f1 m SNR

04.a15

version: 7/12/2006
B.A.
Goossens,
Pizurica, Universiteit Gent, 2006

Noise models: correlation properties

Original image Image with white noise Image with colored noise

Difference with Difference with

the original the original
white uncorrelated
noise
colored correlated

04.a16
version: 7/12/2006 W.Philips,
A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

Some reasons behind noise correlation

Bayer pattern

captured image interpolated

04.a17

version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Some reasons behind noise correlation

Interpolated
noise is correlated

Raw data in one

color channel
noise is white

Resulting color image

with correlated noise

RAW data B channel Denoised RAW data - B ch.

04.a18
version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Types of mixing noise with signal

In many applications it is assumed that noise is additive and statistically
independent of the signal

g ( x , y ) = f ( x, y ) + n ( x , y )

This is a good model for example for thermal noise

Often, noise is signal-dependent. Examples: speckle, photon noise,
Many noise sources can be modelled by a multiplicative model:

g ( x , y ) = f ( x, y ) n ( x , y )

In CMOS sensors there is a fixed-pattern noise and mixture of

additive and multiplicative noise

04.a19

version: 7/12/2006 W.Philips,

A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

Order statistics filters: Median filter

x
Input image Output image
y
1 1 2 2 2
4 3 2 3 5
0 9 1 0 2 2
1 2 3 1 1
1 3 3 2 2

b( x , y ) g ( x, y )
011223349
sorting
Basic idea: remove outliers
The median is a more robust statistical measure than mean

04.a20
version: 7/12/2006 W.Philips,
A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

Reduction of impulse noise

impulse noise median over 3x3

Median filter removes isolated noise peaks, without blurring the image

04.a21

version: 7/12/2006 W.Philips,

A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

... Reduction of impulse noise

Noise-free original median over 3x3

Median filter removes isolated noise peaks, without blurring the image

04.a22
version: 7/12/2006 W.Philips,
A. Pizurica,
Universiteit
Universiteit
Gent, Gent,
1999-2006
2006

Median filter and reduction of white noise

original median over 3x3

For not-isolated noise peaks (e.g., white Gaussian noise) median filter
is not very efficient.

04.a23

version: 7/12/2006 A. Pizurica, Universiteit Gent, 2006

Some simple noise filters

Arithmetic mean Median
1
f ( x, y ) = g ( s, t ) f ( x, y ) = median{g ( s, t )}
mn ( s,t )S xy ( s ,t )S xy

Average within a local window Sxy Efficient for impulsive noise

Aimed for Gaussian noise, Not efficient for Gaussian noise
(but blurs edges)
Median
Alpha-trimmed mean
Discard d/2 lowest and d/2 largest values in Sxy
Denote by gr(s,t) the remaining mn-d pixels

1
f ( x, y ) = g r ( s, t )
mn d ( s ,t )S xy
For d=0: mean filter
For d=mn-1: median filter 04.a24