Image restoration

Image restoration techniques aim at

modelling a degradation corrupting the
image and inverting this degradation to
correct the image so that it is as close as
possible to the original.


 IE PŁ IE PŁ P. Strumiłło
 Image restoration vs. image enhancement

Image enhancement Image restoration

Heuristic criteria; there is Quantitative criterion used;

no image degradation image model and degradation
model constructed, no model required
mathematical criteria used

 IE PŁ P. Strumiłło
 Image degradation model
Source Degraded
image image

degradation η(x,y)

f(x,y) H(.) + g(x,y)

g ( x , y ) = H [ f ( x , y )] + η ( x , y )

 IE PŁ P. Strumiłło
 Image degradation model

Assume H(.) is linear and shift invariant:

∞
g (x , y ) = ∫ ∫ f (α , β ) h(x − α , y − β ) dα dβ + η (x , y )
−∞

thus for the spectrum domain:

G (u , v ) = H (u , v ) ⋅ F (u , v ) + N (u , v )

 IE PŁ P. Strumiłło
 Estimating the degradation function

How to estimate the degradation function?

• build a mathematical model of the degradation
(example given),
• reproduce the degradation process on a known
image,

 IE PŁ P. Strumiłło
 Degradation estimation example
Assume f(x,y) is linearly shifted during exposure and
x0(t), y0(t) are coordinates of motion. The degraded
image is given by:

T
g (x, y ) = ∫ f (x − x0 (t ), y − y0 (t ) )dt
0

Fourier transform of the degraded image:

∞
G (u , v ) = ∫ ∫ g (x , y ) exp[− j 2π (ux + vy )]dx dy
−∞
 IE PŁ P. Strumiłło
 Image restoration example
Hence:
∞ T 
G (u , v ) = ∫ ∫  ∫ f (x − x0 (t ), y − y0 (t ) )dt  exp[− j 2π (ux + vy )]dx dy
−∞  0 

By inverting the order of integration and using Fourier

transform property of the displaced function :
T
G (u, v ) = F (u , v) ⋅ ∫ exp[− j 2π (ux0 (t ) + vy0 (t ) )]dt
0

H (u , v )
 IE PŁ P. Strumiłło
 Image restoration example

Assume the motion is parallel to X axis and it is a liner,

uniform motion, i.e. x0=At/T and y0=0:
T T
 j 2πuAt 
H (u, v ) = ∫ exp[− j 2πux0 (t )]dt = ∫ exp −  dt =
0 0  T 
T
= sin (πuA)e − jπuA
πuA
Note that H(u,v) has zeros (!) for u=i/A,
where i is an integer number.

 IE PŁ P. Strumiłło
 Image restoration example

|H(u,v)| 1

0.9

0.8 H (u , v ) = T sin c(πuA)

0.7

0.6

0.5

0.4

0.3

0.2 T=1; A=32

0.1

0
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
u

Zeros: u=i/A=0.313, 0.626, …..

 IE PŁ P. Strumiłło
 Image restoration example

f(x,y) g(x,y)

Degradation

Source image Degraded image,

N*N =256*256 shift: A=1/8*N=32

 IE PŁ P. Strumiłło
 Inverse filtering

G (u , v ) = H (u , v ) ⋅ F (u , v ) + N (u , v )

Spectrum of the source image can be estimated from:

1 1
Fˆ (u , v ) = ⋅ G (u , v ) = F (u , v ) + ⋅ N (u , v )
H (u , v ) H (u , v )

Inverse filter

Numerical instability for u, v for which H(u,v)≈0; i.e., for

zeros of H(u,v) and frequencies for which N(u,v) >F(u,v).
 IE PŁ P. Strumiłło
 The Wiener filter

The Wiener filter is given by:

F (u , v ) =
ˆ H *
(u, v )
⋅ G (u , v ) = W (u , v ) ⋅ G (u , v )
2 Sη (u , v )
H (u , v ) + γ
S f (u , v )

where: Sf(u,v) - image power spectral density,

Sη(u,v) - noise power spectral density.

2
H (u , v ) = H * (u , v ) ⋅ H (u , v )

 IE PŁ P. Strumiłło
 The Wiener filter
The parametric Wiener filter:
2
1 H (u , v )
Fˆ (u , v ) = ⋅ ⋅ G (u , v )
H (u , v ) 2 Sη (u , v )
H (u , v ) + γ =K
S f (u , v )

• γ adjustable factor,
• for Sη(u,v)≈0 or K→0 the Wiener filter is an inverse
filter,
• for K→∝ the Wiener filter becomes a low-pass filter.

 IE PŁ P. Strumiłło
 The Wiener filter

If Sf(u,v) and Sη(u,v) are unknown, the Wiener filter can

be approximated by the formula:

2
1 H (u , v )
Fˆ (u , v ) = ⋅ ⋅ G (u , v )
H (u , v ) H (u , v ) + K
2

where K∈R+.

In practice, K ~ σ2, where σ2 is the noise variance that is

easy to estimate.
 IE PŁ P. Strumiłło
 Wiener filter spectral characteristics
|Sf(u,v)|
u
|Sη(u,v)|
White noise

u
|H(u,v)|2

u
|W(u,v)|
1
u
0 Inverse filter Low-pass filter
 IE PŁ P. Strumiłło
 Image restoration examples

SNR=1

SNR=10

SNR=100

 Addison-Wesley

Degraded images Inverse filtering Wiener filtering

 IE PŁ P. Strumiłło
 Image restoration examples
An image
and its
spectrum

The filter
Degraded image and
and its
its spectrum
spectrum

(7x7)

 IE PŁ P. Strumiłło
 Image restoration examples

Inverse filter

Wiener filter

Degraded image
(blur+noise)

 IE PŁ P. Strumiłło
 x=imread('text.tif'); %load image MATLAB
figure(1),imshow(x); %show image
N=256;
M=7; % filter order
hg=fspecial('average',M);
hhg=zeros(N,N);
hhg(N/2-(M-1)/2:N/2+(M-1)/2, N/2-(M-1)/2:N/2+(M-1)/2)=hg; %filter mask image
y=filter2(hg,x); %low-pass filtering
y=y+0.001*randn(256,256);
figure(2),imshow(y);

Y=fft2(y); %FFT of the degraded image

X=fft2(x); %FFT of the source image
Hg=fft2(hhg); %FFT of the low-pass impulse response
%----------spectra plots ----------------
figure(3),imshow(log(abs(fftshift(X)))+1,[ ]);
figure(4),imshow(log(abs(fftshift(Y)))+1,[ ]);
figure(5),imshow(log(abs(fftshift(Hg)))+1,[ ]);

Xp=Y./Hg; %inverse filtering

xp=abs(ifft2(Xp)); %IFFT
figure,imshow(fftshift(xp),[ ]);

 IE PŁ P. Strumiłło
 Example of interactive image restoration

FFT

IFFT

 IE PŁ P. Strumiłło
 Geometric image distortions

Assume image pixel coordinates (x,y) undergo

geometric distortions. A new image g ( x' , y ' ) is obtained
with coordinates defined as:

x ' = r ( x, y )
y ' = s ( x, y )
eg. for r(x,y) =x/2 and
s(x,y)=y/2 image size is
reduced by a factor of 2.
 IE PŁ P. Strumiłło
 Correction of geometric image distortions

Spatial transformation functions r(x,y) i s(x,y) can be

estimated from a limited number of tie points eg. 16×16
pixels distributed regularly in the image.

( x' , y ' ) x ' = r ( xk , y k )

( x, y ) y ' = s ( xk , y k )

f ( x, y ) g ( x' , y ' )
 IE PŁ P. Strumiłło
 Correction of geometric image distortions

s ( xk , y k )

r ( xk , y k ) k = 1,K,16

 IE PŁ P. Strumiłło
 Correction of geometric image distortions

Distortion functions r(x,y) i s(x,y) can be

approximated e.g. by second order polynomials:

r ' ( x, y ) = a0 + a1 x + a2 y + a3 x 2 + a4 xy + a5 y 2

s ' ( x, y ) = b0 + b1 x + b2 y + b3 x 2 + b4 xy + b5 y 2

Hence, 12 coeffcients a0, a1,…, a5 and b0, b1,…, b5

need to be determined.

 IE PŁ P. Strumiłło
 Correction of geometric image distortions

Optimum polynomial coefficients can be calculated from:

 K

ε r = ∑ (r ' ( x, y ) − r ( xk , yk ) )
2

 k =1
 K
ε = (s ' ( x, y ) − s ( x , y ) )2
 s ∑ k =1
k k

Samples taken for the tie points K>=6

 IE PŁ P. Strumiłło
 Correction of geometric image distortions

− s ( xk , y k )

Coordinates after correction: ( x' , y ' )

 x′′ = x' − r' ( x' , y' )


 y ′′ = y' − s' ( x' , y' )
( x′′, y′′) − r ( xk , y k )

 IE PŁ P. Strumiłło
 Correction of geometric image distortions

Image geometrically distorted Corrected image

 IE PŁ P. Strumiłło
 Examples of geometric distortions

source barrel pincushion

perspective shift skew rotation

 IE PŁ P. Strumiłło
 Correction of geometric distortions - examples

© Copyright 2005 Altostorm Software

 IE PŁ P. Strumiłło