Image Deblurring

Fu-Wen Yang, Hwei Jen Lin, and Hua Chuang

Department of Computer Science and Information Engineering
Tamkang University
Taipei, Taiwan, R.O.C.
086204@mail.tku.edu.tw

Abstract—Multimedia is ubiquitous and the application of (1)

= ⨂ +
digital imaging is prolific, yet environmental conditions and hard
ware limitations may adversely affect image quality. Advanced Usually, blur kernel estimation is considered as an
techniques such as image enhancement, deblurring, denoise, and optimization problem to minimize the energy function w.r.t.
super resolution have been developed to improve image quality
post-digitization. Image enhancement is primarily concerned blur kernel K is E(K) = , which is an ill-posed
with problems caused by overexposure, underexposure, poor problem. J. Pan et al. [3] and Q. Shan et al. [4] added some
photographic technique, and optical noise. A. Gorai and A. regularization to properly solve such an ill-posed problem and
Ghosh proposed a method based on a heuristic algorithm to estimated the blur kernel in the frequency domain. Z. Hu et al.
enhance images by adjusting brightness and contrast. Lighting [5] used the same regularization as J. Pan et al. did to estimate
problems are resolved effectively through this method, yet there the blur kernel in the spatial domain.
are limitations when applied to blurred images, i.e., images with As soon as the blur kernel K is estimated, the latent
defocus blur, motion blur, handshake blur, or fog blur. For those
kinds of blurred images, we need a specific technique of image
image can be also estimated by performing deconvolution on
deblurring to apply. As a result, this research is primarily the blurred image B with the kernel K. Inverse filtering is the
concerned with (a) image enhancement: improving upon the simplest and most naïve method for deconvolution [6]. The
objective function and transformation function proposed by A. main drawback of the inverse filter is that it generates high-
Gorai and A. Ghosh and (b) image deblurring: a proposed frequency noise and causes significant artifacts. Wiener
method to estimate blur kernel and its application towards image deconvolution [7] and Richardson–Lucy deconvolution [8]
deblurring. were then proposed to reduce the problems and yield better
This paper proposes a blind deblurring method needing to deblurring results. Most of the recent proposed deconvolution
predict a blur kernel in our own way. The color distribution of techniques are variants of Wiener deconvolution or
edge is more distinct in a clear image than in a blurred image. A
filter is proposed to make edges in a blurred image clearer for
Richardson–Lucy deconvolution.
use as a reference image. The blur kernel is estimated from this D. Krishnan and R. Fergus [9] proposed the hyper-
reference image. The blurred image is then deconvolved with the Laplacian regularization deconvolution, which used half-
estimated blur kernel to introduce a latent image. quadratic splitting to divide the minimization problem into
two subproblems. The two sub-problems are solved by
Keywords—image deblurring, ringing artifacts, image alternating between two steps, one where they solve for one
enhancement, edge enhancement, Particle Swarm Optimization problem, given a solution of the other and vice-versa. O.
(PSO), deconvolution Whyte et al. [10] proposed a deconvolution method based on
Richardson–Lucy deconvolution, which used sensor saturation
I. INTRODUCTION to filter out the saturation area so that the artifacts can be
In daily life, image blur occurs in most cases of image suppressed.
deterioration resulting from defocusing or hand shaking. The
II. THE PROPOSED METHOD
above mentioned image enhancement methods cannot tackle
all the blur problems, which are usually resolved by image The proposed image deblurring method estimates a blur
deblurring methods. Image deblurring is one of the most kernel for a blurred image, and uses it to deconvolve the
fundamental problems in image restoration that has been blurred image to obtain an estimated latent image.
studied extensively in the literature [1, 2]. First, confirm that you have the correct template for your
The image blur is usually modeled as a linear image paper size. This template has been tailored for output on the A4
degradation process, as shown in (1), where B, I, and n paper size. If you are using US letter-sized paper, please close
represent the degraded (or blurred) image, the latent (or this file and download the file “MSW_USltr_format”. Similar
to most methods, the proposed method for kernel estimation
unblurred) image, and the additive noise respectively, ⨂ is the
produces in advance a reference image R having sharper edge
convolution operator, and K is an unknown Point Spread
from the given blurred image B, the correspondence between
Function (PSF), called blur kernel. images B and R is then used for blur kernel estimation. A latent
image can be obtained by performing deconvolution on the

978-1-5386-0435-9/17/$31.00 ©2017 IEEE

blurred image with the estimated blur kernel. This section (8)
describes how to estimate a latent image through blur kernel
estimation and deconvolution in detail. B. Latent image estimation
Once the kernel is estimated, we may perform
A. Blur Kernel estimation
deconvolution on the blurred image B with the estimated
For the input blurred image B, we initially set I = B and kernel K to obtain an intermediate latent image I. We use the
perform the proposed edge enhancement on I to introduce a deconvolution method proposed by J. Pan et al. [3] to obtain
reference image IR, as shown in (5), where m(x) is the mean of the latent image. J. Pan et al. considered deconvolution as an
intensity values in the neighborhood N(x) of pixel x in image I, optimiza -tion problem, as shown in (9), where and
mh(x) is the mean of intensity values greater than m(x) in N(x), denote the horizontal and vertical differential operators,
ml(x) is the mean of the intensity values less than m(x) in N(x), respectively.
as shown in (2)-(4). The resulting reference image IR will have
sharper edge than the blurred image B. Fig. 1 shows an example (9)
of such edge enhancement.
Based on the half-quadratic splitting L1 minimization
(2) method, the problem can be solved using an efficient
alternating minimization method. Two auxiliary variables wx
and wy are introduced so that the objective function can be
,
(3) rewritten as the form given in (10).

, (10)
(4) The problem in (10) can be split into three subproblems:
wx subproblem, wy subproblem, and I subproblem, as shown in
(11), (12), and (13), respectively, so that it can be efficiently
(5) solved through alternatively solving the three subproblems
independently by fixing the other variables.

(11)

(12)

(13)
(a) (b) (c)
Given I, wx and wy are computed by the soft
Fig. 1. edge enhancement (a) ground truth I (b) blurred image B (c) edge thresholding/shrinkage operator [11], as given in (14), where t
enhanced image IR = λ/β and xi, j =’hI(i, j) for solving wx in (11); and t = λ/β and xi,
j = ’vI(i, j) for solving wy in (12).
The reference image IR is then used for blur kernel
estimation. We consider blur kernel estimation as an (14)
optimization problem and adopt the objective function the
same as the one used by Z. Hu [5], in which additional In each iteration, the solution of I is obtained by solving
regularizetion constraints are introduced in order to get a the least squares minimization problem in (13) and the closed-
stable solution from Tikhonov regularization, as shown in (6), form solution for (13) is given in (15), where F() and F-1()
where is set to a multiple of the identity matrix. With denote the Fast Fourier Transform (FFT) and inverse FFT,
Tikhonov regularization the solution is given by (7). respectively, is the complex conjugate operator, parameter
β is set to 1000 and divided by two every iteration, and’h = [1,
(6)
-1],’v = [1,-1]t.
(7)
To reduce noise, we reset small entries in the kernel to
zero with a threshold Tn. Finally, the kernel entries are (15)
normalized such that their sum remains 1, as shown in (8),
where Z is the normalization factor. The threshold Tn is set to a
multiple of the maximum value in the kernel.
 C. Image deblurring
The proposed image beblurring algorithm iteratively
TABLE II. COMPARISONS OF IMAGE DEBLURRING
estimates a blur kernel K and produces an intermediate latent
image I by convolving the input blurred image B with the house Picasso clock roof
estimated blur kernel K. The initial latent image I is set to the ours PSNR 33.62 36.10 36.10 29.35
input image B. The blur kernel K is estimated (or updated) SSIM 0.9336 0.9322 0.9322 0.8442
according to the correspondence between the two images B and J. Pan PSNR 31.38 33.35 33.35 28.75
et al. SSIM 0.8891 0.9068 0.9068 0.8257
IR using (5). This process is repeatedly executed until a
L. Xu PSNR 29.17 32.21 32.21 29.14
stopping criterion is met. A flowchart for image deblurring is et al. SSIM 0.7786 0.8442 0.8442 0.8160
given in 䭉䈟!ᵚ᢮ࡠᕅ⭘ⓀDŽ.

Perform edge Estimate K

Input enhancement
blurred with B and
on I to obtain a Ŋœ ġ
image B, reference
and let I = B image Ŋœġ

Noġ ground truth blurred image (PSNR: 32.73), (SSIM: 0.8905)

Update I by
Output the Is
deconvolvi
latent stopping
ng B with Kġ
image Iġ criterion
Yesġ
met?ġ

a. J. Pan (PSNR: 32.01), L. Xu (PSNR: 33.47), ours (PSNR: 35.73),

Fig. 2. Flowchart for image deblurring (SSIM: 0.8696) (SSIM: 0.9057) (SSIM: 0.9420)

III. EXPERIMENTAL RESULTS Fig. 3. Deblurring results of “hair” image

In this section, the proposed image deblurring method is
compared with the state-of-art algorithms [3, 7]. Results were
obtained on an Intel Core2 CPU 2.40GHz CPU, 8.0 G RAM
and Window7 Enterprise 64bytes.
In this study, we focus on deblurring the defocus blurred
image, and thus we adopt the small radius and based on
Gaussian blur kernel.
Tables I and II list the PSNR (Peak Signal-to-Noise Ratio)
blurred image (PSNR: 33.75), (SSIM: 0.9367)
and SSIM (Structural SIMilarity) values for some test images, ground truth
which show that our method achieves the best PSNR values
and SSIM, and estimates blur kernels most similar to ground
truths. To compare our approach with the methods proposed by
J. Pan et al. [3] and L. Xu et al. [7]. Resulting images of part of
the above tests are shown in Figures 3 and 4. Some results on
color images results of part of the above test are shown in
Figures 5 and 6.

ours (PSNR: 35.49),

J. Pan (PSNR: 32.55), L. Xu (PSNR: 31.64),
(SSIM: 0.9550)
(SSIM: 0.9074) (SSIM: 0.8732)
TABLE I. COMPARISONS OF IMAGE DEBLURRING
cameraman hair church face Fig. 4. Deblurring results of “face” image
ours PSNR 33.19 35.73 32.01 35.49
SSIM 0.9163 0.9420 0.8573 0.9550
J. Pan PSNR 31.24 32.01 31.21 32.55
et al. SSIM 0.9057 0.8696 0.8269 0.9074
L. Xu PSNR 32.39 33.47 31.10 31.64
et al. SSIM 0.9057 0.9057 0.8273 0.8732
 IV. CONCLUSION AND FUTURE WORK
This paper proposes an image deblurring algorithm for
improving image quality. With image deblurring, satisfying
results can only be seen when the blur is an outcome of defocus.
Some bottlenecks require improvement such as (a) although we
are able to estimate the blur kernel with great accuracy in the
spatial domain, the processing time increases with the size of
blurred image (PSNR: 31.08), (SSIM: 0.8923) the radius. Therefore, we are unable to estimate large-sized blur
kernels efficiently. (b) During blur kernel estimation, the size
of the kernel is given; yet, we aim, in the near future, to do so
automatically. (c) So far, our image deburring method can only
processes defocus blurring, for matters such as motion blur,
handshake blur, or fog blur, we could not generate ideal
outcomes. (d) The blur kernel has been assumed to be spatially
invariant (i.e., the entire image is assumed to be blurred with
ground truth J. Pan (PSNR: 29.51), (SSIM: 0.6502)
one single blur kernel). In the future, we hope to solve the
above-mentioned problems and to combine other methods in
order to achieve a more effective and comprehensive image
deblurring results.

ACKNOWLEDGMENT
This work was supported by the Ministry of Science and
L. Xu (PSNR: 29.55),
(SSIM: 0.6529)
ours (PSNR: 33.28), (SSIM: 0.9430) Technology, R.O.C. under the grant MOST-105-2221-E-032-
054.
Fig. 5. Deblurring results of image “baby”

REFERENCES
[1] M. Ben-Ezra and S. Nayar, “Motion-based motion deblurring,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, 26(6):689–
698, 2004.
[2] J. Biemond, R. Lagendijk, and R. Mersereau, “Iterative methods for
image deblurring,” Proceedings of the IEEE, 78(5):856–883, 1990.
[3] J. Pan, Z. Hu, Z. Su, and M.-H. Yang, “Deblurring text images via L0-
blurred image (PSNR: 28.5949), (SSIM: 0.8038) regularized intensity and gradient prior,” in Proc. of the Conference on
Computer Vision and Pattern Recognition (CVPR), pp. 2901-2908, June,
2014.
[4] Q. Shan, J. Jia, and A. Agarwala, “High-quality motion deblurring from
a single image,” ACM Transactions on Graphics, Vol. 27, No. 3, pp. 73-
83, August, 2008.
[5] Z. Hu, J. B. Huang, and M. H. Yang, “Single image deblurring with
adaptive dictionary learning,” 17th IEEE International Conference
on Image Processing (ICIP), pp. 1169-1172, September, 2010.
ground truth J. Pan (PSNR: 28.1253), (SSIM: 0.7172)
[6] D. Miller and W. Scott, “Deconvolution with inverse and Wiener
filters,” Connexions, pp. 1-4, 2006.
[7] L. Xu, S. Zheng, and J. Jia, “Unnatural l0 sparse representation for
natural image deblurring,” the Conference on Computer Vision and
Pattern Recognition (CVPR), pp. 1107-1114, June, 2013.
[8] R. L. White, “Image restoration using the damped Richardson-Lucy
method,” Symposium on Astronomical Telescopes & Instrumentation
for the 21st Century, pp. 1342-1348, June, 1994.
L. Xu (PSNR: 28.14),
ours (PSNR: 30.15), (SSIM: 0.9231) [9] D. Krishnan and R. Fergus, “Fast image deconvolution using hyper-
(SSIM: 0.7422)
Laplacian priors,” the Twenty-ninth Annual Conference on Neural
Information Processing Systems (NIPS), pp. 1033-1041, 2009.
Fig. 6. Deblurring results of Image “flower”
[10] O. Whyte, J. Sivic, and A. Zisserman, “Deblurring shaken and partially
saturated images,” International Journal of Computer Vision, 110(2),
185-201, 2014.
[11] N. Komodakis and N. Paragios, “MRF-based blind image
deconvolution,” the 11th Asian Conference on Computer Vision (ACCV
2012), Part III, LNCS 7726, pp. 361-374, 2013