See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/224212806

Real-Time Dehazing for Image and Video

Conference Paper · October 2010

DOI: 10.1109/PacificGraphics.2010.16 · Source: IEEE Xplore

CITATIONS READS
66 2,986

3 authors, including:

Wenbin Chen
Fudan University
125 PUBLICATIONS 2,451 CITATIONS

SEE PROFILE

All content following this page was uploaded by Wenbin Chen on 26 December 2019.

The user has requested enhancement of the downloaded file.

 2010 18th Pacific Conference on Computer Graphics and Applications

Real-time Dehazing for Image and Video

Xingyong Lv Wenbin Chen I-fan Shen

School of Computer Science School of Mathematical Science School of Computer Science
Fudan University Fudan University Fudan University
Shanghai, China Shanghai, China Shanghai, China
fudanlxy@gmail.com wbchen@fudan.edu.cn yfshen@fudan.edu.cn

Abstract—Outdoor photography and computer vision tasks

often suffer from bad weather conditions, observed objects
lose visibility and contrast due to the presence of atmospheric
haze, fog, and smoke. In this paper, we propose a new method
for real-time image and video dehazing. Based on a newly
presented haze-free image prior - dark channel prior and a
common haze imaging model, for a single input image, we
can estimate the global atmospheric light and extract the
scene objects transmission. To prevent artifacts, we refine the
transmission using a cross-bilateral filter, and finally the haze-
Figure 1. Single image dehazing using our approach. Left: input haze
free frame can be restored by inversing the haze imaging
image. Right: image after haze removal by our approach.
model. The whole process is highly parallelized, and can be
easily implemented on modern GPUs to achieve real-time
performance. Comparing with existing methods, our approach
provides similar or better results with much less processing Therefore, many methods have been proposed by using
time. The proposed method can be further used for many multiple images or additional information. In [2], Schechner
applications such as outdoor surveillance, remote sensing, and
intelligent vehicles. In addition, rough depth information of the
et al. use two images taken from the same scene with
scene can be obtained as a by-product. different degrees of polarization. This was done by rotating
the polarizing filter attached to the camera. This method is
Keywords-dehaze; dark channel prior; bilateral filter; GPU;
equipment required and may be invalid when the scene is
I. I NTRODUCTION dynamic. In [3], more constraints are obtained from multiple
Outdoor images or videos taken in bad weather always images of the same scene under different weather condi-
have poor visibility. As the light is scattered and absorbed tions. This approach is effective but impractical, because
by the aerosols in the atmosphere before it reaches observer, the requirements cannot always be satisfied and it is also
and the light come to the camera is also blended with light limited to static scenes. In [4], a single image and the
reflected from other directions, called the airlight [1]. This approximated 3D geometrical model of the input scene is
process fades the color and reduces the contrast of observed required, this method uses only one image, but the demand
objects, such degraded images often lack visual vividness of 3D geometrical model makes it inflexible. In addition, the
and appeal. work of Narasimhan et al. [5] needs user interactions.
Virtually all computer vision tasks or computational pho- Recently, dehazing based on a single input image has
tography algorithms assume that the input images are taken made significant progress. The success of these methods lies
in clear weather. Unfortunately, this is not always true in on using a stronger prior or assumption.
many situations, therefore dehazing is highly desired. For In [6], Fattal separates the scene radiance into surface
these applications , removing haze for the input images will albedo and shading, and then extract the medium transmis-
be a useful pre-processing. Furthermore, depth information sion under the assumption that the transmission and surface
of the scene can be obtained from the haze and benefit other shading are locally uncorrelated. This approach is physically
applications. sound and can produce high quality results, but may be
However, dehazing is not an easy job because the degra- invalid when the haze is thick or the assumption is broken.
dation is spatial-variant, it depends on the unknown scene In [7], Kratz uses a factorial Markov random field [8]
depth. General contrast enhancement techniques such as lin- to model the haze image, and takes scene albedo and
ear mapping, histogram equalization, and gamma correction, scene depth as two statistically independent components,
depend only on pixels values and ignore the spatial relations. he removes the haze by factorizing the image into scene
They are limited to remove spatial-variant haze effect as they albedo and depth, but the results are prone to halo and color
do the same operation to each pixel. distortion.

978-0-7695-4205-8/10 $26.00 © 2010 IEEE 62

DOI 10.1109/PacificGraphics.2010.16
 Figure 2. The framework of our method proposed in this paper. Characters in red show the main function performed at the corresponding step.

Tan [9] removes the haze by maximizing the local contrast tion. In section 4, we show several experimental results on
of the restored image based on the observations that images both image and video, a comparison with a few other state-
with enhanced visibility have more contrast than images of-the-art methods is also contained. Finally, in section 5,
plagued by bad weather and the airlight tends to be smooth. we summarize and discuss our approach.
The results have more visibility, but tend to be over saturated
and may not be physically valid. II. BACKGROUND
He et al. [10] presented a novel image prior - dark channel The formation of a haze image is contributed by two
prior based on the statistics of outdoor haze-free images. terms. The first term is the direct attenuation, scene light
Using this image prior, the transmission can be inferred passing through the scattering medium is absorbed or
easily, to smooth the transmission, a soft matting algorithm scattered to other directions. The attenuation depends on
is applied, this approach is more effective and can also well medium and scene depth. The second term is the airlight [1],
handle heavy haze images. which is contributed by light scattered from other directions.
However, a common drawback of these four methods So the optical model can be described as follows [6] [10]:
above is their speed. In the work of [11], a fast algorithm for 𝐼(𝑥) = 𝑡(𝑥) ∗ 𝐽(𝑥) + (1 − 𝑡(𝑥)) ∗ 𝐴. (1)
visibility restoration from a single image is proposed, they
solve the ambiguity between the presence of fog and the Here 𝑥 indicates the position of a pixel, 𝐼 is the observed
objects with low color saturation by assuming that only small haze image, 𝐽 is the scene radiance which is also the haze-
objects can have colors with low saturation. This algorithm free image that we want to restore, 𝐴 is the global atmo-
is much faster, and has several parameters to control the spheric light, and 𝑡 is the medium transmission describing
restoration process, but median or large objects with similar the portion of the light that is not scattered and reaches the
color to the haze may be restored incorrectly. camera. The transmission has a scalar value (0-1) for each
In this paper, we propose an improved method for real- pixel, and the value indicates the depth information of the
time image and video dehazing based on the work of [10]. scene objects directly. This model can present both color
As we will show, our method produce similar or better and gray images. Assume the medium is uniform, we have
results while the processing time is much less. This paper the transmission expressed as:
is organized as follows. In section 2, we describe the haze 𝑡(𝑥) = 𝑒−𝛽𝑑(𝑥) . (2)
imaging model. In section 3, the framework of our method
is detailed, consist of transmission extraction, transmission where 𝛽 is the scattering coefficient of the medium. It
refinement, airlight estimation, and haze-free image restora- indicates that the scene radiance is attenuated exponentially

63
with the scene depth 𝑑. When 𝑡 is estimated from other way, So we can extract the transmission simply by:
it can be a useful cue for the leverage of scene depth. ( ( 𝑐 ))
The goal of dehazing is to restore 𝐽 from the input 𝐼, ˜ 𝐼 (𝑦)
𝑡(𝑥) = 1 − min min . (7)
hence we should estimate 𝐴 and 𝑡 first, and 𝐽 can be 𝑐∈{𝑟,𝑔,𝑏} 𝑦∈Ω(𝑥) 𝐴𝑐
obtained by just inversing the optical model. We can also
explain the loss of visibility in haze image due to averaging In the sky regions, the color is usually very similar to the
the image a constant color 𝐴 based on the formation model. airlight in a haze image, since the sky is at infinite and tends
For a patch with uniform transmission 𝑡, the visibility (sum to has zero transmission, the Equation (7) well handles both
of gradient) of the input image is reduced by the haze, since sky regions and non-sky regions.
𝑡 < 1: In our implementation, dark channel fetch in Equation (7)
∑ ∑ is done in two steps, first we do the min operation among
∥∇𝐼(𝑥)∥ = ∥𝑡∇𝐽(𝑥) + (1 − 𝑡)∇𝐴∥ three color channels, each pixel needs 2 comparisons. Then
𝑥 𝑥
∑ ∑ we do the min operation among local patch using the fast
= 𝑡 ∥∇𝐽(𝑥)∥ < ∥∇𝐽(𝑥)∥. (3) algorithm of [12]. This algorithm separates the searching
𝑥 𝑥 process into two passes of row and column search. In each
III. S INGLE IMAGE DEHAZING pass, using two buffers, it takes only 3 comparisons per pixel
independent of patch size, so the whole process only need
Our dehazing framework is shown in Fig. 2. In this
8 comparisons per pixel independent of patch size. Thus the
section, we describe our method in detail. Fist we estimate
complexity of dark channel fetch is linear to image size. In
the rough transmission, then we smooth the transmission
our experiment, the patch size is set to 11 × 11.
using a cross-bilateral filter, and finally restore the haze-free
image, a way to estimate the airlight is also contained. As we see from Fig. 3, the extracted transmission map
is a bit rough, and contains some block effects since the
A. Extract the Transmission using Dark Channel Prior transmission is not always constant in a patch, so we need
Assume the airlight is already known, to recover the haze- to smooth it. In the next subsection, we refine this map using
free image, we should extract the transmission map first. a cross-bilateral filter.
Here we use the novel image prior - dark channel prior
which was presented in the work of He et al. [10]. Dark
channel prior is based on the following observation on haze-
free outdoor images: in most of the non-sky patches, at least
one color channel has very low intensity at some pixels. In
other words, the minimum intensity in such a patch should
has a very low value. Formally, for an image 𝐽, the dark
channel value of a pixel 𝑥 is defined as:
( )
𝑑𝑎𝑟𝑘 𝑐
𝐽 (𝑥) = min min (𝐽 (𝑦)) . (4)
𝑐∈{𝑟,𝑔,𝑏} 𝑦∈Ω(𝑥)

Here 𝐽 𝑐 is a color channel of 𝐽, and Ω(𝑥) is a patch around

𝑥. As the dark channel value of a haze-free image should be
very low, we just take it as zero here. We further assume that
the transmission in a local patch is constant. Take the min
operation to both the patch and three color channels, then
make some simple transforms to the haze imaging model,
denote the patch transmission as 𝑡˜(𝑥), we have:
( ( 𝑐 ))
𝐼 (𝑦)
min min
𝑐∈{𝑟,𝑔,𝑏} 𝑦∈Ω(𝑥) 𝐴𝑐
( ( 𝑐 ))
𝐽 (𝑦)
= 𝑡˜(𝑥) min min + (1 − 𝑡˜(𝑥)).(5)
𝑐∈{𝑟,𝑔,𝑏} 𝑦∈Ω(𝑥) 𝐴𝑐
According to the dark channel prior, the dark channel value
of a haze-free image 𝐽 tends to be zero, as A is always Figure 3. Transmission refinement and image restoration. Top: input
haze image(left) and the rough transmission(right). Middle: refined trans-
positive, this leads to: mission(right) and restored image(left) by [10]. Bottom: refined transmis-
( ( 𝑐 ))
𝐽 (𝑦) sion(right) and restored image(left) by our approach.
min min → 0. (6)
𝑐∈{𝑟,𝑔,𝑏} 𝑦∈Ω(𝑥) 𝐴𝑐

64
 the sum weight of the local patch centered by pixel 𝑥:
∑
𝑤(𝑥) = 𝐺𝜎𝑠 (∥𝑥 − 𝑦∥) 𝐺𝜎𝑟 (∣𝐸𝑥 − 𝐸𝑦 ∣) . (9)
𝑦∈Ω(𝑥)

A brute force implementation of bilateral filter is also

computational cost. Fortunately, there are many methods
for the acceleration of bilateral filter as in [18] [19] [20]
[21]. Here we choose Paris and Durand’s signal processing
method [19], in that work they recast the computation as a
higher-dimensional linear convolution followed by trilinear
interpolation and a division. We implement this method on
GPU, it can achieve real-time performance easily with quite
good results. We use 𝜎𝑠 = 16 and 𝜎𝑟 = 0.1 for all the
results shown in this paper.
As shown in Fig. 3, the smoothed transmission is consis-
tent with the boundary of scene objects and preserves the
sharp depth discontinuities.
C. Recover the Haze-free Image
After the transmission map is refined, we can recover the
Figure 4. An example of image dehazing. Top: input haze image. Bottom: haze-free image 𝐽 simply by solving the inverse of Equation
result of our method. (1). When the transmission value is too small, the directly
recovered haze-free image tend to be noisy in some regions
accordingly. Like in [10], here we restrict the transmission
B. Refine the Transmission using Cross-Bilateral Filter to a low bound 𝑡0 , the final haze-free image 𝐽 is recovered
In Fattal’s work [6], only parts of pixels which meet some by:
𝐼(𝑥) − 𝐴
error estimation criteria are selected for computing, then he 𝐽(𝑥) = + 𝐴. (10)
uses an Gauss-Markov random field model [13] guided by max(𝑡(𝑥), 𝑡0 )
the input color image to extrapolate the solution to the whole The value of 𝑡0 is set to 0.05-0.1 in our experiment. The
image. However, this method is computationally intensive. image after haze removal sometimes looks dim, and has a
In [10], He et al. refine the transmission using a soft matting higher dynamic. It can be further enhanced by some normal
method [14]. This approach can produce a quite exquisite image enhancement techniques such as linear mapping,
transmission map and is a little faster, but still a big gap histogram equalization, and gamma correction.
from real-time applications. As we can see from these two
D. Estimate the Airlight
methods, the refined transmission map is smooth everywhere
except at sharp depth discontinuities, here we use an edge- There are many ways to estimate the airlight. A typical
preserving filter to accelerate the refine process. way is to estimate from the most haze-opaque pixels. In
A bilateral filter [15] is a nonlinear filter that smooths the Tan’s work [9], the value of a pixel with highest intensity
images except at strong edges. To capture the sharp edge is chosen as airlight, under the assumption that these pixels
discontinuities and outline the profile of the objects while represent an object at infinite distant. He et al. [10] take
removing the block effects, the original input image should pixels with brightest dark channel values as the most haze-
be taken as a guide. Thus we smooth the transmission using opaque pixels, then among these pixels, the one with highest
a cross-bilateral filter [16] [17], the refined transmission 𝑡(𝑥) intensity can be selected. Fattal [6] computes the airlight
can be obtained by a local mean: by solving an optimization problem. Tarel [11] just sets the
airlight to (1,1,1) after making a white balance to the haze
image. In our paper, for simplicity, we just take the value
1 ∑ of a pixel with highest dark channel value as the airlight.
𝑡(𝑥) = 𝐺𝜎𝑠 (∥𝑥 − 𝑦∥) 𝐺𝜎𝑟 (∣𝐸𝑥 − 𝐸𝑦 ∣) 𝑡˜(𝑥).
𝑤(𝑥)
𝑦∈Ω(𝑥) IV. E XPERIMENTAL RESULTS
(8)
We have implemented the proposed method in two ver-
Here 𝐺 is the common Gaussian function, 𝜎𝑠 controls the
sions, a GPU-based version using OpenCV(for the I/O of
spatial weight and 𝜎𝑟 controls the range weight. The range
images and videos) and NVIDIA CUDA(Compute Unified
weight guided by the intensity 𝐸 of the input image, prevents
Device Architecture), and a CPU version using C++. Both
pixels on one side of a strong edge from influencing pixels
versions were tested on a laptop with a Intel Pentium Dual
on the other side since they have different values. 𝑤(𝑥) is

65
 Figure 5. Comparison with Fattal’s work [6]. Left: input haze image. Middle: Fattal’s result. Right: our result.

Figure 6. Comparison with Tan’s work [9]. Left: input haze image. Middle: Tan’s result. Right: our result.

Figure 7. Comparison with the work of He et al. [10]. Left: input haze image. Middle: result by He et al.. Right: result by our approach.

CPU T3200(2.00 GHz) and a NVIDIA GeForce 9300M Fig. 5 and Fig. 8 show the comparison with Fattal’s work
GS(with only one multiprocessor). The processing time [6]. As we can see, our result outperforms Fattal’s. The
mainly depends on the image size. colors of our result look more natural. Fattal’s method is
For an image with size 600 × 400, the GPU-version can based on statistics and requires sufficient color information
process about 10-12 frames per second, and the processing and variance. If the haze is dense, or the object color is
time of the CPU-version is about 0.5 seconds, with airlight similar to haze, the signal-to-noise ratio is not high enough
estimation 0.08s, transmission extraction 0.1s, refinement for his method to reliably estimate the transmission. Notice
0.3s and restoration 0.06s. While Tan’s work [9] needs the roof on the right margin in Fig. 5, the albedo is almost
about five to seven minutes, Fattal’s work [6] takes about the same direction with the airlight, this causes an overshoot
35 seconds, and He et al.’s work [10] takes 10-20 seconds. of estimated transmission for Fattal’s approach. Our method
Fig. 1 and Fig. 4 show our dehazing results for images. can well handle regions like this.
Depth map in Fig. 2 is computed by inversing Equation(2)
and is further normalized. As we see, our approach can In Fig. 6, we compare our approach with Tan’s work [9].
remove the haze effectively while introducing little artifacts. The colors of Tan’s result are over saturated, and the whole

66
Figure 8. Comparison with other works. From left to right are: the input haze images, the results of He et al. [10], Fattal’s results [6], and our results.

restored image looks unrealistic since his algorithm is not considered to have the same value for each color channel
physically based. Although our result has less local contrast, in our haze imaging model, but this may be invalid as the
it looks more realistic and the color keeps consistent with scattering coefficient is affected by wavelength, so there
the original one. Moreover, there are some halo artifacts in should be difference between color channels. Dehazing with
Tan’s result while no obvious one in ours. differential treatment to color channels will also be our
We also compare our work with that of He et al. [10]. future work.
As we show in Fig. 3, our refined transmission looks a
little coarse. Although the sharp depth discontinuities are
both preserved, the very fine scale details are smoothed by
a cross-bilateral filter while can be saved by a soft matting
algorithm. However, our approach is much faster and in most
instances, the recovered results are similar or close to the
results of He et al. as shown in Fig. 3, Fig. 7, and Fig. 8.
An example of video dehazing is shown in Fig. 9. For
a video file with resolution 600 × 400, our approach can
process about 6-9 frames per second plus the decoding cost.
The airlight can be estimated from the first several frames
and be used for the whole process.

V. C ONCLUSIONS
In this paper, we have proposed a simple but effective
method for real-time image and video dehazing. Using a
newly presented image prior - dark channel prior, we can
Figure 10. Failure case. Top: input haze image with a lot of noise. Bottom:
easily estimate the airlight and extract the transmission, noise is amplified in our result.
then using a cross-bilateral filter, we can further refine the
transmission. Finally we can restore a quite good haze-free
image in real-time. Furthermore, we can also get the rough
depth information of the scene. R EFERENCES
In most cases, our approach can produce quite good
results. However, when the input image contains too many
[1] H. Koschmieder, “Theorie der horizontalen sichtweite,”
noises, the airlight will be estimated in error and the error Beitr.Phys.Freien Atm., vol. 12, pp. 171–181, 1924.
propagates to the whole processing, thereby the recovered
image will suffer the amplified noises as shown in Fig. 10. [2] Y. Y. Schechner, S. G. Narasimhan, and S. K. Nayar, “Instant
Our next work is to find a fast and effective method for noise dehazing of images using polarization,” IEEE Conference on
suppression during dehazing. Moreover, the transmission is Computer Vision and Pattern Recognition, pp. 1–325, 2001.

67
Figure 9. An example of video dehazing. The first and third rows are the input haze frames, the second and fourth rows are corresponding output haze-free
frames.

[3] S. G. Narasimhan and S. K. Nayar, “Contrast restoration [9] R. Tan, “Visibility in bad weather from a single image,” IEEE
of weather degraded images,” IEEE Transactions on Pattern Conference on Computer Vision and Pattern Recognition,
Analysis and Machine Intelligence, vol. 25, pp. 713–724, 2008.
2003.
[10] K. He, J. Sun, and X. Tang, “Single image haze removal using
[4] J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, dark channel prior,” IEEE Conference on Computer Vision
O. Deussen, M. Uyttendaele, and D. Lischinski, “Deep photo: and Pattern Recognition, 2009.
Model-based photograph enhancement and viewing,” SIG-
GRAPH Asia, 2008.
[11] J.-P. Tarel and N. Hautiére, “Fast visibility restoration from a
single color or gray level image,” International Comference
[5] S. Narasimhan and S. Nayar, “Interactive deweathering of on Computer Vision, pp. 2201–2208, 2009.
an image using physical models,” Workshop on Color and
Photometric Methods in Computer Vision, 2003.
[12] M. van Herk, “A fast algorithm for local minimum and max-
[6] R. Fattal, “Single image dehazing,” SIGGRAPH, pp. 1–9, imum filters on rectangular and octagonal kernels,” Pattern
2008. Recogn. Lett., vol. 13, pp. 517–521, 1992.

[7] L. Kratz and K. Nishino, “Factorizing scene albedo and depth [13] P. Perez, “Markov random fields and images,” CWI Quarterly,
from a single foggy image,” International Comference on vol. 11, pp. 413–437, 1998.
Computer Vision, 2009.
[14] A. Levin, D. Lischinski, and Y. Weiss, “A closed form
[8] J. Kim and R. Zabih, “Factorial markov random fields,” solution to natural image matting,” IEEE Conference on
Europeon Conference on Computer Vision, pp. 321–334, Computer Vision and Pattern Recognition, vol. 1, pp. 61–68,
2002. 2006.

68
 [15] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and
color images,” International Comference on Computer Vision,
pp. 839–846, 1998.

[16] E. Eisemann and F. Durand, “Flash photography enhancement

via intrinsic relighting,” SIGGRAPH, 2004.

[17] G. Petschnigg, M. Agrawala, H. Hoppe, R. Szeliski, M. Co-

hen, and K. Toyama, “Digital photography with flash and
no-flash image pairs,” SIGGRAPH, 2004.

[18] M. Elad, “On the bilateral filter and ways to improve it,” IEEE
Transactions On Image Processing, vol. 11, pp. 1141–1151,
2002.

[19] S. Paris and F. Durand, “A fast approximation of the bilateral

filter using a signal processing approach,” Europeon Confer-
ence on Computer Vision, 2006.

[20] J. Chen, S. Paris, and F. Durand, “Real-time edge-aware

image processing with the bilateral grid,” SIGGRAPH, 2007.

[21] Q. Yang, K. H. Tan, and N. Ahuja, “Real-time o(1) bilateral

filtering,” IEEE Conference on Computer Vision and Pattern
Recognition, 2009.

69

View publication stats