IEEE SIGNAL PROCESSING LETTERS, VOL.

27, 2020 1475

Min-Max Average Pooling Based Filter for Impulse

Noise Removal
Piyush Satti , Nikhil Sharma , and Bharat Garg

Abstract—Image corruption is a common phenomenon which determine pixel intensity, but this method is susceptible to
occurs due to electromagnetic interference, and electric signal large processing time. Moreover, weighted mean filters like
instabilities in a system. In this letter, a novel multi procedure Adaptive Switching Weighted Median Filter (ASWMF) [2] and
Min-Max Average Pooling based Filter is proposed for removal
of salt, and pepper noise that betide during transmission. The
Three-Value Weighted Approach (TVWA) [3] use weighted
first procedure functions as a pre-processing step that activates mean methods. Dynamic Adaptive Median Filter (DAMF) [4],
for images with low noise corruption. In latter procedure, the noisy is based on iterative techniques, however it showcases degraded
image is divided into two instances, and passed through multiple performance in high noise density images. Apart from these,
layers of max, and min pooling which allow restoration of intensity Fast Switching Based Median-Mean Filter (FSBMMF) [5],
transitions in an image. The final procedure recombines the parallel Switching Median and Morphological Filter (SMMF) [6] and
processed images from the previous procedures, and performs aver- Unbiased Weighted Mean Filter (UWMF) [7] have also been
age pooling to remove all residual noise. Experimental results were
obtained using MATLAB software, and show that the proposed
considered for a comprehensive comparative analysis.
filter significantly improves edges over exiting literature. Moreover, Moreover, the advent of advanced methods inspired by ar-
Peak Signal to Noise Ratio was improved by 1.2 dB in de-noising tificial intelligence, machine and deep learning, remarkable
of medical images corrupted by medium to high noise densities. improvements have been noted in image restoration processes.
Classification algorithms such as support vector machines [8]
Index Terms—Mean filters, median filters, salt and pepper noise,
pooling, image restoration and de-noising. and fully connected neural networks have been used to learn
embedded patterns and similarity in images, this aids in restora-
tion process with noteworthy results. The concept of Densely
I. INTRODUCTION connected Network for Impulse Noise Removal (DNINR) [9]
MPULSE noise, popularly known as Salt and Pepper (SAP) were also utilized. In [10], a novel two step framework is
I noise is introduced during acquisition and transmission phase
of an image. It is defined as a sharp or sudden disturbance in
proposed where Generative Adversarial Network (GAN) and
Convolution Neural Network (CNN) based blind de-noiser is
input signal due to which the image pixel attain its extreme used. In [11], a novel end-to-end architecture has been proposed
intensity values. In the last three decades, several de-noising which directly generates the de-noised image. Another CNN
filters were proposed to eliminate SAP noise using linear and based approach is adopted by [12] in which five noise level
non-linear filtering techniques. The mean or median value of the CNN prior de-noisers are used. A fusion algorithm based on
3 × 3 window was used to replace the corrupted intensity. This guided filtering is used to combine the images to obtain the noise
methodology observed limited application in systems involving suppressed image. Although, the results for low noise densities
medium to high noise densities due to poor filter performance. are impressive, these method observes catastrophic failure for
Improvements in filtering methods that relied on combination higher noise densities.
and decision based techniques were introduced. These filters In this letter, CNN inspired pooling methods have been ef-
used a combination of mean and median which allowed better fectively modified and used to achieve improved results, even in
quality in medium noise densities. However, output images very high noise densities. Major contributions of the proposed
were observed to have high blurring effects. Further improve- work are as follows:
ments were made using mathematical models like interpolation,
r Exploitation of transition in intensity values along edges of
weighted mean and probabilistic methods which boasted supe- an image for utilization in impulse noise removal, a notable
rior performance especially in high noise density images. advantage is the difference in edge intensity level on using
Recursive cubic Spline Interpolation Filter (RSIF) [1] is a min vs max pooling as first layer.
method in which interpolation of a 3 × 3 window is used to
r Creation and utilization of two copies of a noisy image with
different pooling layer arrangements. Recombination of
Manuscript received June 25, 2020; revised July 31, 2020; accepted August 4, the different de-noised images for improved edge boundary
2020. Date of publication August 17, 2020; date of current version September approximations.
2, 2020. The associate editor coordinating the review of this manuscript and The rest of the paper is organized as follows. Section II
approving it for publication was Dr. Victor Sanchez (Corresponding author:
Bharat Garg.)
presents the proposed work, with explanations for the specific
The authors are with the Department of Electronics and Communication choice of techniques, and the algorithm along with its various
Engineering, Thapar Institute of Engineering and Technology, Patiala 147001, procedures. Section III enumerates the simulation results of the
India (e-mail: psatti_be17@thapar.edu; nsharma1_be17@thapar.edu; bharat. proposed work with comparative analysis. Finally, Section IV
garg@thapar.edu).
Digital Object Identifier 10.1109/LSP.2020.3016868 concludes the letter.

1070-9908 © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Constantin Vertan. Downloaded on October 07,2020 at 11:49:27 UTC from IEEE Xplore. Restrictions apply.
1476 IEEE SIGNAL PROCESSING LETTERS, VOL. 27, 2020

II. PROPOSED WORK

This section presents the proposed method which utilizes
pooling methods for SAP noise removal. SAP has fixed am-
plitude at maximum and minimum pixel intensity value. It’s
noise mask for an 8-bit greyscale image is given by Eq.(1),
where f (i, j) denotes intensity values at pixel (i, j). The noise
mask stores and represents the location of corrupted pixels in an
image, the value 0 identifies the pixel as a noisy pixel whereas
1 represents a noise free pixel. Noise density (Nd ) of an image
can also be calculated from the noisemask using Eq.(2), where
M and N are the dimensions of the image.

0 for allf (i, j) ∈ {0, 255}
noisemask (i, j) = (1)
1 otherwise
M  N Fig. 1. Block diagram of the different procedures in the proposed work.
i=1 j=1 noisemask (i, j)
Nd = (2)
M ×N
These copies are processed simultaneously using the CMMP
Pooling techniques are an approach used in deep learning procedure. Both images are rippled through the pooling layers
based CNN which down samples input image into feature maps. in a pre-determined order, this step gives two output images.
This makes the image local translation invariant. Max, min and Finally, both images are recombined using the R&S procedure
average pooling are the techniques used in the proposed scheme. resulting in the noise free image.
In these techniques, pooling size and stride are two important
parameters which determine quality and size of output image. It A. Intensity Estimation for pixels with High Correlation for
is ideal to choose pooling size as 3 (window size 3 × 3) due to Lower Noise Densities
high correlation between neighbouring pixel values and stride
as 1 to preserves image dimensionality. This procedure activates when the noise density is below 45%.
Further investigations have shown that a single pooling layer If this condition is satisfied, we first calculate the window infor-
is insufficient for complete noise removal. It has been observed mation threshold (α) using α = Nd /0.1. Subsequently, for each
that, for a fixed 3 × 3 window size, a minimum of 4 pooling noisy pixel in the image, we check its 3 × 3 noise free window
nf
layers are required to remove more than 99.8% of noisy pixels, (W3×3 ) and store it as a list of pixel values in variable Wc . The
even in images corrupted by high noise densities (> 85%). number of elements present in the list is given by length(Wc ) and
Testing various combinations, there is a distinct improvement also represents the number of uncorrupted pixels. If the number
in using either max-min-min-max or its complementary min- of uncorrupted pixels are above threshold α, we replace the
max-max-min pooling layer arrangement. This is attributed to corrupted pixel with its median value, otherwise no processing is
two factors: differences between the nature of max from min done. This allows more accurate approximation of noisy pixels
pooling and the amount of pixels processed by each layer. due to its high correlation with its surroundings.
Max pooling selects the uncorrupted brighter pixels from the
window which is useful when the background of an image is B. Complementary Min-Max Pooling
dark. This also enhances lighter edges amongst dark pixels. This procedure takes two noisy images as input along with a
Min pooling selects the darker intensity values and works in the string that decides the mode of pooling. The mode of pooling is
opposite way. Therefore, depending upon the image and pixel governed by the Main algorithm. Taking the two noisy images
being processed, one technique is better suited than the other. as input, for each noisy pixel in the image, we consider a 3 × 3
The percentage contribution of each layer in noise removal falls noise free window (W3×3 nf
). If the noise free window is empty,
exponentially with the first layer having the highest contribution. the pixel is not processed. Otherwise depending upon the string
These two factors reinforce the need for parallel processing of input, we either take the max-min or min-max pooling of the
an image by both the max-min-min-max and min-max-max-min noise free window as replacement pixels for the first and second
arrangements. The image de-noising algorithm has been broken noisy image, respectively. Finally, the two processed images are
down into three sub-procedures. Namely, Intensity Estimation given as output. This step is useful especially in images with
for pixels with High Correlation for Lower Noise Densities frequent low to high and high to low intensity transitions.
(IEHCLND), Complementary Min-Max Pooling (CMMP) and a
final Recombination & Smoothing step (R&S). A block-diagram
has also been shown in Fig. 1, which encapsulates the proposed C. Recombination & Smoothing
scheme. This procedure takes two images. In this procedure, the first
Let us discuss the main algorithm. Based on noise density operation is the pixel-wise recombination of the two images by
of the input image, the IEHCLND procedure is called which calculation of the average intensity value. Finally, a running
removes noisy pixels that can be estimated using surrounding average pooling is performed for all corrupted pixels in the
intensity values. However, if the noise density is below the original noisy image. This results in the final noise free image. It
threshold defined, no processing is carried out. Next, two copies is worth noting that the pixel-wise weighted average is not ideal
of the output image from the previous procedure are created. for recombination as it degrades performance significantly.

Authorized licensed use limited to: Constantin Vertan. Downloaded on October 07,2020 at 11:49:27 UTC from IEEE Xplore. Restrictions apply.
SATTI et al.: MIN-MAX AVERAGE POOLING BASED FILTER FOR IMPULSE NOISE REMOVAL 1477

TABLE I
Algorithm 1: MMAPF(nImg). AVERAGE PSNR VALUES OF 24 GRAYSCALE RESTORED IMAGES FROM
1: Input nImg  Input noisy image KODAK BENCHMARK IMAGE DATASET WITH VARYING NOISE DENSITY
FROM 50% TO 90%
2: Output OutImg  Restored Image
3: if Nd < 0.45 then
4: IIEHCLN D ← IEHCLND(nImg)
5: else
6: IIEHCLN D ← nImg
7: end if
8: Initialize: I1 ← IIEHCLN D , I2 ← IIEHCLN D
9: Initialize: Layers ← [ M ax , M in , M in , M ax ]
10: for each Layer in Layers do  Defines Pooling
Layers
11: [I1 , I2 ] ← CMMP(I1 , I2 , Layer) III. SIMULATION RESULTS
12: end for
13: OutImg ← R&S(I1 , I2 , nImg)  Recombination of This section illustrates the performance of the proposed filter
images over existing algorithms. Kodak benchmark image dataset con-
14: return OutImg taining 24 greyscale images is used for comparing the efficiency
of the algorithms. The dimensions of the images are 512 × 768
or 768 × 512. A special case, consisting of a coloured X-ray
image (Lungs.png) of size 411 × 419 is also considered to check
1: procedure IEHCLNDnImg, Nd the productivity of the proposed algorithm in medical imaging.
2: α = f loor( N
0.1 );
d
 Information Threshold The noise density is varied from 10% to 95%. The results
3: for each Pi,j in nImg do are verified using quantitative (plots and tables) and qualitative
nf
4: Wc ← W3×3  3 × 3 noise free window (visual representation) measures. The Peak Signal to Noise Ratio
5: if length(Wc ) > α then (PSNR) is calculated to substantiate the experimental results.
6: oImgi,j ← median(Wc ) The PSNR is defined as the ratio between the maximum possible
7: else power of the signal and the power of distorting noise. Mathemat-
8: oImgi,j ← Pi,j ically it is given by Eq.(3) where, Max is 255 for 8-bit greyscale
9: end if image. MSE is mean square error given by Eq.(4) where, M and
10: end for N are dimensions of the image, xi,j and yi,j represent the pixels
11: return oImg in the original and restored image respectively.
12: end procedure
P SN R = 10log10 (M AX 2 /M SE) (3)

1 M  N
1: procedureCMMPI1 , I2 , str M SE = (xi,j − yi,j )2 (4)
2: Input noisy images and Layer M ∗ N i=1 j=1
3: for each Pi,j in I1 do
nf
The parameters used in existing algorithms are tuned as men-
4: Wc ← W3×3  3 × 3 noise free window tioned by respective authors in their research. The further sub-
5: if length(Wc ) > 0 then sections include simulation on Kodak benchmark image dataset
6: if str =  M ax then and then on coloured X-ray image.
7: O1 i, j ← max(Wc ), O2 i, j ← min(Wc )
8: else A. Experimental Results on Kodak Benchmark Image Dataset
9: O1 i, j ← min(Wc ), O2 i, j ← max(Wc )
10: end if In Table I, the average values of PSNR for the 24 grayscale
11: end if images are shown with varying noise densities from 50% to 95%,
12: end for and the results are plotted in Fig. 2. The proposed filter has an
13: return O1 , O2 average improvement of around 0.85 dB over the best existing
14: end procedure filter. At higher noise densities, the proposed filter performs
exceptionally well with around 1.2 dB improvement. From the
plot, we can also infer that the proposed filter has stable and
consistent performance throughout the noise densities. Fig. 4(a),
1: procedureR&SI1 , I2 , nImg
shows one such image from the Kodak dataset. The image is
2: Initialize: oImg ← (I1 + I2 )/2  Recombination
nf corrupted with 90% impulse noise. We can observe that Fig. 4(b)
3: for each Pi,j in nImg do has significant quality degradation, Fig. 4(c) and Fig. 4(d) are
nf
4: oImgi,j ← mean(oImg3×3 )  Average Pooling not able to restore the image details due to the streaking effect.
5: end for Fig. 4(g) and Fig. 4(h) are producing blurred restored image.
6: return oImg Fig. 4(i) shows the restored image for the proposed filter. Here,
7: end procedure edges of the image are preserved with relatively less blurring
and low streaking effect.

Authorized licensed use limited to: Constantin Vertan. Downloaded on October 07,2020 at 11:49:27 UTC from IEEE Xplore. Restrictions apply.
1478 IEEE SIGNAL PROCESSING LETTERS, VOL. 27, 2020

TABLE II
PSNR VALUES FOR DIFFERENT FILTERS ON MEDICAL IMAGE (LUNGS) FOR
VARYING NOISE DENSITY FROM 10% TO 90%

Fig. 2. Average PSNR values for Kodak Benchmark Dataset with noise density
from 50% to 95%.

Fig. 3. PSNR values for Lungs.png with noise density from 10% to 90%

Fig. 5. (a) Original Lungs image, which was filtered after 95% corruption
using: (b) ASWMF, (c) FSBMMF, (d) RSIF, (e) DAMF, (f) TVWA, (g) SMMF,
(h) UWMF, and (i) Proposed Filter

1.2 dB for medium-high noise densities. Same can be concluded

from the plot where the proposed filter has an better performance
and margin over other existing filters. Fig. 5. shows the restored
images for various filters when corrupted with 95% impulse
noise. In Fig. 5 (b), (e), (g) and (h) we can see patches of irregular
colours throughout the image. Fig. 5(c) and Fig. 5(d) are not able
to restore the structure of the image due to blurring and streaking
effects. Fig. 5(i) has the best restored results with no streaking
and low blurring.

IV. CONCLUSION
Fig. 4. (a) Original Kodak image chosen from Kodak dataset, which was In this letter, a novel Min-Max Average pooling based Filter
filtered after 91% corruption using: (b) ASWMF, (c) FSBMMF, (d) RSIF, (e) is proposed for removal of salt and pepper noise. The proposed
DAMF, (f) TVWA, (g) SMMF, (h) UWMF, and (i) Proposed Filter algorithm is divided into three procedures. The first procedure
is used to improve the performance in images with lesser cor-
ruption. The second procedure splits the image into two copies
B. Experimental results on Coloured X-ray image and utilizes min-max pooling with different layer arrangements
Simulation was also performed on Lungs.png with varying to capture the transitions (Bright to dark and vice-versa) in
noise densities from 10% to 90%. PSNR values are calculated the image. The last stage performs recombination followed by
for each algorithm and the results are shown in Table II. The average pooling operations to obtain finer edge and boundary
corresponding plot is mapped in Fig. 3. From the values, we details. Simulation results show that the proposed algorithm pro-
can deduce that even for medical images the proposed filter out- vides significantly improved results as compared to established
performs other existing filters with an average improvement of literature.

Authorized licensed use limited to: Constantin Vertan. Downloaded on October 07,2020 at 11:49:27 UTC from IEEE Xplore. Restrictions apply.
SATTI et al.: MIN-MAX AVERAGE POOLING BASED FILTER FOR IMPULSE NOISE REMOVAL 1479

REFERENCES [7] C. Kandemir, C. Kalyoncu, and Ö. Toygar, “A weighted mean filter with
spatial-bias elimination for impulse noise removal,” Digit. Signal Process.,
[1] T. Veerakumar, S. Esakkirajan, and I. Vennila, “Recursive cubic spline in- vol. 46, pp. 164–174, 2015.
terpolation filter approach for the removal of high density salt-and-pepper [8] A. Roy, J. Singha, S. S. Devi, and R. H. Laskar, “Impulse noise removal
noise,” Signal, Image Video Process., vol. 8, no. 1, pp. 159–168, 2014. using SVM classification based fuzzy filter from gray scale images,” Signal
[2] S. Akkoul, R. Ledee, R. Leconge, and R. Harba, “A new adaptive switching Process., vol. 128, pp. 262–273, 2016.
median filter,” IEEE Signal Process. Lett., vol. 17, no. 6, pp. 587–590, [9] G. Li, X. Xu, M. Zhang, and Q. Liu, “Densely connected network for
Jun. 2010. impulse noise removal,” Pattern Anal. Appl., vol. 23, pp. 1–13, 2020.
[3] C.-T. Lu, Y.-Y. Chen, L.-L. Wang, and C.-F. Chang, “Removal of salt-and- [10] J. Chen, J. Chen, H. Chao, and M. Yang, “Image blind denoising with
pepper noise in corrupted image using three-values-weighted approach generative adversarial network based noise modeling,” in Proc. IEEE Conf.
with variable-size window,” Pattern Recognit. Lett., vol. 80, pp. 188–199, Comput. Vision Pattern Recognit., 2018, pp. 3155–3164.
2016. [11] S. Zhu, G. Xu, Y. Cheng, X. Han, and Z. Wang, “BDGAN: Image blind
[4] P. Patel, B. Majhi, B. Jena, and C. Tripathy, “Dynamic adaptive median denoising using generative adversarial networks,” in Proc. Chin. Conf.
filter (DAMF) for removal of high density impulse noise,” Int. J. Image, Pattern Recognit. Comput. Vision, 2019, pp. 241–252.
Graph. Signal Process., vol. 4, no. 11, pp. 53–62, 2012. [12] S. Liu, T. Liu, L. Gao, H. Li, Q. Hu, J. Zhao, and C. Wang, “Convolutional
[5] V. Vijaykumar, G. S. Mari, and D. Ebenezer, “Fast switching based neural network and guided filtering for SAR image denoising,” Remote
median–mean filter for high density salt and pepper noise removal,” Sens., vol. 11, no. 6, pp. 1–19, 2019.
AEU-Int. J. Electron. Commun., vol. 68, no. 12, pp. 1145–1155, 2014.
[6] C. Yuan and Y. Li, “Switching median and morphological filter for impulse
noise removal from digital images,” Optik, vol. 126, no. 18, pp. 1598–1601,
2015.

Authorized licensed use limited to: Constantin Vertan. Downloaded on October 07,2020 at 11:49:27 UTC from IEEE Xplore. Restrictions apply.