MULTIMODE FOREGROUND DETECTION USING

SURF ALGORITHM
Abstract—Video surveillance is active research topic in any matcher. For matching and recognition as well as to many
computer vision research area for humans & vehicles, so it is other applications, local features are used as they are robust to
used over a great extent. Multiple images generated using a fixed blockage, background noise and other changes. The challenge
camera contains various objects, which are taken under different
is to obtain invariance to viewing conditions. It can first detect
variations, illumination changes after that the object’s identity
and orientation are provided to the user. This scheme is used to
features and then compute a set of descriptors for these
represent individual images as well as various objects classes in a features. In the case of significant transformations, feature
single, scale and rotation invariant model. The objective is to de
improve object recognition accuracy for surveillance purposes & tection has to be adapted to the transformation, as at least a
to detect multiple objects with sufficient level of scale
subset of the features must be present in both images in order
invariance. Multiple objects detection& recognition is important
in the analysis of video data and higher level security system. This to allow for correspondences. Features which have proved to
method can efficiently detect the objects from query images as be particularly appropriate are interest points.
well as videos by extracting frames one by one. When given a
query image at runtime, by generating the set of query features ORGANIZATION OF THE PAPER
and it will find best match it to other sets within the database. This paper organizes as follows, section II covers the
Using SURF algorithm find the database object with the best methodology of this scheme & using SURF algorithm which
feature matching, then object is present in the query image. extracts the features. Section III covers the result of proposed
methodology which is to be carried out in VISUAL STUDIO
Keywords— Image recognition, Query image, Local feature,
and Section IV is summarizes this project work in terms of
Surveillance system, SURF algorithm.
conclusion.
I. INTRODUCTION II. METHODOLOGY
An object recognition system finds objects in the real First, for each image in the training set, select a set of
world from an image of the world, using object models which interest points and construct their local feature descriptors
are known a priori. This task is surprisingly difficult. Humans using SURF[3] (Speeded Up Robust Features).Then, by using
perform object recognition effortlessly and instantaneously. statistical analysis, select representative points from the
Algorithmic description of this task for implementation on Interest points. Representative points of an object are interest
machines has been very difficult. The problem of recognizing points that deliver rich and distinguishing information about
multiple object classes in natural images has proven to be a the object for recognition. All interest patch pairs and selected
difficult challenge for computer vision. The object recognition the patches are similar score was higher than some threshold
problem can be defined as a labelling problem based on as representative patches. If an interest point has an enough
models of known objects. Formally, given an image number of similar interest points in terms of the SURF
containing one or more objects of interest (and background) descriptor, consider the interest point to be a representative
and a set of labels corresponding to a set of models known to point. Based on the representative points of the objects,
the system, the system should assign correct labels to regions, calculate a threshold for each object type from the training set.
or a set of regions, in the image. Speeded‐Up framework,
newly‐developed Robust Features (SURF)
which is very likely to[3] is a
becoming
The object recognition problem is closely tied to the the next fact feature detector in the industry. To improve
segmentation problem i.e. without at least a partial recognition performance of object recognition system, there is need to
of objects, segmentation cannot be done, and without address two issues: first, there is need to emphasize the
segmentation, object recognition is not possible. importance of the number of feature pairs when object
In order to recognize the objects from the database, following recognition is done. Since this number is already present in the
steps are carried out: recognition score, one potential improvement would be to
The images in the test set are compared to all images in raise its value to a larger power. Using this algorithm, it can
the reference set by matching their respective interest points. generate a set of feature pairs between the query image and
The object shown on the reference image with the highest each individual database image. For object recognition task,[4]
number of matches with respect to the test image is chosen as SURF algorithm is used because of its powerful attributes,
the recognized object. including scale invariance, translation invariance, lighting
The matching is carried out as follows: An interest point invariance, contrast invariance, and rotation invariance & it
in the test image is compared to an interest point in the can detect objects in images taken under different extrinsic and
reference image by calculating the Euclidean distance between intrinsic settings.
their descriptor vectors. A matching pair is detected, if its Algorithm consists of four main parts:
distance is closer than 0.7 times the distance of the second 1) Integral image generation,
nearest neighbour. The geometric constraints reduce the 2) Fast-Hessian detector (interest point detection),
3) Descriptor orientation assignment (optional),
impact of false positive matches, yet this can be done on top of
4) Descriptor generation.

Fig.1. Flow of SURF algorithm

Integral image is used by all subsequent parts of algorithm

to significantly accelerate their speed. Eq. (1) shows integral
image. When using integral image, it is necessary to always
read only four pixel values to calculate surface integral of any
size from original image.
I = (i , j) ........Eq.1
parameter creates the three dimensional space of
This fact is widely used when calculating Gaussian and Haar determinant results, usually referred to as ―scale
wavelet filter responses. space‖. Scale differs (and is quantized) according to
octaves and intervals.
(x,y) = det If SURF algorithm is used [4], all the representative points are
treated with same weight. This can be accounted for by
assigning dynamic weights to the representative points.
2
( ) = Dxx( ) Dyy( ) – (0.9 Dxy( )) Intuitively, true representative points will appear in images in
= (x,y,s) ………….Eq. 2 the training set, and false representative points will appear
rarely.
SURF uses determinants of Hessian matrices to locate image’s
significant points. Equation (2) shows original definition of
this determinant for general two dimensional function.
Fast-Hessian detector modifies this equation in two significant
ways: 1) Second order partial derivatives are replaced by
convolutions of image with approximated Gaussian kernels
second order derivatives. Approximation is done by box filters
with coefficients 1,-1,2,-22. Coefficient0:9 in eq. (3) is used to
compensate this approximation.2) Gaussian kernels are not
parameterized only by positionin image, but also by their size.

(x) = + + =
Scaling is represented by third parameter s 3, this
 Based on this intuition, the weight of each representative point
can be defined as follows

III. EXPERIMENTATION
The simulation is done in VISUAL STUDIO R2013a. As
shown in the fig.2. below, the query image with multiple
objects in it is given as input. The objects to be detected
from that query image are first taken as input training
images such as clock.jpg,vicks.jpg,pen.jpg.

Fig.2. Training object and query image

Fig.4.Representation of matched points among
Extract features of that training object & query image. Training object and query image
Then calculate interest points of both the images and based
on that matching is done. At final stage by comparing all features of that object from
image, it is detected in image. This process will go on in
iterative manner till all objects are identified in image.

Fig.3. Representative Points of training object 1 &

Interest Points of query image.

Now in some cases, there are matched points among Fig.5. Identifying the detected object in query image.
the two images which are not actually belong to the
object are called as outlier points and hence they should be As shown in fig.5. Object is detected in query image, as it is
excluded and only Inlier points are considered. highlighted in yellow box.

I. CONCLUSION
A proposed object recognition scheme is implemented to
evaluate the performance of the scheme. The SURF algorithm
is used in this scheme for recognizing multiple objects,
calculation of thresholds and measuring the object recognition
accuracy under variable conditions of scale, orientation &
illumination.
Analysis & Machine Intelligence, vol.27, no. 10, pp. 1615– 1630,
REFERENCES 2005.
[1] Q. Gu, T. Takaki, and I. Ishii, ―A fast multi-object extraction [6] Chong, R.M., Tanaka, T., ―Motion blur identification using
algorithm based on cell-based connected components labeling,‖ maxima locations for blind colour image restoration,‖J.
IEICE Trans. Inform. Syst., vol. E95-D, no. 2, pp. 636–645, Convergence 1 (1), 2010, 49–56.
2012. [7] Del Bimbo, A., Nunziati, W., Pala, P., 2009. DAVID: Discriminant
[2] Baugh, G., Kokaram, A., ―Feature-based object modelling for analysis for verification of monuments in image data, Proc.
visual surveil- lance,‖ in 15th IEEE International Conference on IEEE Int. Conf. on Multimedia and Expo (ICME’09), pp. 334–
Image Processing, ICIP 2008, pp. 1352–1355. 337.
[3] Bay, H., Tuytelaars, T.,VanGool, L.,―SURF: Speeded Up [8] Baugh, G., Kokaram, A., ―Feature-based object modelling
Robust Features,‖In ECCV (1), 2006, pp. 404–417. for visual surveil- lance, ‖ in 15th IEEE International
[4] Swapnali B.,Vijay K.,Varsha H., ―Feature based object Conference on Image Processing, ICIP 2008, pp. 1352–1355.
detection scheme.‖, Technovision International Conf.,2014 [9] Lindeberg, T. 1998. ―Feature detection with automatic scale
[5] KrystianMikolajczyk and CordeliaSchmid, ―A performance selection,‖ International Journal of Computer Vision, 30(2):79–
evaluation of local descriptors,‖ IEEE Transactions on Pattern
116.