International Journal of Technical Research and Applications e-ISSN: 2320-8163,

www.ijtra.com Volume 7, Issue 3 (MARCH-APRIL 2019), PP. 36-39

VIRTUAL PAINT APPLICATION BY HAND

Prajakta Vidhate1, Revati Khadse2, Saina Rasal3,

Abstract— Gesture recognition is an emerging technology field. methods. optical cameras are easy to use and also inexpensive
We create a natural user interface for interacting with MS- but the quality of the captured images is sensitive to lighting
PAINT on windows machine.input will be be hand gestures of conditions and clutter backgrounds so it is difficult to detect
user which will be processed using camera for issuing a and track the hands properly.
command to MS PAINT application. The paint application
becomes virtual because the drawing happens in air. The virtual II. LITERATURE SURVEY
paint screen is made by help of UI. Web camera is used to extract
the gestures of hand in order to achieve fast and stable gesture In the past decades, gestures were usually identified and
recognition in real time without any distance restrictions. judged by wearing data gloves to obtain the angles and
positions of each joint in the gestureSeveral papers and projects
Index Terms— Gesture, web camera, Hand, Algorithm , Sense, have targeted the issue of hand gesture recognition. Francis et
Paint ,Human ,control etc, al[1]However, it is difficult to use widely due to the cost and
I. INTRODUCTION inconvenience of wearing the sensor. In contrast, the non
contact visual inspection methods have the advantage of low
Nowadays, the interaction between people and the cost and comfort for the human body, which are the currently
machines is mainly completed through the mouse, keyboard, popular gesture recognition methods. Chakraborty proposed
remote control, touch screen, and other direct contact manner, the skin colour models utilizing image pixel distribution in a
while the communication between people is basically achieved given colour space, which can significantly improve the
through more natural and intuitive non-contact manner, such as detection accuracy in the presence of varying illumination
sound and physical movements. The communication by natural conditions. However, it was difficult to achieve the desired
and intuitive non-contact manner is usually considered to be results using the model-based methods because of the light
flexible and efficient; many researchers have thus tried efforts sensitivity during the imaging process. The algorithm-based
to make the machine identify other intentions and information non-contact visual inspection methods were also used to
through the non-contact manner like people, such as sound, conduct the gesture recognition, such as the hidden Markov
facial expressions, physical movements, and gestures. Among model the particle filter, and Heer features AdaBoost learning
them, gesture is the most important part of human language, algorithm; however, it is difficult to execute real time due to
and its Gestures play very important roles in human the complicated algorithms. The above results cannot acquire
communication also. They are considered as the most easy gestures efficiently in real time since only the insufficient 2D
means of communication between humans and computers image information was used. Therefore, it is inevitable that
gesture recognition has wide applications including sign gesture recognition by 2D image is replaced by 3D with depth
language recognition, robotics and so on .gesture recognition information. In general, 3D information can be acquired by
can be simply categorized into two methods based on devices binocular cameras, Kinect sensor, Leap Motion and other
which are used to capture gestures: wearable sensor-based devices. Those devices can be usually utilized to obtain depth
methods and optical camera-based methods. The example of information by spatial relationship of different direction or
device used in the wearable sensor based method is the data infrared reflection, which can conveniently acquire non-contact
glove which is capable of exactly capturing the motion image for recognition and classification instead of wearing the
parameters of the user’s hands and it can achieve high complicated equipment. To facilitate this process many gesture
recognition performance. This devices used in wearable sensor recognition applications resort to the use of uniquely coloured
method affect the naturalness of the user interaction and they gloves or markers on hands or fingers [2].But computer vision
are also expensive. In optical camera based method, optical is a rapidly growing field, partly as a result of both cheaper and
cameras are used which record a set of images to capture more capable cameras, partly because of affordable processing
gesture movements from a distance. This optical cameras power, and partly because vision algorithms are starting to
recognize gestures by analyzing visual information extracted mature. By using Hand gestures user can communicate more
from the captured images so they are also called vision-based information in less time period. So for improving the interface

36 | P a g e
 International Journal of Technical Research and Applications e-ISSN: 2320-8163,
www.ijtra.com Volume 7, Issue 3 (MARCH-APRIL 2019), PP. 36-39

between users and computers human computers interaction which are needless and will not be required for detection of
(HCI) technology has great utilization [3]. The OpenCV itself hand image.
has played a role in the growth of computer vision by enabling After background substraction the coloured image we get
thousands of people to do more productive work in vision. from web camera gets converted to black and white image
With its focus on real-time vision. It gives reader a boost in where background is in black colour and white is image .
implementing computer vision and machine learning We find the contour with maximum area and then after
algorithms by providing many working coded examples to start checking if a ceratin value is greater than area we get convex
from. OpenCV, that allows the reader to do interesting and fun hull.in countour we get information about boundary pixels and
things rapidly in computer vision. It gives an intuitive in convex hull we get information about the lines which
understanding as to how the algorithms work, which serves to connects the fingers that will not exceed the boundary pixel
guide the reader in designing and debugging vision means contour . first Find the Convexity defects between
applications and also to make the formal descriptions of adjacent fingers of hand and then it counts the number of
computer vision and machine learning algorithms in other texts fingers.
easier to comprehend and remember. The gesture recognition We create a virtual paint application same as MS Paint by
methods such as template matching or finite-state machines using user interface libraries then this paint application is
were usually used, where high classification rates could also be controlled by gestures. Gesture are recognized and then action
obtained. However, only specific gestures can be recognized by is performed based on that gesture.
the above methods. Recent trends of Computer vision
techniques are easy, natural and less cost comparing[4] The A. BALL TRACKING:
convex hull detection algorithm recognizes gestures with the We will create a glove with ping pong ball attached to it.we
finger hull and can identify each fingertip position of the map the motion of ball to predefined gestures .first the video
human hand. It can get more gesture information and have capture starts then image is converted to from RGB space to
potential advantage. HSV space . HSV(hue saturation value) space gives us better
results then doing color based segmentation.
III. TECHNOLOGY USED Image is seprated into its 3 component images(i.e H S V
A. Open CV each of which is a one dimensional image or intensity
image).we use a condition for intensity values in the image and
OpenCV (Open Source Computer Vision Library) is a get a Binary image. i.e we have taken H intensity image and
library which mainly focuses at real-time computer vision. It is our ball is of red color .Then in the image we will find that the
free for both academic and commercial use. It has C++, C, values of the pixel where the ball is present , lies in a specific
Python and Java interfaces and supports Windows, Linux, Mac range. so we define a condition for every pixel : if (pixel >
OS, iOS and Android. OpenCV was designed for threshold_min & pixel < threshold_max )= pixel of o/p image
computational efficiency and with a strong focus on real-time is 1 else it is zero.
applications. The library has more than 2500 optimized
After doing this for H component,we do this for S & v
algorithms, which includes a comprehensive set of both classic
component.Now we have three binary images( only black and
and state-of-the-art computer vision and machine learning
only white) . Which has the region of ball as 1's and every
algorithms. It provides basic data structures for image
thing else which has the intensity values greater(less) than
processing with efficient optimizations.
threshold .The pixels that do not pass this conditions will be
B. C++ zero.We then combine all the above three Binary images (i.e
C++ is is a general purpose programming language. we AND them all). All the pixels that are white in the three
It is easy to leran and widely used language.it is object images will be white in the output of this step.So there will be
oriented language and includes classes, inheritance, regions too which will have 1's but with lower areas and of
polymorphism, data abstraction and encapsulation. It is a random shapes.
portable language and is often used for multi-device, multi- NOW WE USE HOUGHS TRANSFORM ON THE
platform app development.it has a rich function library. OUTPUT OF LAST OPERATION TO FIND THE REGIONS
It allows exception handling, and function overloading WHICH ARE CIRCULAR IN SHAPE.
It is a powerful, efficient and fast language. It finds a wide THEN WE DRAW THE MARKER ON THE DETECTED
range of applications – from GUI applications to 3D graphics CIRCLES AS WELL AS DISPLAY THE CENTER AND
for games to real-time mathematic RADIUS OF THE CIRCLES.LIKE THIS THE MOTION OF
BALL IS TRACKED.
IV. PROPOSE METHODOLOGY
B. Background substraction
Our project aims to create a virtual paint application where Background substraction (BS) is a common and widely
the drawing happens in air as the gestures are recognized by used technique for generating a foreground mask (namely, a
the system. a real time image which is coloured when captured binary image containing the pixels belonging to moving
by web camera.this colored image consists of many objects objects in the scene) by using static cameras. As the name

37 | P a g e
 International Journal of Technical Research and Applications e-ISSN: 2320-8163,
www.ijtra.com Volume 7, Issue 3 (MARCH-APRIL 2019), PP. 36-39

suggests, BS calculates the foreground mask performing a finger detection the Find Contours algorithm is utilized to
subtraction between the current frame and a background extract hand gesture contour from the ROI region The Graham
model, containing the static part of the scene or, more in Scan algorithm is used to calculate the convex hull of the
general, everything that can be considered as background given detected hand clusters and then gestures are recognized by
the characteristics of the observed scene. passing them through set of classifier.
Output:
C. Hand segmentation
hand segmentation is used to extract the hand image from
the background. There are several methods for segmentation.
The important step in segmentation is transformation and
thresholding. Segmentation partitions an image into distinct
regions containing each pixel with similar attributes. To be
meaningful and useful for image analysis and interpretation,
the regions should strongly relate to depicted objects or
features of interest. In this algorithm, the BGR image taken by
a camera is considered as input to the algorithm. The BGR
image is transformed into gray scale image. The gray scale
image is blurred to get the exact boundary. The blurred image
is threshold to the particular value.
D. Hand detection
It includes of contours,covex hull,convexity defects and
finger count
In Contours are the curves joining all the continuous points
along the boundary, having same color or intensity. The
contours are a useful tool for shape analysis and object
detection and recognition. The contour is drawn along the
boundary of the hand image which is found after thresholding.
In covex hull is the set of continuous points in the
Euclidean space that is connected to contours. Convex hull is
drawn around the contour. Contour points within the convex
hull. Convex hull works as an envelope around the hand.
In convexity defect,When the convex hull is drawn around
the contour of the hand, it fits set of contour points of the hand
within the hull. It uses minimum points to form the hull to
include all contour points inside or on the hull and maintain the
property of convexity. This causes the formation of defects in
the convex hull with respect to the contour drawn on hand.
E. Gesture controlling the paint application
Gesture recognition lets you control your virtual paint
application without touching a remote control or even the
screen of your laptop.the ball on glove is detected by a camera
and an action occurs.ie for eg,if we move the hand to our right
side then a rectangle will be drawn or if we move our hand to
left side then a circle will be drawn.
Like this the action will happen based on the gestures we VI. CONCLUSION
do. We propose a solution for hand recognition system for
virtual paint application. Generally, gestures could not be
V. RESULT
recognized by far distance but we proposed a system when
In this paper, web camera used as data acquisition device HGR system based on 1Iicrosoft Kinect for Xbox is
and the experimental tests are conducted by visual studio 2010 introduced. The system is motivated by the importance of real-
platform using the C++ program open CV is used for image time communication under It is capable of working in the dark,
processing such as image data and contour points seeking invariant to signer's skin color, clothing, and background
according to above analysis, there are two steps in lighting conditions, and it can be easily transplanted to other
experimental tests. Three point alignment algorithm is used for applications. The system is robust against clutter in the

38 | P a g e
 International Journal of Technical Research and Applications e-ISSN: 2320-8163,
www.ijtra.com Volume 7, Issue 3 (MARCH-APRIL 2019), PP. 36-39

background.. In future.it is expected that more applications (Eds), Proc. European Conference on Computer Vision,
with excellent recognition results will be developed. ECCV04, v. 1: 391-401, LNCS3022, Springer-Verlag, 2004.
[3] [3] compared methods for gesture recognition in cars,
REFERENCES evaluating accelerometers-based, glove-based, and Kinect-based
[1] J. Francis and A. B K, "Significance of Hand Gesture approaches. Mitra et al[2]
Recognition Systems in Vehicular Automation-A Survey", [4] [4]Radhika Bhatt, Nikita Fernandes, Archana Dhage "Vision
International Journal of Computer Applications, vol. 99, no. 7, Based Hand Gesture Recognition for Human Computer
pp. 50-55, 2014. Interaction" University Of Mumbai (IJESIT) Volume 2, Issue 3,
[2] [2] R. Bowden, D. Windridge, T. Kadir, A. Zisserman, M. May 2013.
Brady, “A Linguistic Feature Vector for the Visual
Interpretation of Sign Language”, in Tomas Pajdla, Jiri Matas

39 | P a g e