Hand Gesture Recognition: Jay Prakash, Uma Kant Gautam
Hand Gesture Recognition: Jay Prakash, Uma Kant Gautam
                                                                                  Published By:
                                                                                  Blue Eyes Intelligence Engineering
Retrieval Number: F90300476C19\19©BEIESP                               54         & Sciences Publication
                                                Hand Gesture Recognition
    After all these techniques the Vision Based approach               A. Research Status
introduced which worked mainly on shape. As every person                   Many researchers and students work on Hand Gesture
have one thumb and four fingers in normal condition so this            Recognition so they use different approaches to make their
technique is more successful for hand gesture recognition              system efficient but in each work there is always some
than other techniques. This method is very successful also             limitation which cannot be nullify so to make our system
described in this paper Mehra et. al. (5) for hand gesture             efficient we are taking help of other research papers and
recognition.                                                           analyze where the author made errors. Like some researchers
        According to Anant Atray (6) in his paper "Automatic           used algorithm based on Genetic Algorithms (8) and also
Hand Gesture Recognition" he is using best algorithm for               algorithms based on neural network (12) these two methods
hand gesture recognition which give optimal result and                 are good in gesture observation purpose but inefficient as
complexity is very low but there is one limitation in his              compare to Hidden Markov Model. Both of take huge amount
project that if the skin color is same as background or                of time to train the system and also large amount of training
environment where hand present then his system is not able to          data that's why we are using algorithm based on Hidden
distinguish between skin color and background which is                 Markov Model as it comparatively faster than both GA and
major issue cause it can change our entire result and can lead         Recurrent Neural Network.
to false output. So in my project we are using YUV color                       There is also some problem with many system which
space and camshift algorithms which help to differentiate              are using histograms and also skin detection problem (13).
between skin color and background.                                     Skin problem issue is major because in this noise increases
       According to T. Freeman et. al. (6) they are using the          drastically if skin color and background color is similar that's
orientation histogram to classify different gestures and
                                                                       why we are using YUV color space and Camshift algorithm
interpolation. Histogram is very simple and powerful as even
                                                                       which to distinguish between background and skin color.
in different lights and colors the gesture recognition will be
always correct. The main advantage of histogram is in
dynamic gestures because in dynamic gesture the movement                   III. HAND GESTURE RECOGNITION BASICS
of hand is changed over time period so according to this paper
histogram helps to find dynamic gestures very easily. But only         A. Human Computer Interface(HCI)
limitation in using histograms that similar gestures can have             Developing some technology that let human to interact or
different types of histogram orientations and similar                  communicate with machine is Human Computer Interface. To
histogram orientation can represent completely different               interact with computer human can use many things like hand
gestures so this is bit confusing in gestures using orientation        gestures , eye recognition or some other device like
histogram (6).                                                         instrumented glove etc.
       As gesture recognition is very useful for nowadays so
some researchers also forming Gesture recognition using                B. Gestures
recurrent neural networks. According to Murakami et.al. (7)               Gesture (14) is also a way of communication like we
recurrent neural network also has its own advantage in gesture         communicate in daily life. But in daily life, we used verbal
recognition as in sign language to differentiate between words         and vocal communication while gesture is non verbal way of
like trained-untrained , mom-dad , present-absent are very             communication or we can say it is a silent communication.
hard. So in this neural network help a lot cause using this our           Gestures constitute every type of communication except
system can capture past history to identify the word are used          verbal and written. The other way of communication apart
in pairs that are confusing. But recurrent neural network in           from that is making hand movement , giving facial
gesture recognition has its own disadvantages it is very slow          expressions , giving different type of body postures these all
means even to learn only 10 to 20 words while training the             are gestures.
system , the system can take 4 to 5 days which is large amount            We can say gesture is the body language that means to
of time that's why in my system we are not using neural                communicate with others by just making the movement and
network for gesture recognition (7).                                   motion of body parts. Gesture is very quick communication if
    Some researchers used algorithms based on Hidden                   both parties understands the gesture meaning same. By this
Markov Model and some use the algorithms based on Genetic              we can also show our emotions by just some movements.
Algorithms. According to X. H. Wang et.al.(9) Genetic                     According to movement gestures are of two types :
Algorithm(GA) convert capture input as a discrete points and              a. Dynamic Gesture - Change over period of time (16).
after that convert the problem of recognition gesture image               Examples of dynamic gesture :
into problem of combinatorial optimization that include                       Waving of hand means “goodbye”.
discrete points and then gesture recognition algorithm applied
to detect gestures. But This genetic algorithm have
disadvantage that it require high amount of training data and
long period of time. So, we are using Hidden Markov
Model(HMM) is our work because learning algorithms used
for HMM is very efficient. Using HMM learning can take
place directly from sequence of raw data. Our work using
HMM which also advantage over genetic algorithms that it
uses less number of training data as compare to GA and also
training time is less in HMM.
                                                                            Published By:
                                                                            Blue Eyes Intelligence Engineering
Retrieval Number: F90300476C19\19©BEIESP                          55        & Sciences Publication
                                                     International Journal of Recent Technology and Engineering (IJRTE)
                                                                         ISSN: 2277-3878, Volume-7 Issue-6C, April 2019
  b. Static Gesture -- observed over a spurt of time (17).             environment. The virtual environment that are using in
  It is type of static gestures (12) which means "stop".               present days can be displayed on screens and allow user to
                                                                       implement all of system applications through it. We can
                                                                       divide Virtual reality into:
                                                                       a. Forming a real environment copy or simulation for testing
                                                                            and training of software , project or systems. We can also
                                                                            use this for education purpose.
C. Gesture Recognition                                                 b. We can develop a environment which is like real life places
                                                                            but actually doesn't exist . For example in Games like
  Understanding a full message, interpretation of all static
                                                                            Pubg and GTA Vice city we see many maps or places
and dynamic gestures is necessary for over a period of time.
                                                                            which is same as real life places but in actual don't exist.
This complex process is GESTURE RECOGNITION.
  example of gesture: Shoulder Shrugging                               D. Sign Language
                                                                          This language is different from our daily language we use
                                                                       verbally or which we speak. This sign language (2) includes
                                                                       all the non- verbal way of communication (5) likewise the
                                                                       movement of body on any statement made, gesture made by
                                                                       hands to illustrate things to some person or making
                                                                       expressions by face, All these things are included in sign
                                                                       language. This language is frequently used by deaf and mute
                                                                       persons.
                 IV. PROPOSED WORK
   There are many use of Hand Gesture Recognition and that's
the reason significant number of Gesture Based application
evolved, Some of them are listed below :
  A. 3 Dimensional Geometry Design
   Auto CAD (computer aided design) is an Human Computer
Interface which is used for designing and drafting of 2
dimensional and 3 dimensional images. By using mouse and
keyboard it is difficult for an programmer or user to design 3D
design because making a 3D images involves all 6 Degree of
Freedom(DOF) and allocating points in space using mouse is
very hectic and complex. Now CAD provides facility to
translate points or rotate points of image in any direction.
Using this we can also see image from each and every
direction according to our requirement to analyze it (9).
Massachusetts institute of technology (10) has come up with
the 3D RAW technology that uses a pen embedded in
polhemus device to track the pen position and orientation in
3D.
  B. Telepresence
   Tele Presence (2) is way of communicating to person
which are not in same room by using internet services like
digital video or 3D visualizer. Tele Presence advantage is that
any user can remotely present on location where he is permit                      Fig 1. Gestures used for English Alphabet
to connect itself. For instances, in many companies, if
employees of company in different countries or in different
                                                                          Before this language it is very difficult to communicate
places then meeting still can be done by using video camera,
                                                                       with some physically challenged persons. Some time they
microphone and video screens. The Tele Presence include
                                                                       understand you but cannot express themselves and other time
collaborations especially instructions often, depends on the
                                                                       they can express themselves but cannot understand you.
physical act of one person showing another person how to do
                                                                               After this language, by watching, a deaf person can
something and even if your Tele presence robot has an arm of
                                                                       easily understand what you want to say and a mute person can
two it may not be at all intuitive for a remote user have
                                                                       even express themselves.
effective direct interactions.
  C. Virtual Reality
  Virtual reality (11) is computer generated 3 dimensional
environment with the help of software for users so that user
can run its program, test its system by assuming it be a real
                                                                             Published By:
                                                                             Blue Eyes Intelligence Engineering
Retrieval Number: F90300476C19\19©BEIESP                          56         & Sciences Publication
                                                  Hand Gesture Recognition
   Even in some news channels we can see one side reporter                 E. Linear Fingertip Model
speak and according to that reporter report a person besides it             The linear finger model are finger movement are assume
use sign language to sparse that report in deaf person too and           the linear rotational movement. Finger tissue modeling
 also by that a normal person can learn how to correctly use             requires linear deformation models The Fingertip model
                     that sign language.                                 posses capability of linear movement of finger pad, suitable
                                                                         fingertip model which can respond for every type of hand and
   V. ALGORITHMIC TECHNIQUES USED FOR                                    fingers because person to person size of hand, size of fingers ,
      RECOGNITION OF HAND GESTURES                                       their joint angle and their movement could vary. Linear
   To collect raw data we use either vision or glove based data          Fingertip model enforced to various type of deformation
collection system and various algorithms (14) used in order to           effects which are taken through the cameras during computer
collect the raw data smoothly and correctly.                             vision based tracking, like the video stream is captured with
  Various algorithm used are :                                           less control by which boundaries condition are hard to extract.
                                                                                           Gesture Recognition Steps
  A. Template Matching
                                                                            Step 1: First image is captured taken from stream using
   The template matching method for hand recognition                     camera which is working by OpenCV.
postures and gestures recognition used experimental method                  Step 2: Now the image is processed that what data it is
to know the required number of template of a certain gesture             containing.
to be taken that should be saved on the database for the                    Step 3: On data algorithms are applied to find optimal
matching process of the algorithm. If the system will not be             gesture.
able to detect and recognizes the gesture given with the                    Step 4: Obtained gesture is now compared with gesture
templates an additional templates must be trained and stored             dataset.
in the database until the system accurately recognize the                   Step 5: When if any dataset gesture is matched with input
gestures. The proponents will sum up all the time in second              gesture then the result will displayed on screen or gesture is
under a certain number the same number of template gesture.              recognized.
                                                                              Published By:
                                                                              Blue Eyes Intelligence Engineering
Retrieval Number: F90300476C19\19©BEIESP                            57        & Sciences Publication
                                                      International Journal of Recent Technology and Engineering (IJRTE)
                                                                          ISSN: 2277-3878, Volume-7 Issue-6C, April 2019
To segment the hand from other body parts we use logic that              finger in one hand and our hand's thumb which have joint with
is the hand is largest connected region.                                 our index finger is also detects by this sensor based glove
5) Hand position is calculated in each frame of video stream
and to calculate it centroid of hand is calculated . To calculate
centroid ,from initial to last position is calculated.
6) To find the path of our input i.e. hand movement we have
to join the all centroid points which form a trajectory and by
using all these procedures we can track the hand movement.
                                                                              Published By:
                                                                              Blue Eyes Intelligence Engineering
Retrieval Number: F90300476C19\19©BEIESP                            58        & Sciences Publication
                                                Hand Gesture Recognition
   Next constituent in this approach to captures every image                System performance enhance when we train our system
from stream of images which are running. The last                      with maximum number of datasets. As we want to be our
constituent to work this device efficiently some efficient             system more and more reliable for that we should train it with
algorithm should be applied in it which can take input,                maximum number of datasets.
compare it with entries in database and then give result in
minimum time. Here we are showing how a Vision Based                   REFERENCES
Approach work(20).
                                                                       1.      Matthias Rehm, Nikolaus Bee, Elisabeth André. Wave Like an
C. Comparison Between both Models(2)                                   Egyptian – Accelerometer Based Gesture Recognition for Culture Specific
                                                                       Interactions. s.l. : British Computer Society, 2007.
                                                                            https://ewic.bcs.org/content/ConWebDoc/21329
                         Glove                                         2. Sanjay Meena. A Study on Hand Gesture Recognition Technique(Thesis
                                           Vision-Based
     Methods            Based                                          Submitted in partial fulfillment of the requirements for the degree of Master
                                             Model                     of Technology in Telematics and Signal Processing)
                        Model
                                                                       3. Thomas G. Zimmerman, Jaron Lanier, Chuck Blanchard, Steve Bryson
  Price                 High            Less                           and Young Harvill. A HAND GESTURE INTERFACE DEVICE.
   Alleviation          Less            High                           .(published in CHI '87 Proceedings of the SIGCHI/GI Conference on Human
                                                                       Factors in Computing Systems and Graphics Interface
   for User                                                            Pages 189-19 https://dl.acm.org/citation.cfm?id=275628
   Hand Shape          To    be         Can differ                     4. Toward a vision-based hand gesture interface. Quek, F. Singapore : s.n.,
                                                                       August 23–26, 1994. Proceedings of the Virtual Reality System Technology
                     same                                              Conference. pp. 17-29.
  Calculation          Complex             Less complex                5. Hand Gesture Recognition for Human Computer Interaction. Mehra,
                                            than GB                    Meenakshi Panwar and Pawan Singh. Waknaghat, India : s.n., November
                                                                       2011. IEEE International Conference on Image Information Processing
     Efficient          Lesser          High                           (ICIIP 2011).
                                                                       6. AnantAtray. Automated Hand Gesture recognition.
                                                                       7. William T. Freeman, Michal Roth,. Orientation Histograms for Hand
                                                                       Gesture Recognition. Cambridge : s.n.
                                                                       8. Kouichi Murakami, Hitomi Taguchi. Gesture Recognition using
      VIII. CHALLENGES IN HAND GESTURE                                 Recurrent Neural Networks. Japan : s.n.
                RECOGNITION                                            9. X. H. Wang, H. J. Bao,. Recognition Based on Adaptive Genetic
                                                                       Algorithm. Journal of Computer-aided Design & Computer Graphics. 2007.
   Hand gesture recognition confronts many challenges (20),            10. A framework for hand gesture recognition and spotting using
these challenges are:                                                  sub-gesture modeling. M. R. Malgireddy, J. J. Corso, S. Setlur, V.
                                                                       Govindaraju, and D. Mandalapu. 20th International Conference on Pattern
A. Changed illumination: Change in light effects can affects           Recognition (ICPR '10). p. 3780.
     1our gesture input as it can change extracted skin region.        11. R. Pradipa, Ms S. Kavitha. Hand Gesture Recognition- Analysis of
B. Rotation Problem: Problem of Degree of freedom. if                  Various Techniques and their algorithms. 2006.
                                                                       12. Dynamic hand pose recognition using depth data. P. Suryanarayan, A.
     degree of freedom is changes then our gesture input may           Subramanian, and D. Mandalapu. August 2010. 20th International
     differ and by this output could vary                              Conference on Pattern Recognition (ICPR '10). pp. 3105–3108.
C. Distinguishing Problem: If with hand there is other things          13. Real-time hand status recognition from RGB-D imagery. A. D.
                                                                       Bagdanov, A. Del Bimbo, L. Seidenari, and L. Usai. November 2012. 21st
     which have shape and color like our skin but that is not          International Conference on Pattern Recognition (ICPR '12). pp.
     our gesture input then it can create problem for system to        2456–2459.
     distinguish between our hand gesture input and                    14. Vladimir I. Pavlovic, Rajeev Sharma, Thomas S. Huang. Visual
                                                                       Interpretation of Hand Gestures for Human-Computer Interaction;, . s.l. :
     background.
                                                                       IEEE Transactions on Pattern Analysis and Machine Intelligence,
D. Size Problem: This As per human being , we have different           1997.https://pdfs.semanticscholar.org/8dd3/d5ef2e1c40433417cd90e21ce9
     shapes and sizes of hand like small child have small              a0468a7626.pdf
     hands and adult have big hands so it can create problem           15. Simon, Haykin. Neural Networks: A Comprehensive Foundation. s.l. :
                                                                       2nd edition Prentice Hall PTR.
     for system.                                                       16. Virtual                  Reality.          agentic.ca.           [Online]
E. Position Problem: While giving input if hands position              http://www.agentic.ca/virtual-reality.
     differs like hand placed in corner of screen or all dots          17. Stuart, A. and Ord, K. Kendall's Advanced Theory of Statistics:
                                                                       Volume I—Distribution Theory. s.l. : Edward Arnold, 1994.
     which detecting hand position doesn't lies on hand then it        18. Xu, Yanan and Dai, Yunhai. Review of Hand Gesture Recognition
     can create problem to capture input from user.                    Study and Application. Contemporary Engineering Sciences. 2017, Vol. 10.
                                                                       Available: .https://doi.org/10.12988/ces.2017.7326
                                                                       19. Model-based hand tracking using a hierarchical Bayesian filter.
                    IX. CONCLUSION                                     Stenger B, Thayananthan A, Torr PH, Cipolla R. Sep 2006. IEEE
   Hand Gesture Recognition is an important Human                      Transactions on Pattern Analysis and Machine Intelligence. pp. 1372-84.
                                                                       20. Shigeo, Abe. Support Vector Machines for Pattern Classification.
Computer interface for interaction between living thing like           Kobe University.
human and machine system. Hand Gesture Recognition
System works like this : first user give input to the system by
making hand gestures, then system scanned the gestures by
using cam or sensor and deducts it into signal and passes the
program, now its program responsibility to first accept the
signal then examine what is the input given using gestures,
then check if there is any corresponding data is saved into
dataset then we will get our result.
                                                                             Published By:
                                                                             Blue Eyes Intelligence Engineering
Retrieval Number: F90300476C19\19©BEIESP                          59         & Sciences Publication