Skip to main content

Recognizing Two Handed Gestures with Generative, Discriminative and Ensemble Methods Via Fisher Kernels

  • Conference paper
Multimedia Content Representation, Classification and Security (MRCS 2006)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 4105))

Abstract

Use of gestures extends Human Computer Interaction (HCI) possibilities in multimodal environments. However, the great variability in gestures, both in time, size, and position, as well as interpersonal differences, makes the recognition task difficult. With their power in modeling sequence data and processing variable length sequences, modeling hand gestures using Hidden Markov Models (HMM) is a natural extension. On the other hand, discriminative methods such as Support Vector Machines (SVM), compared to model based approaches such as HMMs, have flexible decision boundaries and better classification performance. By extracting features from gesture sequences via Fisher Kernels based on HMMs, classification can be done by a discriminative classifier. We compared the performance of this combined classifier with generative and discriminative classifiers on a small database of two handed gestures recorded with two cameras. We used Kalman tracking of hands from two cameras using center-of-mass and blob tracking. The results show that (i) blob tracking incorporates general hand shape with hand motion and performs better than simple center-of-mass tracking, and (ii) in a stereo camera setup, even if 3D reconstruction is not possible, combining 2D information from each camera at feature level decreases the error rates, and (iii) Fisher Score methodology combines the powers of generative and discriminative approaches and increases the classification performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Ong, S.C.W., Ranganath, S.: Automatic sign language analysis: A survey and the future beyond lexical meaning. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 873–891 (2005)

    Article  Google Scholar 

  2. Pavlovic, V., Sharma, R., Huang, T.S.: Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 677–695 (1997)

    Article  Google Scholar 

  3. Heckenberg, D., Lovell, B.C.: MIME: A gesture-driven computer interface. In: Visual Communications and Image Processing, SPIE, Perth, Australia, vol. 4067, pp. 261–268 (2000)

    Google Scholar 

  4. Wu, Y., Huang, T.S.: Hand modeling, analysis, and recognition for vision based human computer interaction. IEEE Signal Processing Magazine 21, 51–60 (2001)

    Google Scholar 

  5. Jaakkola, T.S., Haussler, D.: Exploiting generative models in discriminative classifiers. In: Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems II, pp. 487–493. MIT Press, Cambridge (1998)

    Google Scholar 

  6. Smith, N., Gales, M.: Using SVMs to classify variable length speech patterns. Technical report, Cambridge University Engineering Department (2002)

    Google Scholar 

  7. Holub, A., Welling, M., Perona, P.: Combining generative models and fisher kernels for object class recognition. In: Int. Conference on Computer Vision (2005)

    Google Scholar 

  8. Liu, N., Lovell, B.C., Kootsookos, P.J., Davis, R.I.A.: Model structure selection and training algorithms for a HMM gesture recognition system. In: International Workshop in Frontiers of Handwriting Recognition, Tokyo, pp. 100–106 (2004)

    Google Scholar 

  9. Marcel, S., Just, A.: (IDIAP Two handed gesture dataset), available at: http://www.idiap.ch/~marcel/

  10. Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines (2001), Software available at: http://www.csie.ntu.edu.tw/~cjlin/libsvm

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Aran, O., Akarun, L. (2006). Recognizing Two Handed Gestures with Generative, Discriminative and Ensemble Methods Via Fisher Kernels. In: Gunsel, B., Jain, A.K., Tekalp, A.M., Sankur, B. (eds) Multimedia Content Representation, Classification and Security. MRCS 2006. Lecture Notes in Computer Science, vol 4105. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11848035_23

Download citation

  • DOI: https://doi.org/10.1007/11848035_23

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-39392-4

  • Online ISBN: 978-3-540-39393-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics