International Journal of Research Publication and Reviews, Vol 3, no 5, pp 2289-2294, May 2022

International Journal of Research Publication and Reviews

Journal homepage: www.ijrpr.com ISSN 2582-7421

GESTURE BASED MEDIA PLAYER CONTROLLER

Sakshi Shinde1, Sarthak Mushrif1, Aditya Pardeshi1, Dhairyasheel Jagtap1, Guide: Prof.
Vandana Rupnar2
1
Department of Computer Engineering Marathwada Mitra Mandal College of Engineering, Pune
2
Department of Computer Engineering Marathwada Mitra Mandal College of Engineering, Pune

ABSTRACT

In today’s world, everyone goes for fast interaction with complex systems that ensure a quick response.Therefore, with increa sing technological
advances, response time and ease of operation are a major concern. This is where human computer interaction begins. This interaction is not limited
and challenges used devices such as keyboard and mouse input. Touch recog- nition has received a lot of attention. Touch is natural and is often used in
everyday interactions. Thus, communication using touch and computers creates a new level of interaction.In this paper we have discussed a low cost
system which uses hand gesture recognition technique to control the VLC media player.t increases efficiency and makes interaction effortless by letting
the user control his/her laptop/desktop from a distance.

Index Terms: Human Computer Interaction, VLC media player, Gesture Recognisation, OpenCV, PyAutoGui, Google Media Pipe, Image Processing

1. INTRODUCTION

Gesture recognition provides real-time information on a computer to enable it to fulfill user instructions. It is the site of current research
applicable to computer vision and machine learning. Normally, communication between a com- puter and a person is done through a keyboard and a
mouse. However, control and interaction with the media player using the keyboard and mouse. To make this interaction easier and simpler without the
use of additional devices such as the keyboard and mouse, we suggest building interactive gesture- based media player controls.In this gesture based
media player system we will see both hand gesture. Nowadays media player has become an integral part of our daily lives and is used by anyone and
anyone. Typically, a media player has functions such as volume boost, volume reduction, playback, pause, forward, backward and mute functionality.
This gesture- based interaction controls those media player activities with a webcam. We will be accomplishing this by processing Images and
Pyautogui. Image processing using OpenCV and other functions such as Gaussian and Morphological transformation. Pyautogui is a python library
that allows mouse and keyboard control. We use hand recognition to play and pause videoand touch recognition for other functions. Vision-based
gesture recognition is an important technology for human computer interaction and friendship, and has received more attention in recent years. Apps
designed for touch attention usually requirea limited background, a set of touch commands and a camera to take pictures.In this paper we will introduce
an application that uses flexible hand gestures such as installing control of the VLC media player. We have considered one hand touch and your
directional movement defines the touch of the app. In this application the image acquisition is done using a webcam. Others functions in VLC media
players are frequently used andthus VLC media player controls are used in those functions using the pre-defined touch.

In this paper we present an application designed for personal computer interaction that uses a variety of computer recogni- tion techniques to detect hand
gestures to control a VLC mediaplayer. The purpose and objectives of this app are to use the free interface of a natural device, which sees hand gestures
as commands. The app uses a webcam used to capture images.To control a VLC media player using a defined touch, the app focuses on a specific
VLC function that is widely used.

2. LITERATURE SURVEY

1) A Static Hand Gesture and Face Recognition System for Blind People 2019 Saransh Sharma, Samyak Jain, Khushboo IEEE This paper
presents a recognition system, which can be helpful for a blind person. Hand gesture recognition system and face recognition system has
been implemented in this paper using which various tasks can be performed.

2) Gesture based system for user interface control 2021 Georgi Krastev, Ivan Ralev IEEE The paper researches the features of camera as a
means of hand recognition and using them to certain application software. For this purpose, is created console application in java operate the
camera and hand recognition that used to the photo view, change zoom and regime change maps.
 International Journal of Research Publication and Reviews, Vol 3, no 5, pp 2289-2294, May 2022 2290

3. PROPOSED SYSTEM

To run this project, we used a webcam to find user input. In transferring input such as images through the image processing process to process the
image and to obtain information from the images. The model provides basic types of VLC media player functions, that is, play, pause, volume up,
volume down, mute, forward and backward the video. This is performed by hand gesture recognition. Hand detection is to play a nd pause the video.
Hand gesture recognition is to perform other functions. We make use of different libraries and modules of Python, like OpenCV, Google Media pipe,
Pyautogui, and sub process to build the application. The hand gesture recognition system on controlling media player works using real time gestures
input from user using integrated webcam and provided gesture matches with a function to control the media player. The implementation of the proj ect
is as follows:

 First webcam will open and user provides gesture input through that.
 The VLC app and video to play must be opened with a sub process shell command.
 The hand landmarks should be done through Google media pipe.
 After reorganization of gesture then the video starts play- ing.
 It detects and detects hand gestures through the image processing process and corresponds to the keyboard keys provided by the Pyautogui
library.
 Perform those operations.

Fig. 1. Working of system

4. METHODOLOGY

Following are the ways through which gesture controlled applications were created in the past:

A. Image Detection and Processing

Image Detection and Processing: OpenCV (Open Source Computer Vision), is one of the most widely used tools for computer vision and
image detection and processing tasks. It is used in various applications such as face detection, video capture, motion detection, object disclosure,
face mask detection, social distance, and much more. Steps For Image detection and processing which include capturing, resizing, converting,
extracting, detecting then finally recognizing.

1. At first we capture the image in the form of frame from the video. img = cv2.imread(’/image-path, cv2.IMREAD.COLOR)

2. Resize this image frame to 300 X 200.

3. Convert the BGR image into RGB image using the below code line: imgRGB = cv2.cvtColor(img, cv2.COLOR-BGR2RGB)

4. Detecting the boundary points.

5. Extract the Region of Intrest.

6. Recognize the Gesture

B. Gesture Recognition
 International Journal of Research Publication and Reviews, Vol 3, no 5, pp 2289-2294, May 2022 2291

Gesture Recognition: When the image frame is extracted, The Gesture in the image is recognised according to the cal culations based
on coordinates of the Media pipe Landmarks. Let us consider the figure 3 which shows the Hand Landmarks defined by MediaPipe over a palm.
Here,the ’0th’ landmarks is at the wrist of the palm, Landmarks 1 to 4 are distributed all over the thumb where the ’1st’ landmark is at the base of
the thumb and the ’4th’ landmark is at the tip of the thumb; Similarly landmarks 5 to 8 are distributed all over the index finger wh ere ’8th’
landmark is at the tip and ’5th’ is at the base; Landmarks 9 to 12 are distributed all over the middle finger where the ’12th’ landmark is at the tip
and ’9th’ is at the base; Landmarks 13 to 16 are distributed all over the ring finger where’13’is at the base and ’16th’ is at the tip, Landmarks 17 to
20 are distributed on the little finger where ’17th’ is at the base and ’20th’ is at the tip. Initially, the landmarks at the tip of the finger have y-
coordinate greater than the y-coordinate of the landmark at the base of the finger (eg:The y-coordinate of the 12th landmark is greater than the y-
coordinate of the 9th landmark).

For the Pause and Play function spacebar is used and the Gesture assigned for that is a closed wrist. Here In this case, when the wrist is closed the
tips of the fingers get lowered than the base of the fingers, Thus the landmarks at the tips of the finger s gets lowered than the landmarks at the
base of the fingers. So the new y-coordinates of the landmarks at the finger tips become smaller than the y-coordinates of the landmarks at the
finger bases. Thus, the closed wrist is detected.

5. ALGORITHM

A. Co-ordinate axes for Computer Screen

The diagram below shows the ax-wielding interface axes used when working with Computer Vision. By using these links we can track the
area of interest and arrange various activities according to the layout of the object on the screen.

Fig. 2. Co-ordinate axes for screen in Computer Vision

B. Hand Landmarks defined by MediaPipe

Using these hand landmarks we can define various gestures and link them to their corresponding intended functionalities to control the
media player. Co-ordinates on the screen and the landmarks on the hand are mapped together to generate the desired output.

Fig. 3. Hand Landmarks

 International Journal of Research Publication and Reviews, Vol 3, no 5, pp 2289-2294, May 2022 2292

Fig. 4. Different hand gestures

6. RESULTS

Different hand gestures for different functions.

A. Pause

Fig. 5. Pause

The system has succeeded in getting a hand gesture to pause and detect the action to be performed, so the corresponding video stop action is
active.

B. Volume Up

Fig. 6. Volume Up
 International Journal of Research Publication and Reviews, Vol 3, no 5, pp 2289-2294, May 2022 2293

The system has succeeded in detecting the Volume Up hand touch and detecting the action to be performed, so the corresponding action to
increase the video volume is effective.

C. Volume Down

Fig. 7. Volume Down

D. Seeking Forward

Fig. 8. Seeking Forward

The system has succeeded in detecting the hand-forward touch and detecting the action to be performed, so the corre- sponding video transfer
action is active.

E. Seeking Backward

Fig. 9. Seeking Backward

 International Journal of Research Publication and Reviews, Vol 3, no 5, pp 2289-2294, May 2022 2294

7. CONCLUSION

In the current world many resources are available to provide input to any application some require physical touch and some without the use of
physical touch (speech, hand touch etc.), the user can manage the system remotely without using the keyboard and mouse. This application provides a
novel human computer interface where the user can control the media player (VLC) using hand gestures. The system specific touch to control the VLC
player functions. The user will provide a touch as inserted depending on the activity you are interested in. The app provides the flexibility to define a
user’s touch of interest with a specific command that makes the app more useful for people with physical disabilities, as they can define touch
according to their ability.

The system managed to detect the volume down of the Volume Down and detect the action to be performed, so the corresponding action to lower the
video volume is active.

The program has successfully detected the rewind touch and detected the action to be performed, so the corresponding video rewind action is active.

8. FUTURE SCOPE

To overcome the drawbacks of the current system, we can modify it for better. While the application is running, if the user brings the hand closer
to his/her face, not intending to command the application, it nonetheless recognizes it and accordingly alters the volume or seek controls. To avoid this,
we can integrate iris detection to this project to make it run more smoothly. In these times of the Pandemic, where we are cauti oned about everything
we touch in a public place, this project can be extended to other public service technical systems to avoid direct contact. ATM machines, Ticket
Counters, etc can make use of the extended version.

REFERENCES

[1] Saransh Sharma; Samyak Jain; Khushboo,”A Static Hand Ges- ture and Face Recognition System for Blind People”,2019 6th In-
ternational Conference on Signal Processing and Integrated Net- works,IEEE,March.2019.

[2] Georgi Krastev; Ivan Ralev,”Gesture Based System For User Inter- face”,2021 5th International Symposium on Multidisciplinary Studies
and Innovative Technologies,IEEE,Oct,2021.

[3] Y. Wu and T. S. Huang, ”Vision-Based Gesture Recognition: A Review”, Lecture Notes in Computer Science, Vol. 1739,pp. 103-115,
1999.

[4] V. Pavlovic, R. Sharma and T. S. Huang, “Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review,” IEEE
Trans. On Pattern Analysis and Machine Intelligence, vol. 19(7), pp. 677-695,1997.

[5] Granit Luzhnica, Elizabeth Lex, Viktoria Pammer. A Sliding Window Approach to Natural Hand Gesture Recognition using a Custom Data
Glove. In: 3D User Interfaces (3DUI); 2016 IEEE Symposium on ; 2016 Mar 19 ; New York : IEEE; 2016 ; p.81-90.

[6] CH Hung, YW Bai, HY Wu Home outlet and LED array lamp controlled by a smartphone with a hand gesture recognition.Consumer
Electronics (ICCE); 2016 IEEE International Conference on ; 2016 Jan 7, IEEE, New York (2016), pp. 5-6.

[7] Meng, H., Furao, S., Jinxi, Z.: Hidden Markov models based dynamic hand gesture recognition with incremental learning method. Paper
Presented at the Neural Networks (IJCNN), 2014 International Joint Conference on (2014)

[8] Hasan, H., Abdul Kareem, S.: Fingerprint image enhancement and recognition algorithms: a survey. Neural Comput. Appl. 23, 1606–1608
(2013)

[9] Bretzner, L., Laptev, I., Lindeberg, T.: Hand gesture recognition using multi-scale colour features, hierarchical models and particle filtering.
Paper Presented at the Automatic Face and Gesture Recognition, 2002. Proceedings of Fifth IEEE International Conference on (2002)