VIRTUAL MOUSE USING HAND GESTURE

ABSTRACT:

In today’s rapidly evolving technological world, traditional input devices like the keyboard
and mouse are being reimagined to improve user experience and accessibility. One such
innovation is the Virtual Mouse using Hand Gestures, which offers a touchless, intuitive, and
modern way of interacting with computers. This project focuses on developing a system that
enables users to control mouse functionalities—such as cursor movement, left-click, right-
click, and scrolling—through simple hand gestures captured by a webcam. The system
leverages computer vision and machine learning techniques to detect and interpret hand
gestures in real- time. Using MediaPipe, a powerful framework developed by Google, the
system identifies 21 hand landmarks and tracks them with high accuracy. These landmarks
are then analyzed to recognize specific finger movements and gestures. With the help of
OpenCV, a popular open- source computer vision library, the system processes video frames
from the webcam, detects the hand position, and maps it to corresponding mouse operations.
The primary objective of this project is to eliminate the dependency on physical input devices
by creating a contactless interface, especially useful in scenarios where hygiene, accessibility,
or convenience is a priority. For example, during presentations, in healthcare environments,
or for users with physical impairments, the virtual mouse provides a hands-free and efficient
alternative. This virtual mouse system is not only cost-effective, as it requires only a webcam
and a computer, but also highly portable and easy to set up. Unlike hardware-based gesture
recognition systems that rely on sensors or wearable devices, this solution uses purely
software-based hand tracking, making it a lightweight and scalable application. In conclusion,
the Virtual Mouse using Hand Gestures represents a significant step toward enhancing
human-computer interaction. As future work, the system can be improved by integrating
more advanced AI models, gesture customization, and support for multi-hand interaction,
making it more robust and adaptable across various platforms and user needs.
INTRODUCTION:

With the rapid growth of technology, the way we interact with computers and digital devices
is constantly evolving. Traditional input devices such as the mouse and keyboard have been
the primary means of communication with computers for decades. However, in the era of
smart and contactless technology, there is a growing need for more natural and intuitive
interfaces. One such innovative solution is the Virtual Mouse using Hand Gesture
Recognition. This system allows users to control the cursor and perform mouse operations
such as clicking and scrolling, simply by using their hand gestures, without the need for any
physical contact. The main idea behind this project is to use computer vision and artificial
intelligence techniques to interpret real-time hand movements through a webcam. The camera
captures the video feed of the user’s hand, and then the system processes the frames using
libraries such as OpenCV and MediaPipe. MediaPipe, developed by Google, is a powerful
framework for building multimodal applied ML pipelines and is used here for hand tracking
and landmark detection. Once the hand landmarks are detected, various gestures such as
finger pointing, pinching, and tapping are mapped to specific mouse events like cursor
movement, left-click, right-click, and scrolling. The system offers several practical
advantages. It is cost-effective, requiring only a standard webcam and software tools that are
freely available. It is also hygienic and accessible, making it ideal for use in environments
where physical contact is limited, such as hospitals or clean rooms. Furthermore, it can be a
helpful tool for individuals with physical disabilities who may find it difficult to use
conventional input devices. In summary, the Virtual Mouse using Hand Gesture project
demonstrates how technology can be used to enhance human-computer interaction in a
modern and contactless way. It represents a significant step toward building intelligent
systems that are more intuitive, inclusive, and user-friendly.
 SYSTEM SPECIFICATION

HARDWARE SPECIFICATION:

Camera: A high-resolution webcam or camera (720p or higher)

Processor: A dual-core processor (Intel Core i3 or equivalent)

RAM: At least 4GB RAM

Storage: Sufficient storage space for the operating system and software

Operating System: Compatible with Windows, macOS, or Linux

Monitor: To display GUI and output of virtual mouse actions

Mouse: Physical mouse only for fallback. Main control will be virtual through gestures.

Computer / Laptop: System to run Python code and perform image processing.
SOFTWARE SPECIFICATION:

Programming Language: Python

Computer Vision Library: OpenCV

Machine Learning Library: TensorFlow or Pycharm

Operating System: Compatible with Windows, macOS, or Linux

Software Requirements: OpenCV, Python, and machine learning libraries

Python Libraries:

• OpenCV – For capturing video and image processing

• Mediapipe – For hand tracking and gesture recognition

• PyAutoGUI – To control mouse cursor using gestures

• Numpy – For numerical operations

• Tkinter (optional) – For GUI (if needed)

IDE / Editor: Any Python IDE (e.g., VS Code, PyCharm, Jupyter Notebook)

Driver Software: Camera drivers should be properly installed and configured.

EXISTING SYSTEM:
In the traditional setup, computers are operated using physical input devices such as:

 Mouse
 Keyboard
 Touchpad
 Touchscreen (in some devices)

Some advanced systems may use voice control or eye tracking for accessibility, but these are
limited in flexibility and may require specialized hardware. In gesture control, earlier systems
required special gloves or sensors to detect hand motion.

Characteristics:

 Requires physical mouse or touchpad.

DRAWBACK:

Physical dependency: Requires physical contact with a mouse or touchpad.

Wear and tear: Physical devices are prone to mechanical failure or damage over time.

Limited accessibility: Difficult for users with disabilities or hand injuries.

Restricted freedom: Users must remain near the device to operate it.

Cost of specialized sensors: Systems using gloves or advanced sensors can be expensive and
not easily accessible for all users.
PROPOSED SYSTEM:

The proposed system introduces a Virtual Mouse using Hand Gestures by utilizing:

Webcam or external camera

Computer vision techniques (OpenCV)

Hand landmark 3detection (e.g., Mediapipe)

Features:

Uses hand gestures to control the mouse.

Powered by Python with OpenCV, MediaPipe, and PyAutoGUI.

Provides touchless and contact-free control.

Works using a simple webcam, no need for expensive

hardware. User-friendly and low-cost.

ADVANTAGE:

Touchless control, more hygienic and

modern. Cost-effective using only a webcam.

Enhances accessibility for users with physical limitations.

Can be integrated with AI to improve accuracy and customization.

1. Capture Hand Gesture Using Webcam:

Use OpenCV to access webcam.

Continuously read video frames.

Flip image horizontally for mirror effect.

2. Hand Detection and Tracking:

Use MediaPipe Hand module.

Detect landmarks like fingertips and joints.

Get coordinates of index finger and thumb.

3. Gesture Recognition:

Define gestures:

Index finger up → move mouse

Index + thumb close → left click

Index + middle up → right click

Measure distance between fingers to identify gestures.

4. Mouse Control with PyAutoGUI:

Map hand movement on screen using pyautogui.moveTo().

Perform clicks using pyautogui.click(), pyautogui.rightClick().

5. Calibration and Smoothing:

Normalize coordinates from camera to screen size.

Add smoothing filter to prevent jittery movement.