TRIBHUVAN UNIVERSITY

INSTITUTE OF ENGINEERING

Himalaya College of Engineering

A Minor Project Proposal On

“DRIVER’S DROWSINESS DETECTION”

Submitted to:

Department of Electronics and Computer Engineering

Himalaya College of Engineering

Chyassal, Lalitpur

Submitted by:

Anamika Rai(HCE077BEI001)

Barsat Rai(HCE077BEI005)

Sanish Regmi(HCE077BEI014)

December, 2023
“DRIVER’S DROWSINESS DETECTION”
A Minor Project Proposal

In partial fulfillment for the award of a Bachelor’s degree in

Electronics, Communication and Information Engineering

Submitted to:

Tribhuvan University

Institute of Engineering

Department of Electronics and Computer Engineering

Himalaya College of Engineering

Chyassal, Lalitpur

Submitted by:

Anamika Rai(HCE077BEI001)

Barsat Rai(HCE077BEI005)

Sanish Regmi(HCE077BEI014)

December, 2023
Acknowledgement

First of all, we are deeply grateful and we give special thanks to our project
coordinators Er. Ramesh Tamang and Er. Narayan Adhikari Chhetri and HOD
Er. Ashok GM and Deputy HOD Er. Devendra Kathayat for providing the best
support and required guidelines to understand the feasibility, scopes, problems
and system requirement analysis for this project. And then to develop the
project proposal. We were very grateful for the time you dedicated to provide
constructive feedback and valuable suggestions for the project. We thank both
for encouragement and mentorship you give to us.

We are incredibly grateful to our friends and seniors for their valuable time,
support and feedback throughout process. In all humility and deepest
appreciation, we acknowledge the contributions of all who have helped us reach
this stage. We are overwhelmed by their generosity, support, and guidance
which helped us learn, grow and develop as engineers.

Thank you all

Anamika Rai (HCE077BEI001)

Barsat Rai (HCE077BEI005)

Sanish Regmi(HCE077BEI014)

i
Abstract
Traffic accidents are the leading cause of death worldwide, with drunk driving
and fatigue playing a major role. Detecting driver fatigue early on is crucial to
prevent accidents. Traditional methods can distract drivers or require expensive
sensors. Therefore, there is a need for innovative and non-intrusive techniques
to improve accuracy and usability of fatigue detection systems and enhance
road safety.

However, this project presents an innovative driver fatigue detection

framework. It uses image processing to detect facial landmarks, calculate
important metrics such as eye ratio (EAR) and eye closure ratio (ECR), and
dynamically adjust the sleep threshold.

This new approach offers a promising alternative to traditional methods for

identifying fatigue-related risks on the road. The effectiveness of this
innovative framework was evaluated using machine learning algorithms.
Empirical results showed a remarkable 84% accuracy.

This shows the potential of the proposed method to significantly improve the
predictive recognition of driver fatigue, which promotes road safety and
reduces the risks associated with drowsy driving.

ii
Table of contents

Acknowledgement ....................................................................................................... i
Abstract ....................................................................................................................... ii
List of figures .............................................................................................................. v
List of Tables.............................................................................................................. vi
List of Abbreviation ................................................................................................. vii
Introduction ................................................................................................................ 2
1.1 Introduction ................................................................................................ 2
1.2 Problem Statement ..................................................................................... 1
1.3 Objectives .................................................................................................... 1
1.4 Scope ............................................................................................................ 1
1.5 Applications and Limitations .................................................................... 3
1.6 Report organization ................................................................................... 4
.................................................................................................................................. 4
Literature Review ....................................................................................................... 5
3.1 Physiological Signals Analysis ................................................................... 6
3.1.1 Electroencephalogram (EEG) ........................................................... 6
3.1.2 Electrocardiogram (ECG): .................................................................... 7
3.1.3 Electrooculogram (EoG) ........................................................................ 7
3.1.4 Electromyogram (EMG) ........................................................................ 7
3.2 Facial Features Analysis ............................................................................ 7
3.3 Driving Pattern Analysis............................................................................ 9
3.3.2 Speed and Acceleration:................................................................... 10
3.3.2 Steering Wheel Movement (SWM): ................................................ 10
3.3.2 Standard Deviation of Lateral Position (SDLP): ........................... 10
3.2 Limitations ................................................................................................ 11
System Requirements ............................................................................................... 13

iii
 4.1 Requirement analysis ............................................................................... 13
4.1.1 Functional Requirements: ............................................................... 13
4.1.2 Non-functional Requirements: ........................................................ 13
4.2 System Configuration............................................................................... 14
4.2.1 Libraries to be used:......................................................................... 15
System Analysis & Design ....................................................................................... 16
5.1 Feasibility study ........................................................................................ 16
5.2 System Design ........................................................................................... 18
5.2.1 System architecture .......................................................................... 18
5.2.2 Detailed Design: ..................................................................................... 22
METHODOLOGY ................................................................................................... 23
PROJECT SCHEDULING ..................................................................................... 32
Gantt chart:........................................................................................................... 32
EXPECTED OUTPUT............................................................................................. 33
References: ................................................................................................................. 34

iv
List of figures

Fig 3.1 Road accidents due to drowsy driving according to NHTSA………..6

Fig 5.2.1.1 High level System architecture…………………………………18

Fig 5.2.2.2 Physical diagram…………………………………………………19
Fig 5.2.2.3 Class Diagram……………………………………………………20

Fig. 6.1 Block Diagram…………..…………………………………………..21

Fig. 6.2 Block Diagram…………..…………………………………………..22
Fig. 6.3 Block Diagram…………………………………………….………..24
Fig. 6.4 Architecture Diagram……………………………………………….25
Fig. 6.5 Block Diagram………………………...…………………………….29

v
List of Tables

fig 3.1 Table of advantages and limitations of each class……………...…….1

vi
List of Abbreviation

DDDS: Driver Drowsiness Detection System

HMI: Human Machine Interface

WHO: World Health Organization

NHTSA: National Highway Traffic Safety Administration

EEG: Electroencephalogram

EOG: Electrooculogram

EMG: Electromyogram

PERCLOS: Percentage of Time The Eyes Are Closed

ROI: Region of Interest

SWM: Steering Wheel Movement

SDLP: Standard Deviation of Lane Position

IDE: Integrated Development Environment

EAR: Eye Aspect Ratio

MOR: Mouth Opening Ratio

vii
Introduction
1.1 Introduction
In today's fast-paced world, road safety remains essential. Outside of
Kathmandu, the increasing number of deaths in the road safety landscape,
especially in heavy vehicles, highlights the need for innovative solutions. The
main factors in these accidents - speed, drunk driving, mechanical failure, poor
road conditions and the critical factor of driver fatigue.

The stark disparity in death tolls outside Kathmandu is a sign of regional road
safety problems, often exacerbated by the involvement of heavy trucks. We are
aware of this regional difference, our project aims to adapt solutions to the
challenges, especially in areas where heavy trucks play an important role in
traffic accidents.

Our project aims to adapt solutions to the challenges of regional road safety,
especially in areas where heavy trucks play an important role in traffic
accidents. By detecting driver fatigue, we hope to significantly reduce the
number of accidents caused by this critical factor and make the roads safer for
all. The goal of this project is to develop a reliable drowsiness detection system
that focuses on monitoring the open or closed state of the driver's eyes in real-
time.

By monitoring the eyes, it is believed that the symptoms of driver fatigue can
be detected early enough to avoid a potential accident. The detection of
drowsiness involves observing eye movements and blink patterns during a
sequence of facial images. There are various detection systems designed based
on the measurement of driver's fatigue through camera-based behavior analysis.
These systems utilize image processing to identify driver fatigue by analyzing
facial movements captured by cameras.

1
1.2 Problem Statement
Driver drowsiness is a significant threat to public safety and the driver's life.
Existing solutions have limitations of accuracy, cost, and practicality, with
some relying on social perception or costly sensors. Our proposed DDDS
framework can continuously monitor eye and facial states, accurately
distinguishing between open and closed states over time. Unlike existing
systems that are impractical in driving conditions, our user-friendly approach
uses a camera placed in front of the driver. However, we must identify reliable
indicators of sleepiness to develop a dependable and precise detection algorithm
that could prevent numerous traffic accidents worldwide.

1.3 Objectives
Our project aims to prioritize driver safety by developing a comprehensive
safety framework that detects signs of fatigue and steers the vehicle
accordingly. We focus on detecting the driver's blinks and assessing their
condition in order to reduce the speed of the vehicle and reduce the risk of
accidents. The project revolves around six main goals, including effective
recognition methods, analyzing facial features, studying physical changes that
indicate fatigue, developing a detection system, reducing driving speed and
improving traffic management.

1.4 Scope
The scope for a Driver Drowsiness Detection System (DDDS) using machine
learning can be listed as:

 Fatigue Detection: Implementation of machine learning algorithms for

the real-time analysis of facial expressions, eye movements, and other
physiological indicators to detect signs of driver fatigue.

2
  Alerting mechanism:
Development of mechanisms to generate timely alerts or warnings when
the machine learning model identifies significant signs of drowsiness in
the driver.
 Integration with the Vehicle Systems:
Interfacing the DDDS with the vehicle's alert systems or Human-
Machine Interface (HMI) for seamless communication and alert
delivery.
 Adaptability to Environmental Factors:
Incorporating features to consider environmental factors, such as
lighting conditions and road characteristics, for improved accuracy in
fatigue detection.
 Real-time Monitoring:
Continuously monitor the driver's state in real-time, ensuring that alerts
are generated promptly as soon as signs of fatigue are detected.

1.5 Applications and Limitations

The DDDS boasts a comprehensive scope, extending beyond a mere alert
mechanism. With further advancements, this system can seamlessly integrate
into various types of vehicles, providing not only fatigue detection but also
addressing concerns related to drunk driving and speeding. The adaptability of
the DDDS makes it a versatile solution for enhancing road safety across diverse
transportation contexts.
However, it's crucial to recognize the limitations inherent in the proposed
DDDS, understanding that while it excels in specific areas, it may not
comprehensively address all aspects of driver behavior or external factors
influencing road safety.

3
 1.6 Report organization

• includes introduction about major reasons of road accidents ,the problem statement,about the
Introduction
project ,it's objectives ,scope ,application and limitations

Literature
•includes past literature reviews on topic related to DDDS
Review

Requirement
•includes the system analysis with diagrams and flow charts and lists of requirements
Analysis

•includes the system design analysis

System Design

•includes the core technical part of the project like how it works
Methodology

•includes a gantt chart showcasing our estimated timeline to develop this project for practical
Project implementation
scheduling

•includes all other informations like the expected output

conclusion

4
Literature Review
Drowsiness and fatigue are often overlooked contributors to road crashes,
primarily due to drivers underreporting and technical challenges in
investigation. Addressing and mitigating the causes of falling asleep at the
wheel could significantly enhance traffic safety.

On January 3, 2009, a devastating event unfolded on the Pokhara-Baglung

highway in Parbat district, Nepal. A bus traveling from Kathmandu to Beni
deviated from the road, leading to four fatalities and over 30 injuries as it
plunged into the stony banks a River. Surprisingly, the driver survived. Upon
inspection, he displayed no indications of alcohol influence, maintained clear
vision, and refuted any substance abuse. He attributed the tragic incident to
drowsiness while driving.

According to the latest WHO data published in 2020 Road Traffic Accidents
Deaths in Nepal reached 4,654 or 2.90% of total deaths. The age adjusted Death
Rate is 20.65 per 100,000 of population ranks Nepal #72 in the world. Review
other causes of death by clicking the links below or choose the full health
profile.

The National Highway Traffic Safety Administration (NHTSA) in the United

States estimated that driver fatigue and sleepiness played a role in
approximately 56,000 vehicle crashes annually in the mid-1990s. Out of these
incidents, around 1,550 resulted in fatalities.

With the escalating number of daily road traffic accidents, there is an urgent
need to identify drivers potentially at risk.

Considering the increasing number of road traffic accidents day per day, it is a
need of the hour to detect drivers possibly at risk

5
Fig 3.1 Road accidents outcomes due to drowsy driving according to NHTSA

3.1 Physiological Signals Analysis

Driver drowsiness poses a significant threat on the road, contributing to
accidents and fatalities. Recognizing early signs of drowsiness can help prevent
these tragedies. Physiological signals offer valuable insights into a driver's state
of alertness, making them a powerful tool for drowsiness detection.

3.1.1 Electroencephalogram (EEG)

Electroencephalogram (EEG), the most commonly used technique, measures
the electrical activity of the brain through electrodes placed on the scalp.
Different frequency bands in the EEG signal correspond to various states of
alertness. For example, delta and theta waves are more prominent during
drowsiness, while alpha and beta waves indicate wakefulness. By analyzing
these frequency bands, researchers can identify drivers experiencing
drowsiness with high accuracy.

6
3.1.2 Electrocardiogram (ECG):
Electrocardiogram (ECG) measures the electrical activity of the heart. Changes
in heart rate and the LF/HF ratio, which reflects the balance between the
sympathetic and parasympathetic nervous systems, can indicate drowsiness.
Specifically, a decrease in heart rate variability and an increase in the LF/HF
ratio are associated with a decline in alertness.

3.1.3 Electrooculogram (EoG)

Electrooculogram (EoG) measures the electrical potential difference between
the cornea and the retina. This signal can detect eye movements, including rapid
eye movements (REM) and slow eye movements (SEM). REM typically occur
during sleep, while SEM are common during drowsiness. By analyzing the
patterns of these eye movements, researchers can identify drivers who are
struggling to stay awake

3.1.4 Electromyogram (EMG)

Electromyogram : While these techniques provide valuable information about
a driver's alertness, they also have limitations. Physiological signals can be
intrusive due to the need for sensors attached to the body. Additionally, the
equipment can be expensive, and obtaining sufficient labeled data for all states
of alertness can be challenging. Nevertheless, ongoing research aims to address
these limitations and improve the accessibility and accuracy of physiological
signal-based drowsiness detection systems.

3.2 Facial Features Analysis

Facial features analysis offers a non-intrusive and easily accessible approach
for detecting driver drowsiness. By analyzing video footage of the driver's
face, researchers can identify fatigue-related behaviors like:

7
  Yawning: Frequent yawning is a clear indicator of drowsiness.
 Eye Closure Duration: The percentage of time the eyes are closed
(PERCLOS) is a reliable measure of drowsiness, especially when
detecting micro sleep periods.
 Head Pose: Changes in head pose, such as drooping head or tilting to
the side, can suggest the driver is struggling to stay awake.

Facial features analysis typically involves the following steps:

 Region of Interest (ROI) detection: Identifying the driver's face in the

video frame.
 Feature extraction: Extracting relevant features from the ROI, such as
PERCLOS, yawning frequency, and head angle.
 Drowsiness detection: Analyzing the extracted features to determine
the driver's state of alertness.

This technique has several advantages:

 Non-intrusive: No sensors need to be attached to the driver, making it

more comfortable and less distracting.
 Accessible: Requires only a video camera, making it a cost-effective
solution.
 Easy to implement: Feature extraction and analysis algorithms are
well-established and readily available.

However, facial feature analysis also faces limitations:

 Lighting dependence: Normal cameras perform poorly in low-light

conditions, requiring additional illumination strategies like infrared
LEDs.

8
  Occlusion: Glasses, sunglasses, or facial hair can partially occlude the
driver's face, impacting the accuracy of feature extraction.
 Head pose variations: Changes in head pose due to driving maneuvers
can affect the reliability of some features.

Despite these limitations, facial features analysis remains a promising

approach for driver drowsiness detection due to its non-intrusive nature and
ease of implementation. Ongoing research aims to improve the robustness of
this technique by addressing the limitations and developing more
sophisticated feature extraction and analysis algorithms.

3.3 Driving Pattern Analysis

Analyzing driving patterns offers a non-intrusive approach for detecting driver
drowsiness by monitoring vehicle behavior. Several parameters can reveal
abnormal driver actions, including:

Speed and Acceleration: Excessive speed variations, aggressive acceleration,

and high RPM can indicate drowsy driving.

Steering Wheel Movement (SWM): Drowsiness can lead to decreased

steering wheel corrections and fewer reversals.

Standard Deviation of Lane Position (SDLP): Increased SDLP reflects

unstable path control and suggests drowsiness.

9
3.3.2 Speed and Acceleration:
Researchers have used deep convolutional neural networks to analyze driving
signals like speed, acceleration, and throttle position to classify five driving
styles, including drowsy driving.

Other studies employed recurrent neural networks and long short-term

memory models to predict normal and aggressive driving based on various
parameters.

High variability in driving speed has been linked to a state of hypovigilance.

3.3.2 Steering Wheel Movement (SWM):

 Sensors mounted on the steering column measure the driver's steering
behavior.
 Drowsy drivers tend to make fewer micro-corrections on the steering
wheel compared to normal driving.
 Small steering wheel movements (0.5° to 5°) are used to determine the
driver's drowsy state.
 Car manufacturers like Nissan and Renault have implemented SWM-
based systems, but their reliability is limited by specific environmental
conditions and road geometry.

3.3.2 Standard Deviation of Lateral Position (SDLP):

 SDLP represents the change in lateral position and indicates path control
stability.
 Front cameras measure the vehicle's lateral position to calculate SDLP.
 Increased SDLP is associated with reduced driver alertness and
drowsiness.
 Studies have shown a positive correlation between increased KSS
(Karolinska Sleepiness Scale) points and higher SDLP values.

10
 While reliable, the SDLP technique depends on external factors like road
markings, weather, and lighting conditions.

Advantages of Driving Patterns Analysis:

 Non-intrusive and does not require attaching sensors to the driver.

 Information readily available through vehicle networks.
 Cost-effective solution compared to other methods.

Limitations of Driving Patterns Analysis:

 SDLP technique is dependent on road conditions and lane visibility.

 Not all driving data may be available in real-time.
 Performance may be affected by environmental factors like weather and
lighting.

Driving pattern analysis offers a valuable and non-intrusive approach for

detecting driver drowsiness. While performance can be affected by certain
limitations, ongoing research aims to improve the accuracy and reliability of
this technique.

3.2 Limitations
Driver drowsiness detection systems (DDDS) have limitations that affect their
effectiveness and acceptability. A limited variety of data sets, false positives
and negatives, and difficulties in generalizing to real situations can lead to
unreliable alerts. Hardware limitations, individual variability, and external

11
factors can also present problems, as can ethical and privacy issues, and system
calibration. ML models may not understand context, and issues with regulatory
compliance, user acceptance, and cost and availability may arise. By addressing
these limitations, we can improve the effectiveness and acceptance of DDDS.

fig 3.1 Table showing the advantages and limitations of each class.

12
System Requirements

4.1 Requirement analysis

The proposed system must be able to detect drowsiness given a proper real-time
driving environment. The performance will depend upon the quality of the
camera as well. The proposed system must be available for use to the user as
and when needed provided that the user’s system meets the specified
requirements. The proposed system must be able to recover from failure in case
of the application crashing abruptly and become ready-to-use after recovery.

The prototype of the drowsiness detection system will be implemented on the

Raspberry Pi microcontroller board, along with the necessary peripheral
hardware, and Python 3 will be used to implement the software functionality of
drowsiness detection.

4.1.1 Functional Requirements:

I. Recording Driver's Behavior:

The system must record the driver's behavior from the moment the trip
begins.
II. Continuous Evaluation of Facial Features:
Continuous monitoring and evaluation of the driver's facial features
throughout the duration of a long trip.
III. Drowsiness Alarm:
The system should raise an alarm promptly if it detects signs of driver
drowsiness.

4.1.2 Non-functional Requirements:

I. High-Resolution Camera:
The camera capturing the video should have high resolution to
ensure accurate monitoring.

13
 II. Low Light Conditions:
The system must work effectively even in low light conditions,
ensuring robust performance in various environments.
III. High-Volume Alarm:
The raised alarm should have a high volume to effectively wake
the driver up in case of drowsiness.

4.2 System Configuration

Software Requirements:
 Operating System: Windows 10/8 (64-bit)
 Programming Language: Python 3
 Integrated Development Environment (IDE): Visual Studio Code

Hardware Requirements:
 Processor: 64-bit, quad-core, 2.5 GHz minimum per core
 RAM: 4 GB or more
 Display: 1024 x 768 or higher resolution monitors
 Camera: A webcam
 Raspberry Pi for onboard processing and integration with the vehicle's
systems.

14
 4.2.1 Libraries to be used:

I. SciPy:
A scientific Python open-source library used for calculating Euclidean
distance between the eyelids and lips.
II. NumPy:
A Python library for multidimensional array manipulation and fast
operations on arrays.
III. Datetime:
Python Datetime module for working with dates, times, and time
intervals. Used for fetching current time and calculating time intervals.
IV. Dlib:
An open-source C++ library implementing various machine learning
algorithms, including classification, regression, clustering, and
structured prediction.
V. Raspberry Pi Libraries:
Libraries specific to the Raspberry Pi for interfacing with its hardware
components and ensuring seamless integration into the vehicle system.
VI. OpenCV:
OpenCV is an open-source computer vision library utilized in Python
for pre-processing images received from the webcam. It enhances the
capabilities of our smart PC for vision-related tasks.
VII. TensorFlow:
TensorFlow is a free and open-source software library designed for
machine learning and artificial intelligence. It specifically focuses on
training and inference of deep neural networks. TensorFlow is used in
conjunction with Keras in our project.
VIII. Keras:
Keras, an open-source software library, serves as a Python interface
for artificial neural networks. It facilitates the implementation of
neural network architectures and is integrated with TensorFlow in our
project for efficient machine learning tasks.

15
System Analysis & Design
5.1 Feasibility study
Since drowsiness is a leading cause of serious traffic accidents, there is a
growing need for prevention techniques. This study proposes a sleepiness
prediction method based on eye-opening time. One problem with using eye-
opening time is that it predicts severe drowsiness before the driver feels sleepy.
Since ignoring potential dangers is one of the causes of traffic accidents and is
closely related to the thinking and sleepiness of the driver, this study focuses on
opening the eyes while driving. First, this report describes sleepiness and eye-
opening time hypotheses based on the results of previous studies. This
feasibility study's main objective is to determine if a driver drowsiness detection
system would be economically, functionally, technically, scalable and
environmentally feasible. Thus, we evaluate the following feasibility criteria
for a driver drowsiness detection system.

Technical Feasibility

Since driver drowsiness detection system uses technologies and materials that
are easily available in the market and general technical skills should required
which can be managed easily so the project is technically feasible.

Economic Feasibility

Since the materials and technologies that are easily affordable and available so
the project is economically feasible also the initial investment for the hardware
and software is estimated previously so feasible economically.

Operational Feasibility

Since DDDS operates accurately and reliable by ensuring the accuracy in

detecting driver’s drowsiness and alerting. It also gives feedback mechanisms
to alert the drivers when driver’s drowsiness is detected. And it also operates in

16
different varying environmental conditions. The system also allows to add extra
functionalities and maintenance so it is operationally feasible.

Environmental Feasibility

Since the materials and system components that are used in the system is
environmentally friendly and no any serious impact on environment. During
manufacturing the processes that minimizes the energy consumption, emission
of harmful gases and garbage which directly impacts the environment. So due
to economic friendly materials, energy efficiency and reusable properties of the
components it is environmentally feasible.

Scalability

So the DDDS is compatible with almost all types of vehicles and models. So
this system can be integrated into different types of the vehicles. Due to its
modularity it can be easily maintain and addition of the additional features and
updates. It can be also incorporate with other systems for better functions for
the performance optimization.

17
 5.2 System Design

5.2.1 System architecture

While driving, the camera records the driver's face and converts it into a video
stream. The program then analyzes the video to detect drowsiness and fatigue
and monitors the level of drowsiness. The main parts to consider in the
analysis are driver and face tracking, fatigue detection and detection of key
facial regions based on eye closure and yawning. If drowsiness is detected, a
warning sound will sound. The advanced system architecture consists of
model input and step-by-step preprocessing and evaluation. Step 1
preprocesses the video stream to track a person's face, step 2 removes key
areas of the face such as eyes and mouth, and step 3 detects signs of sleep
such as closing eyes, blinking and yawning.

Fig 5.2.1.1 High level System architecture

Human Face Feature

Evaluation
Tracking extraction

Facial Key
Eye closure
Real Time Region
Preprocessing
Video
Monitoring HAAR-
Blinking
Cascades
Eyes Ear
Yawning

18
Fig 5.2.1.2 System Architecture Flow Chart

19
Fig 5.2.1.2 System Architecture Flow Chart

20
5.2.2 Detailed Design:
The system has been designed such that the face and hence the eyes and mouth of
the driver are always monitored and if the predefined levels of alertness are
observed to be defaulted and compromised, then an appropriate alarm is set off,
and accordingly, action is taken to prevent any fatalities. Fig. 2 depicts the System
Design of Driver Drowsiness and Yawn Detection System. It can be seen that the
camera is used for monitoring the driver’s face continuously and upon detection of
drowsiness or fatigue, the system in the dashboard generates a voice alert type
warning to the driver.

Fig 5.2.2.1 High level System architecture

21
Fig 5.2.2.2 Class Diagram of DDDs

22
METHODOLOGY
This chapter contains the detail methodologies and algorithms for detection of
driver’s drowsiness and to alert the driver.

Detection of drowsiness involves a sequence of images of a face, and the

observation of eye movements and blink patterns. The analysis of face images
is a popular research area with applications such as face recognition, virtual
tools, and human identification security systems Region of Interest (ROI) is
estimated depending on Eye state (open/closed) and Mouth (yawning state).
Depending on these Features, Drowsiness of a person is detected and generates
an alert message.

Fig. 6.1 Block Diagram

Capture the Convert the Face detection

video video into
frames of photo

Alert System Drowsiness ROI Extraction

Detection (Eye and mouth
Estimation)

To distinguish driver's laziness various machine learning algorithms has been

used for comparison as to which one gives more accurate results. Facial
landmarks EAR (Eye Aspect Ratio), Mouth opening ratio (MOR) are used to
compute. If the EAR and MOR esteem is not exactly the threshold value, then
this would indicate a state of fatigue. In case of Drowsiness, the driver and the
passengers would be alerted by an alarm. Using classification model’s
prediction is done based on training and testing dataset. If the driver blinks his

23
eyes, it shouldn’t be categorized as drowsy to which the EAR has a threshold
value and the minimum time bound.

A camera is set up that looks for faces in the input video stream and monitors frames
of faces. In the event that a face is identified, facial milestone identification is
connected and the eye districts removed from the edges of the video stream. Based
on the work by Soukupova and Czech in their 2016 paper Real-Time Eye Detection
Using Facial Landmarks, we can then derive an equation that reflects this relation
called eye aspect ratio (EAR). EAR stand for Ear Aspect Ratio where numerator
denotes height of eye and denominator denotes the width of eye, the numerator
calculates the distance between the upper eyelid and the lower eyelid. The
denominator represents the horizontal distance of the eye. When the eyes are open,
the numerator value increases, thus increasing the EAR value, and when the eyes are
closed the numerator value decreases, thus decreasing the EAR value. details of eye
and its landmark is portrayed in Figure 1 represents a snapshot of facial landmark
points using Dlib library, which are used to compute EAR.

(|p2 – p6| + |p3 – p5|)

EAR =
2∗|p1−p4|

Fig. 6.2 Block Diagram

24
The numerator of this condition registers the distance between the vertical eye
milestones while the denominator figures the distance between flat eye tourist spots,
weighting the denominator around since there is just one bunch of level focuses yet
two arrangements of vertical focuses. On the off chance that the angle proportion of
the eye shows that they have been shut for more than a committed fixed time, we
will sound an alert framework so the driver awakens. We notice the viewpoint
proportion of the eye stays consistent for a while demonstrating that the eye was
open, at that point it falls quickly to nothing and afterward increments again which
shows the individual flickered. We will be noticing this eye angle proportion in our
laziness identifier case to check whether the worth remaining parts consistent or
tumbles to zero yet not increments again inferring that the driver has shut his eye for
an all-encompassing period.

In the event that the viewpoint proportion of the eye shows that they have been shut
for more than a committed fixed time, we will sound an alert framework so the driver
awakens. We notice the perspective proportion of the eye stays consistent for a while
demonstrating that the eye was open, at that point it falls quickly to nothing and
afterward increments again which shows the individual flickered. We will be
observing this eye aspect ratio in our drowsiness detector case to see if the value
remains constant or falls to zero but not increases again implying that the driver has
closed his eye for an extended period.

25
Fig. 6.3 Block Diagram

Video Extraction

Face Detection

Facial feature Open mouth

Detection Detection

Closed Eye Drowsiness Alert Mechanism

Detection
Detection

The method developed based on image processing are fast and precise to detect
driver drowsiness. Drowsiness can be detected merely based on PERCLOS. A
small camera conducted the observing test detection the levels of drowsiness
by the number of eyes blinks and accuracy of 93%. The facial expression, as
well as location of the eyes were detected by decision making algorithm. criteria
for detecting drivers’ level of drowsiness by eyes tracking included eye blink
duration blink frequency and PERCLOS that was used to confirm the result.

26
Fig. 6.4 Architecture Diagram

Input image Pre -processing ROI Eye Detection

Alert Recognition Creating

mechanism Tracking bounding box

In the above architecture diagram the first send the input image then this image is
processing in the region of interest (ROI) then detect the eyes from ROI and feed it to
the classifier then classifier will categorize whether eyes are open or closed the
calculate score to check whether the person is drowsy. Then the create the bounding
box of the image then recognition of tracking then the drowsiness alert.

Input Image: This block contains the sample images of the driver who is interacting
with our system. The images will act as input for our system and they will be helpful
to detect the facial expression and also the eye pattern of the driver in question. The
images taken are pretty important as they play a major part in output generation.
These images are then subjected to image processing, Eigen algorithm etc. for
detecting that the driver is drowsy or not.

Pre-processing: In this block the main focus is on image. In some cases, the image
may be distorted or captured incorrectly or it may happen that the image needs to
be enhanced as per the need. The process of correcting the missing components in
the image or making the required enhancements takes place in this block. It includes
operations such as (e.g., rotation, scaling, translation).

27
ROI (Region of Interest): Once the face is found its ROI is detected and processed. In
the second block the detection of driver drowsiness via eyelid closure takes place.
The eye of the driver is continuously monitored and if found closed for a certain
period of time the alarm sound.

Eye Detection: The total count of frames in which eyes are closed are identified.
When this number of frames is above a certain threshold, the diver will get a visual
warning on the navigation display that points out he is drowsy. Then the alarm will be
generated.

Creating Bounding Box: As we all know a bounding box is an imaginary rectangle that
serves as a point of reference for object detection and creates a collision box for that
object. Data annotators places these rectangles over images, outlining the object of
interest within each image by defining its X and Y coordinates as implemented in our
system the imaginary box is created for eye detection and also the collision box is
formed with its reference.

Recognition Tracking: The main face detection, recognition, and tracking features
and functions consists of: Detecting faces Decoding of an image file and detecting
faces on it can be done here. Recognizing faces by using example faces you can
recognize faces in an image. Tracking faces Tracking of faces using the camera preview
images, starting from a specific location in the image can be carried out here.

Drowsiness Alert: This block comes into play or has a role to play specifically when
the particular driver is detected drowsy. If the driver is drowsy the process of
generating an alert alarm is carried out here.

28
Eigen Face Recognition Algorithm:
Step 1: Start
Step 2: Original Faces training Set
Step 3: E = eigenfaces (training Set)
Step 4: W=weights (E training Set)
Step 5: input unknown image X
Step 6: Wx=weight (E, X)
Step 7: D = avg (distance {W, Wx})
Step 8: D < 0
Step 9: if {
Step 10: X is a Face then
Step 11: Store X and Wx
Step 12: else
Step 13: X is not face}
Step 14: End
Decision Making Algorithm:
Step 1: Start
Step 2: take image as input from a camera.
Step 4: Detect the face in the image and create a Region of Interest (ROI)
Step 5: Detect the eyes from ROI and feed it to the classifier,
Step 6: Classifier will categorize Whether eyes are open or closed.
Step 7: Calculate score to check Whether the person is drowsy.
Step 8: Stop

29
Mathematical Model: -
I= {I1, I2, I3, I4…………………. In}
I as a set of Inputs.
I1 = Face image.
F= {F1, F2, F3, F4………………. Fn}
Where F is a set of Function.
F1 = Detection of Face.
F2 = Detection of Eyes.
F3 = Extract Pupil.
F4 = Detect Eyes Aspect Ratio.
F5 = Detect Drowsiness.
O = is a set of Output.
O1 = Detection of Drowsiness.

The framework is formed utilizing the incremental model. The Centre model of
the framework is first created and afterwards augmented during this way within
the wake of testing at each turn. The basic undertaking skeleton was refined
into expanding levels of capability. At the subsequent incremental level, its
incorporated new execution backing and improvement. In our project we used
Haar cascading and Eigen algorithm. After passing our video feed to the dib
frame by frame, we are able to detect left eye and right eye features of the face.
Now we have drew contours around it using OpenCV. We calculated sum of
both eyes ratio which is that the sum of two distinct vertical distance Between
the eyelids divided by its horizontal distance. Now we check if ratio value is a
smaller amount than 0.25 if it's less an alarm is sounded a user is warned.

30
Drowsiness detection system created to scale back the chance of accident while
driving the system will record image of driver then face and eyes are detected
results of eyes detection each frame are going to be analyze if eyes are closed
for 4 second.

Fig. 6.5 Block Diagram

Face detection

Eye Detection

Eye Region Detection

Pupil center Detection

Iris Boundary Localization

EYE STATE ANALYSYS

Feeling
No
Drowsy??

yes

Warning message

31
PROJECT SCHEDULING

Gantt chart:

32
EXPECTED OUTPUT
1. Accuracy in Detecting Drowsiness: The system would accurately detect
driver drowsiness based on their facial and physical characteristics. The
facial recognition feature would be capable of detecting the signs of
drowsiness in the driver's eyes.

2. Low False Positive Rates: The system would maintain low false positive
rates to ensure that the driver is not alerted unnecessarily. For example, if
the driver blinks suddenly or adjusts their posture, the system would not
consider it as a sign of drowsiness.

3. User-Friendly Interface: The system would provide a simple and intuitive

interface for the driver to understand and use. The interface would display
clear and concise instructions on how to operate the system effectively.

4. Reliable and Precise Alert System: When the system detects driver
drowsiness, it would prompt the driver with an auditory or visual alert.
The alert would be designed to be attention-grabbing and effectively alert
the driver without causing excessive discomfort.

5. Integration with Other Driving Safety Systems: The DDDS would be

compatible with other advanced driving safety systems such as collision
avoidance systems and adaptive cruise control. This integration will
ensure a comprehensive approach to driving safety, with the DDDS
contributing to overall prevention of driver The expected output of the
driver drowsiness detection system is a reliable, user-friendly solution that
effectively prevents driver drowsiness-related accidents and improves
overall driving safety.

33
References:
[1] A. G. Wheaton, R. A. Shults, D. P. Chapman, E. S. Ford, and J. B. Croft, “Drowsy
Driving and Risk Behaviors — 10 States and Puerto Rico, 2011–2012,” Morb. Mortal.
Wkly. Rep., vol. 63, no. 26, pp. 557–562, Jul. 2014.

[2] B. C. Tefft, “Prevalence of motor vehicle crashes involving drowsy drivers,

United States, 1999– 2008,” Accid. Anal. Prev., vol. 45, pp. 180–186, Mar. 2012, doi:
10.1016/j.aap.2011.05.028.

[3] P. Philip et al., “Fatigue, sleep restriction and driving performance,” Accid. Anal.
Prev., vol. 37, no. 3, pp. 473–478, May 2005, doi: 10.1016/j.aap.2004.07.007.

[4] F. Riguelle and C. Goldenbeld, “Fatigue et somnolence. Bruxelles, Belgique.”

Institut Belge pour la Sécurité Routière – Centre de connaissance Sécurité routière,
Feb. 18, 2016.

[5] A. Shahid, K. Wilkinson, S. Marcu, and C. M. Shapiro, “Karolinska Sleepiness Scale

(KSS),” in STOP, THAT and One Hundred Other Sleep Scales, A. Shahid, K. Wilkinson,
S. Marcu, and C. M. Shapiro, Eds. New York, NY: Springer, 2012, pp. 209–210. doi:
10.1007/978-1-4419-9893- 4_47.

[6] C. Berka et al., “EEG Correlates of Task Engagement and Mental Workload in
Vigilance, Learning, and Memory Tasks,” Aviat. Space Environ. Med., vol. 78, no. 5,
pp. B231–B244, May 2007.

[7] H.-S. Shin, S.-J. Jung, J.-J. Kim, and W.-Y. Chung, “Real time car driver’s condition
monitoring system,” in 2010 IEEE SENSORS, Nov. 2010, pp. 951–954. doi:
10.1109/ICSENS.2010.5690904.

[8] M. Gillberg, G. Kecklund, and T. Åkerstedt, “Sleepiness and performance of

professional drivers in a truck simulator—comparisons between day and night
driving,” J. Sleep Res., vol. 5, no. 1, pp. 12–15, 1996, doi: 10.1046/j.1365-
2869.1996.00013.x.

34
[9] C. Cajochen, D. P. Brunner, K. Kräuchi, P. Graw, and A. Wirz-Justice, “EEG and
subjective sleepiness during extended wakefulness in seasonal affective disorder:
circadian and homeostatic influences,” Biol. Psychiatry, vol. 47, no. 7, pp. 610–617,
Apr. 2000, doi: 10.1016/S0006- 3223(99)00242-5.

[10] S. Otmani, T. Pebayle, J. Roge, and A. Muzet, “Effect of driving duration and
partial sleep deprivation on subsequent alertness and performance of car drivers,”
Physiol. Behav., vol. 84, no. 5, pp. 715–724, Apr. 2005, doi:
10.1016/j.physbeh.2005.02.021.

35