DROWSINESS ALERT SYSTEM USING DEEP LEARNING

thesis Submitted

in the partial fulfilment of the requirements for

the award of the degree of

BACHELOR OF TECHNOLOGY
in
COMPUTER SCIENCE & ENGINEERING

BY

NIHARIKA NEERUMALLA 15281A0579

SANDEEP ELKAPALLY 15281A0543
RAJKUMAR VALUSA 15281A05B5
REETHURAMAN PORIKA 15281A05A3

PROJECT GUIDE
Mr.K. BHASKER PRAKASH
(Assistant Professor)

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

KAMALA INSTITUTE OF TECHNOLOGY & SCIENCE
(Approved by AICTE, New Delhi and Affiliated to JNTU, Hyderabad)
Singapur, Huzurabad, Karimnagar, Telangana, India-505468
2018– 2019
 KAMALA INSTITUTE OF TECHNOLOGY & SCIENCE, SINGAPUR,
KARIMNAGAR, INDIA.

CERTIFICATE

This is to certify that N.NIHARIKA(15281A0579),E.SANDEEP(15281A0543),

V.RAJKUMAR(15281A05B5), P.REETHURAMAN(15281A05A3) of the IV B.Tech
(CSE) has satisfactorily completed the dissertation work for Project Work entitled
DROWSINESS ALERT SYSTEM USING DEEP LEARNING towards the partial
fulfillment of B. Tech degree in the academic year i.e. 2018-2019.

Project Guide Project Coordinator

Head of the Department External Examiner

 INDEX
Acknowledgment i
Abstract ii
List of Diagrams iii
Chapter1: Introduction
1.1 About Project 1
1.2 Existing System 1
1.3Proposed System 1
Chapter2: Analysis
2.1 Hardware and Software Requirements 2
2.2 Functional and Non-Functional Requirements 2
2.3 Module Description 3
2.4 Modules in Python 4
Chapter3: Design
3.1 Block Diagram 7
3.2 Data Flow Diagrams 7
3.3 UML Diagrams 10
3.3.1 Use case Diagram 11
3.3.2 Sequence Diagram 12
3.3.3 Collaboration Diagram 13
3.3.4 Activity Diagram 14
Chapter4: Implementation
4.1 Python 15
4.2 Machine Learning 16
4.3Deep Learning 17
Chapter5: Testing 26
Chapter6: Results 29
Chapter7: Limitations 32
Chapter8: Conclusion 33
Chapter9: Future Scope & Enhancements 34
Bibliography 35
 ACKNOWLEDGEMENT

The success of any course depends mostly on the teachers who us. Only good teaching
can interpret the syllabus and produce desirable changes and competent citizens. This one was a
team effort and many people whose names do not appear on the cover deserve credit. First, we
thank god almighty for his manifold mercies in carrying out of our project successfully.
We heart fully thank our Principal, Dr. K. SHANKAR, Professor and Head of the
Department of Computer Science & Engineering Dr. Lt. RAVINDRA BABU KALLAM,
project coordinators Mr. D. PRASHANTH KUMAR Assistant Professor in CSE for providing
all the resources in completing our project.
We thank our internal guide Mr. K. BHASKER PRAKASH, Assistant Professor in
Department of Computer Science & Engineering, for his guidance and support.
We also thank other teaching and non-teaching staff of Kamala Institute of Technology
and Science, for supporting us in every stage of our major project, entitled “DROWSINESS
ALERT SYSTEM USING DEEP LEARNING’’.
We thank our Parents and Friends for their moral support throughout the project which
helped to strengthen our will.

NIHARIKA NEERUMALLA (15281A0579)

SANDEEP ELKAPALLY (15281A0543)
RAJKUMAR VALUSA (15281A05B5)
REETHURAMAN PORIKA (15281A05A3)

i
 ABSTRACT

Now-a-days, more and more professions require long-term concentration for driving.
Drivers must keep a close eye on the road, so they can react to sudden events immediately.
Driver fatigue often becomes a direct cause of many traffic accidents. Therefore, there is a need
to develop the systems that will detect and notify a driver of her/him bad psychophysical
condition, which could significantly reduce the number of fatigue-related car accidents.
The aim of this project is to develop a prototype drowsiness detection system. The focus
will be placed on designing a system that will accurately monitor the open or closed state of the
driver’s eyes in real-time.
By monitoring the eyes, it is believed that the symptoms of driver fatigue can be detected
early enough to avoid a car accident. Detection of fatigue involves the observation of eye
movements and blink patterns in a sequence of images of a face.

ii
 LIST OF FIGURES
Title Page no
2.3 Module description
2.3.1 Facial Landmarks 03
2.3.2 Formula For Eye Aspect Ratio 04
3.1 Block Diagram 07
3.2 DFD Diagrams
3.2.1 Context level DFD 09
3.2.2 Top level DFD 09
3.2.3 Detailed level DFD 10
3.3 UML Diagrams
3.3.1 Use case Diagram 11
3.3.2 Sequence Diagram 12
3.3.3 Collaboration Diagram 13
3.3.4 Activity Diagram 14
4.3 Types of Pooling 18
6.1 Drowsiness detection
6.1.1Screenshot for Eye Detection 29
6.1.2 Screenshot for Drowsiness detection 29
6.1.3 Screenshot for Drowsiness alert 30
6.2 Yawn Detection
6.2.1 Screenshot for Mouth Detection 30
6.2.2 Screenshot for Yawn Detection 31

iii
 CHAPTER 1

INTRODUCTION

ABOUT PROJECT:

In this project, we have used deep learning to predict drowsiness of the driver. Deep
learning is an artificial intelligence function that imitates the workings of the human brain in
processing data and creating patterns for use in decision making. Deep learning is a subset
of machine learning in artificial intelligence (AI) that has networks capable of learning
unsupervised from data that is unstructured or unlabeled.
Driver fatigue is one of the major causes of accidents in the world. Detecting the
drowsiness of the driver is one of the surest ways of measuring driver fatigue. In this project we
aim to develop a prototype drowsiness detection system. This system works by monitoring the
eyes of the driver and sounding an alarm when he/she is drowsy.
In this project the eye blink of the driver is detected. If the drivers eyes remain closed for
more than a certain period of time, the driver is said to be drowsy and an alarm is sounded. The
programming for this is done in OpenCV using the Haarcascade library for the detection of facial
features.
EXISTING SYSTEM:

• There is no such application which continuously keep track of drowsiness of the driver.

PROPOSED SYSTEM:
• We proposed a different approach in providing security for driver using drowsiness
detection. By this project, once the face is detected, we have to extract the eye regions.
• We have to compute the eye aspect ratio (EAR), to determine if the eyes are closed..If the
eye aspect ratio indicates that the eyes have been closed for a sufficiently long enough
amount of time, we’ll sound an alarm to wake up the drive

1
 CHAPTER 2

ANALYSIS
2.1 HARDWARE AND SOFTWARE REQUIREMENTS

2.1.1 SOFTWARE REQUIREMENTS (PREFERABLE):

Software specifications of the system that is used in developing this application are:
• Operating System: WINDOWS or UBUNTU.
• Programming Language: Python.
• Tools Used: OpenCV.
2.1.2 HARDWARE REQUIREMENTS (PREFERABLE):
Hardware specifications of the system that is used in this project are:
• Processor: i3 or above.
• RAM: 2 GB or above.

2.2 FUNCTIONAL AND NON FUNCTIONAL REQUIREMENTS:

2.2.1 FUNCTIONAL REQUIREMENTS:
The following are the functional requirements of our project. They are:
 An OpenCV Algorithm to capture the movements of the objects or the users.
 It should detect each and every object at any position.
 It should analyze each and every movement.
 It should intimate by giving alarm.
2.2.2 NON-FUNCTIONAL REQUIREMENTS:
Following are the non-functional requirements:
 Convenience: This software can be easily used by the authorized user. It helps the user
to view all the movements in the restricted area.
 Availability: This software provides authorized user to view. Only authorized user can
only receive mail attached with a recorded video.
 Reliability: This software provides effective method to maintain the video.
 Size: The size of particular application plays a major role, if the size is less,then the
efficiency is very high.
 Speed:If the speed is high then it is good.Since the no. of lines in the code is less, hence
the speed is high.

2
2.3 MODULES DESCRIPTION:
Module: - A module allows you to logically organize your Python code. Grouping related code
into a module makes the code easier to understand and use. A module is a Python object with
arbitrarily named attributes that you can bind and reference.
The following modules are involved. They are:
 OpenCV
 Drowsiness Detection
 Yawn Detection
 Sound
OpenCV:-
Open CV is used to analyze the detected object from camera.Open CV-python makes use of
Numpy, which is a highly optimized library for numerical operations.Open CV is used for image
processing and video processing
Drowsiness Detection: -
Once the face is detected, we have to extract the eye regions.We have to compute the eye aspect
ratio(EAR),to determine if the eyes are closed..If the eye aspect ratio indicates that the eyes have
been closed for a sufficiently long enough amount of time, we’ll sound an alarm to wake up the
driver.
There are 68 facial landmarks

Figure:2.3.1 Facial landmarks

3
The Eye Aspect Ratio is calculated by using the below formula

Figure 2.3.2: Formula for Eye Aspect Ratio

Yawn Detection

Yawn is detected by distance between the upper lip and bottom lip. Here calculated value is the
absolute distance of the upper lip and bottom lip values.
If the calculated value is greater than the threshold value, then the driver gets the alert message
and it warns background automatically.
Sound

When the drowsiness of the driver is detected successfully background alarm will be played.The
playsound module provides access to the basic sound-playing machinery provided by Windows
platforms.
2.4Modules In Python

2.4.1 OpenCV: -

Python is a general-purpose programming language started by Guido van Rossum, which

became very popular in short time mainly because of its simplicity and code readability. It
enables the programmer to express his ideas in fewer lines of code without reducing any
readability. Compared to other languages like C/C++, Python is slower. But another important
feature of Python is that it can be easily extended with C/C++.
This feature helps us to write computationally intensive codes in C/C++ and create a
Python wrapper for it so that we can use these wrappers as Python modules. This gives us two
advantages: first, our code is as fast as original C/C++ code (since it is the actual C++ code
working in background) and second, it is very easy to code in Python. This is how OpenCV-

4
Python works; it is a Python wrapper around original C++ implementation.And the support of
NumPy makes the task easier. NumPy is a highly optimized library for numerical operations. It
gives MATLAB-style syntax. All the OpenCV array structures are converted to-and-from
NumPy arrays. So whatever operations you can do in NumPy, you can combine it with OpenCV,
which increases number of weapons in your arsenal. Besides that, several other libraries like
SciPy, Matplotlib which supports NumPy can be used with this. So OpenCV-Python is an
appropriate tool for fast prototyping of computer vision problems.

Installing OpenCV from prebuilt binaries:

1. Below Python packages are to be downloaded and installed to their default location
1.1. Python-2.7. x.
1.2. NumPy.
1.3. Matplotlib (Matplotlib is optional, but recommended since we use it a lot in
our tutorials).
2. Install all packages into their default locations. Python will be installed tC:/Python27/.
3. After installation, open Python IDLE. Enter import NumPy and make sure NumPy is
working fine.
4. Download latest OpenCV release from source forge site and double-click to extract it.
5. Go to OpenCV/build/python/2.7 folder
6. Copy cv2.pyd to C:/Python27/lib/site-packeges.
7. Open Python IDLE and type following codes in Python terminal.
> import cv2 >>> print cv2. __version__
If the results are printed out without any errors, congratulations!!! You have installed
OpenCV-Python successfully.
2.4.2 NumPy: -

NumPy, which stands for Numerical Python, is a library consisting of multidimensional

array objects and a collection of routines for processing those arrays. Using NumPy,
mathematical and logical operations on arrays can be performed.
This tutorial explains the basics of NumPy such as its architecture and environment. It
also discusses the various array functions, types of indexing, etc. An introduction to Matplotlib is
also provided. All this is explained with the help of examples for better understanding.

5
 NumPy is a Python package. It stands for 'Numerical Python'. It is a library consisting of
multidimensional array objects and a collection of routines for processing of array. Numeric, the
ancestor of NumPy, was developed by Jim Humulin.
Another package Numara was also developed, having some additional functionality. In
2005, Travis Oliphant created NumPy package by incorporating the features of Numara into
Numeric package. There are many contributors to this open source project.
Operations using NumPy: -
Using NumPy, a developer can perform the following operations:
 Mathematical and logical operations on arrays.
 Fourier transforms and routines for shape manipulation.
 Operations related to linear algebra. NumPy has in-built functions for linear algebra and
random number generation.
2.4.3 Scipy: -
SciPy is a collection of mathematical algorithms and convenience functions built on the
Numpy extension of Python. It adds significant power to the interactive Python session by
providing the user with high-level commands and classes for manipulating and visualizing data.
With SciPy an interactive Python session becomes a data-processing and system-prototyping
environment rivaling systems such as MATLAB, IDL, Octave, R-Lab, and SciLab.
The additional benefit of basing SciPy on Python is that this also makes a powerful
programming language available for use in developing sophisticated programs and specialized
applications. Scientific applications using SciPy benefit from the development of additional
modules in numerous niches of the software landscape by developers across the world.
Everything from parallel programming to web and data-base subroutines and classes have been
made available to the Python programmer. All of this power is available in addition to the
mathematical libraries in SciPy.

6
 CHAPTER 3

DESIGN

3.1 BLOCK DIAGRAM

The block diagram is typically used for a higher level, less detailed description aimed
more at understanding the overall concepts and less at understanding the details of
implementation.

3.1.1 BLOCK DIAGRAM

3.2 DATA FLOW DIAGRAMS:

Data flow diagram (DFD) is a graphical representation of “flow” of data through an

information system, modelling its process concepts. Often they are a preliminary step used to
create an overview of the system which can later be elaborated. DFD’s can also be used for the
visualization of data processing (structured design).
A DFD shows what kinds of information will be input to and output from the system, where the
data will come from and go to, and where the data will be stored. It doesn’t show information

7
about timing of processes, or information about whether processes will operate in sequence or
parallel. A DFD is also called as “bubble chart”.
DFD Symbols:
In the DFD, there are four symbols:
• A square defines a source or destination of system data.

• An arrow indicates dataflow. It is the pipeline through which the information flows.

• A circle or a bubble represents transforms dataflow into outgoing dataflow.

• An open rectangle is a store, data at reset or at temporary repository of data.

Dataflow: Data move in a specific direction from an origin to a destination.

Process: People, procedures or devices that use or produce (Transform) data. The physical
component is not identified.

Sources: External sources or destination of data, which may be programs, organizations or other
entity.

Data store: Here data is stored or referenced by a process in the system’s #

In our project, we had built the data flow diagrams at the very beginning of business
process modelling in order to model the functions that our project has to carry out and the
interaction between those functions together with focusing on data exchanges between processes.

8
3.2.1 Context level DFD:

A Context level Data flow diagram created using select structured systems analysis and
design method (SSADM). This level shows the overall context of the system and its operating
environment and shows the whole system as just one process. It does not usually show data
stores, unless they are “owned” by external systems, e.g. are accessed by but not maintained by
this system, however, these are often shown as external entities.

The Context level DFD is shown in fig.3.2.1

Figure 3.2.1 Context Level DFD for drowsiness detection

The Context Level Data Flow Diagram shows the data flow from the application to the database
and to the system.
3.2.2 Top level DFD:
A data flow diagram is that which can be used to indicate the clear progress of a business
venture. In the process of coming up with a data flow diagram, the level one provides an
overview of the major functional areas of the undertaking. After presenting the values for most
important fields of discussion, it gives room for level two to be drawn.

Figure 3.2.2 Top Level DFD for drowsiness detection

9
After starting and executing the application, training and testing the dataset can be done as
shown in the above figure
3.2.3 Detailed Level Diagram
This level explains each process of the system in a detailed manner. In first detailed level
DFD (Generation of individual fields): how data flows through individual process/fields in it are
shown.
In second detailed level DFD (generation of detailed process of the individual fields): how data
flows through the system to form a detailed description of the individual processes.

Figure 3.2.3 Detailed level DFD for drowsiness detection

After starting and executing the application,training the dataset is done by using cnn algorithm,
and then testing is done

3.3 UML DIAGRAMS:

The Unified Modelling Language (UML) is a Standard language for specifying, visualizing,
constructing and documenting the software system and its components. The UML [3] focuses on
the conceptual and physical representation of the system. It captures the decisions and
understandings about systems that must be constructed. A UML system is represented using five
different views that describe the system from distinctly different perspective. Each view [4] is
defined by a set of diagram, which is as follows.
• User Model View
i. This view represents the system from the users’ perspective.
ii. The analysis representation describes a usage scenario from the end-
users perspective.

10
 • Structural Model View
i. In this model the data and functionality are arrived from inside the system.
ii. This model view models the static structures.
• Behavioural Model View
It represents the dynamic of behavioral as parts of the system, depicting the
interactions of collection between various structural elements described in the user model
and structural model view.
• Implementation model View
In this the structural and behavioral as parts of the system are represented as
they are to be built
• Environmental Model View
In this the structural and behavioral aspects of the environment in which the system is
to be implemented are represented.
3.3.1 USECASE DIAGRAM:
Use case diagrams are one of the five diagrams in the UML for modeling the dynamic
aspects of the systems (activity diagrams, sequence diagram, state chart diagram, collaboration
diagram are the four other kinds of diagrams in the UML for modeling the dynamic aspects of
systems).use case diagram are central to modeling the behavior of the system, a sub-system, or
a class. Each one shows a set of use cases and actors and relations.

11
 Figure 3.3.1 USECASE DIAGRAM

3.3.2 SEQUENCE DIAGRAM:

Sequence diagram is an interaction diagram which is focuses on the time ordering of
messages. It shows a set of objects and messages exchanged between these objects. This
diagram illustrates the dynamic view of a system.
Sequence diagrams belong to a group of UML diagrams called Interaction Diagrams.Sequence
diagrams describe how objects interact over the course of time through an exchange of
messages. A single sequence diagram often represents the flow of events for a single use

Figure 3.3.2 SEQUENCE DIAGRAM

12
The figure 3.3.2 represents the sequence diagram for our application which relates the messages
or the actions to be performed between User and Application. The Life time of particular object
is shown in the Diagram.
3.3.3 COLLABORATION DIAGRAM:
Collaboration diagram is an interaction diagram that emphasizes the structural
organization of the objects that send and receive messages. Collaboration diagram and
sequence diagram are isomorphic.
A collaboration diagram is an interaction diagram that emphasizes the structural
organization of the objects that send and receive messages. Collaboration diagrams are
isomorphic, meaning that you can take one and transform it into the other.
The collaboration diagram is also called as communication diagram or interaction
diagram, is an illustration of the relationships and interactions among software objects in the
unified modelling language.
 It represents the structural organization of the system.
 Structural organization consists of objects and links.
 In the collaboration diagram, the method call sequence is indicated by some numbering
technique.

Figure 3.3.3 COLLABORATION DIAGRAM

This figure shows the Collaboration Diagram for this application. In this, users and
organizations can perform Collaboration of actions.

13
3.3.4 Activity Diagram:
The process flows in the system are captured in the activity diagram similar to a state diagram
an activity diagram also consists of activities, action, transitions
 Initial and final states and guard conditions
 Processes involving different use case are show in work flows.

Figure 3.3.4 ACTIVITY DIAGRAM

The figure shows the activity diagram of our application. In this, users and organizations
can perform actions.

14
 CHAPTER 4

IMPLEMENTATION

Technologies Used:
4.1: What is Python?
Python is an interpreter, high-level programming language for general-purpose
programmingby “Guido van Rossum” and first released in 1991, Python has a design philosophy
that emphasizes code readability, and a syntax that allows programmers to express concepts in
fewer lines of code, notably using significant whitespace. It provides constructs that enable clear
programming on both small and large scales. Python features a dynamic type system and
automatic memory management. It supports multiple programming paradigms, including object-
oriented, imperative, functional, procedural, and has a large and comprehensive standard library.
Python interpreters are available for many operating systems. Python, the reference
implementation of Python, is open source software and has a community-based development
model, as do nearly all of its variant implementations. Python is managed by the non-profit
Python Software Foundation.
Python is a general purpose, dynamic, high level and interpreted programming language. It
supports object-oriented programming approach to develop applications. It is simple and easy to
learn and provides lots of high level data structures.
• Windows XP
• Python Programming
• Open source libraries: Pandas, NumPy, SciPy, matplotlib, OpenCV
Python Versions
Python 2.0 was released on 16 October 2000 and had many major new features, including
a cycle-detecting, garbage collector, and support for Unicode. With this release, the development
process became more transparent and community-backed.
Python 3.0 (initially called Python 3000 or py3k) was released on 3 December 2008 after
a long testing period. It is a major revision of the language that is not completely backward-
compatible with previous versions. However, many of its major features have been back ported
to the Python 2.6.xand 2.7.x version series, and releases of Python 3 include the 2to3 utility,
which automates the translation of Python 2 code to Python 3.

15
Python 2.7's end-of-life date (a.k.a. EOL, sunset date) was initially set at 2015, then postponed to
2020 out of concern that a large body of existing code could not easily be forward-ported to
Python 3.
In January 2017, Google announced work on a Python 2.7 to go Tran’s compiler to improve
performance under concurrent workloads.
Python 3.6 had changes regarding UTF-8 (in Windows, PEP 528 and PEP 529) and Python
3.7.0b1 (PEP 540) adds a new "UTF-8 Mode" (and overrides POSIX locale).
WHY PYTHON?
 Python is a scripting language like PHP, Perl, and Ruby.
 No licensing, distribution, or development fees
 It is a Desktop application.
 Linux, windows
 Excellent documentation
 Thriving developer community
 For us job opportunity

4.2: MACHINE LEARNING

What is Machine Learning?
Machine Learning is an application of artificial intelligence (AI) that provides system
the ability to automatically learn and improve from experience without being explicitly
programmed. Machine learning focuses on the development of computer programs that can
access data and use it learn for themselves.
Why Machine Learning?
 It was born from pattern recognition and theory that computers can learn without being
programmed to specific tasks.
 It is a method of Data analysis that automates analytical model building.
Machine learning tasks are typically classified into two broad categories, depending on
whether there is a learning "signal" or "feedback" available to a learning system:
Supervised learning:The computer is presented with example inputs and their desired
outputs,given by a "teacher", and the goal is to learn a general rule that maps inputs to outputs.
Asspecial cases, the input signal can be only partially available, or restricted to special feedback:

16
Semi-supervised learning: The computer is given only an incomplete training signal: a
trainingset with some (often many) of the target outputs missing.
Active learning: The computer can only obtain training labels for a limited set of
instances(based on a budget), and also has to optimize its choice of objects to acquire labels for.
When used interactively, these can be presented to the user for labelling.
Reinforcement learning: Training data (in form of rewards and punishments) is given only
asfeedback to the program's actions in a dynamic environment, such as driving a vehicle or
playing a game against an opponent.
Unsupervised learning: No labels are given to the learning algorithm, leaving it on its own
tofind structure in its input. Unsupervised learning can be a goal in itself (discovering hidden
patterns in data) or a means towards an end (feature learning).
In regression, also a supervised problem, the outputs are continuous rather than discrete.

4.3 DEEP LEARNING:

Deep learning is an artificial intelligence function that imitates the workings of the
human brain in processing data and creating patterns for use in decision making. Deep learning is
a subset of machine learning in artificial intelligence (AI) that has networks capable of learning
unsupervised from data that is unstructured or unlabeled. Also known as deep neural learning or
deep neural network.

Convolutional neural network:

A convolutional neural network (CNN, or ConvNet) is a class of deep neural networks, most
commonly applied to analyzing visual imagery. CNNs use relatively little pre-processing
compared to other image classification algorithms. A convolutional neural network consists of an
input and an output layer, as well as multiple hidden layers. The hidden layers of a CNN
typically consist of convolutional layers, RELU layer i.e. activation function, pooling layers,
fully connected layers.

17
Convolution layer:

The objective of the Convolution Operation is to extract the high-level features such as edges,
from the input image. ConvNets need not be limited to only one Convolutional Layer.
Conventionally, the first ConvLayer is responsible for capturing the Low-Level features such as
edges, color, gradient orientation, etc. With added layers, the architecture adapts to the High-
Level features as well, giving us a network which has the wholesome understanding of images in
the dataset, similar to how we would.
Pooling Layer :

Similar to the Convolutional Layer, the Pooling layer is responsible for reducing the spatial size
of the Convolved Feature. This is to decrease the computational power required to
processthedata through dimensionality reduction. Furthermore, it is useful for extracting
dominantfeatureswhich are rotational and positional invariant, thus maintaining the process of
effectively training of the model.

There are two types of Pooling: Max Pooling and Average Pooling. Max Pooling returns
the maximum value from the portion of the image covered by the Kernel. On the other
hand, Average Pooling returns the average of all the values from the portion of the image covered
by the Kernel.

Max Pooling also performs as a Noise Suppressant. It discards the noisy activations altogether
and also performs de-noising along with dimensionality reduction. On the other hand, Average
Pooling simply performs dimensionality reduction as a noise suppressing mechanism. Hence, we
can say that Max Pooling performs a lot better than Average Pooling.

18
 Figure 4.3Types of Pooling

The Convolutional Layer and the Pooling Layer together form the i-th layer of a
Convolutional Neural Network. Depending on the complexities in the images, the number of
such layers may be increased for capturing low-levels details even further, but at the cost of more
computational power.

After going through the above process, we have successfully enabled the model to understand
the features. Moving on, we are going to flatten the final output and feed it to a regular Neural
Network for classification purposes.

RELU layer :

ReLU is the abbreviation of rectified linear unit, which applies the non-
saturating activationfunction.It effectively removes negative values from an activation map by
setting them to zero

Classification-Fully connected Layer(FC layer) :

Adding a Fully-Connected layer is a (usually) cheap way of learning non-linear combinations of

the high-level features as represented by the output of the convolutional layer. The Fully-
Connected layer is learning a possibly non-linear function in that space.

Now that we have converted our input image into a suitable form for our Multi-Level
Perceptions, we shall flatten the image into a column vector. The flattened output is fed to a
feed-forward neural network and back propagation applied to every iteration of training. Over a
series of epochs, the model is able to distinguish between dominating and certain low-level
features in images and classify them using the Softmax Classification technique.

CODE:
Code for drowsiness detection:

from scipy.spatial import distance as dist

from imutils.video import VideoStream
from imutils import face_utils
from threading import Thread
19
import numpy as np
import playsound
import argparse
import imutils
import time
import dlib
import cv2
def sound_alarm(path):
playsound.playsound(path)
def eye_aspect_ratio(eye):
A = dist.euclidean(eye[1], eye[5])
B = dist.euclidean(eye[2], eye[4])
C = dist.euclidean(eye[0], eye[3])
ear = (A + B) / (2.0 * C)
return ear
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--shape-predictor", required=True,
help="path to facial landmark predictor")
ap.add_argument("-a", "--alarm", type=str, default="",
help="path alarm .WAV file")
ap.add_argument("-w", "--webcam", type=int, default=0,
help="index of webcam on system")
args = vars(ap.parse_args())
EYE_AR_THRESH = 0.3
EYE_AR_CONSEC_FRAMES = 48
COUNTER = 0
ALARM_ON = False
print("[INFO] loading facial landmark predictor...")
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor(args["shape_predictor"])
(lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"]
(rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye"]
print("[INFO] starting video stream thread...")

20
vs = VideoStream(src=args["webcam"]).start()
time.sleep(1.0)
while True:
frame = vs.read()
frame = imutils.resize(frame, width=450)
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
rects = detector(gray, 0)
for rect in rects:
shape = predictor(gray, rect)
shape = face_utils.shape_to_np(shape)
leftEye = shape[lStart:lEnd]
rightEye = shape[rStart:rEnd]
leftEAR = eye_aspect_ratio(leftEye)
rightEAR = eye_aspect_ratio(rightEye)
ear = (leftEAR + rightEAR) / 2.0
leftEyeHull = cv2.convexHull(leftEye)
rightEyeHull = cv2.convexHull(rightEye)
cv2.drawContours(frame, [leftEyeHull], -1, (0, 255, 0), 1)
cv2.drawContours(frame, [rightEyeHull], -1, (0, 255, 0), 1)
if ear < EYE_AR_THRESH:
COUNTER += 1
if COUNTER >= EYE_AR_CONSEC_FRAMES:
if not ALARM_ON:
ALARM_ON = True
# check to see if an alarm file was supplied,
# and if so, start a thread to have the alarm
# sound played in the background
if args["alarm"] != "":
t = Thread(target=sound_alarm,
args=(args["alarm"],))
t.deamon = True
t.start()
# draw an alarm on the frame

21
 cv2.putText(frame, "DROWSINESS ALERT!", (10, 30),
cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)

# otherwise, the eye aspect ratio is not below the blink

# threshold, so reset the counter and alarm
else:
COUNTER = 0
ALARM_ON = False

# draw the computed eye aspect ratio on the frame to help

# with debugging and setting the correct eye aspect ratio
# thresholds and frame counters
cv2.putText(frame, "EAR: {:.2f}".format(ear), (300, 30),
cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0, 0, 255), 2)
# show the frame
cv2.imshow("Frame", frame)
key = cv2.waitKey(1) & 0xFF
# if the `q` key was pressed, break from the loop
if key == ord("q"):
break
# do a bit of cleanup
cv2.destroyAllWindows()
vs.stop()

Code For Yawn Detection

import cv2
import numpy as np
import dlib
from playsound import playsound
path="C:/Users/NIHARIKA/Desktop/Yawn_Detector/shape_predictor_68_face_landmarks.dat"
predictor=dlib.shape_predictor(path)
detector=dlib.get_frontal_face_detector()
def get_landmarks(im):

22
rects=detector(im,1)#image and no.of rectangles to be drawn
if len(rects)>1:
print("Toomanyfaces")
return np.matrix([0])
if len(rects)==0:
print("Toofewfaces")
return np.matrix([0])
return np.matrix([[p.x,p.y] for p in predictor(im,rects[0]).parts()])
def place_landmarks(im,landmarks):
im=im.copy()
for idx,point in enumerate(landmarks):
pos=(point[0,0],point[0,1])
cv2.putText(im,str(idx),pos,fontFace=cv2.FONT_HERSHEY_SCRIPT_SIMPLEX,fontScale=0.
3,color=(0,255,255))
cv2.circle(im,pos,3,color=(0,255,255))
return im
def upper_lip(landmarks):
top_lip=[]
for i in range(50,53):
top_lip.append(landmarks[i])
for j in range(61,64):
top_lip.append(landmarks[j])
top_lip_point=(np.squeeze(np.asarray(top_lip)))
top_mean=np.mean(top_lip_point,axis=0)
return int(top_mean[1])
def low_lip(landmarks):
lower_lip=[]
for i in range(65,68):
lower_lip.append(landmarks[i])
for j in range(56,59):
lower_lip.append(landmarks[j])
lower_lip_point=(np.squeeze(np.asarray(lower_lip)))
lower_mean=np.mean(lower_lip_point,axis=0)

23
 return int(lower_mean[1])
def decision(image):
landmarks=get_landmarks(image)
if(landmarks.all()==[0]):
return -10#Dummy value to prevent error
top_lip=upper_lip(landmarks)
lower_lip=low_lip(landmarks)
distance=abs(top_lip-lower_lip)
return distance
cap=cv2.VideoCapture(0)
yawns=0
while(True):
ret,frame=cap.read()
if(ret==True):

landmarks=get_landmarks(frame)
if(landmarks.all()!=[0]):
l1=[]
for k in range(48,60):
l1.append(landmarks[k])
l2=np.asarray(l1)
lips=cv2.convexHull(l2)
cv2.drawContours(frame, [lips], -1, (0, 255, 0), 1)
distance=decision(frame)
if(distance>21): #Use distance according to your convenience
yawns=yawns+1

cv2.putText(frame,"Yawn Count: "+str(yawns),

(50,100),fontFace=cv2.FONT_HERSHEY_SIMPLEX,fontScale=1.0,color=(0,255,255))

if(yawns>1):

24
playsound('C://Users/NIHARIKA/Music/abc.mp3')
cv2.imshow("Subject Yawn Count",frame)
if cv2.waitKey(1)==13:
break
else:
continue
cap.release()
cv2.destroyAllWindow

25
 CHAPTER 5

TESTING

5.1 Testing Objectives:

• To make sure whether system meets user requirements or not.
• To make sure that during the operation, incorrect input, processing and output will
be detected.
• To see that when correct inputs are fed to the system the outputs are correct.
• To verify that the controls incorporated in the same system as intended.
5.1.1 White-box and black-box testing:
White box and black box testing are terms used to describe the point of view a test
engineer takes when designing test cases. Black box [7] being an external view of the test object
and white box being an internal view. Software testing is partly intuitive, but largely systematic.
Good testing involves much more than just running the program a few times to see whether it
works. Thorough analysis of the program under test, backed by a broad knowledge of testing
techniques and tools are prerequisites to systematic testing. Software Testing is the process of
executing software in a controlled manner; in order to answer the question “Does this software
behave as specified?” Software testing is used in association with Verification and Validation.
Verification is the checking of or testing of items, including software, for conformance and
consistency with an associated specification. Software testing is just one kind of verification,
which also uses techniques as reviews, inspections, walk-through. Validation is the process of
checking what has been specified is what the user actually wanted.

 Validation: Are we doing the right job?

 Verification: Are we doing the job right?
In order to achieve consistency in the Testing style, it is imperative to have and follow a set of
testing principles. This enhances the efficiency of testing within SQA team members and thus
contributes to increased productivity. The purpose of this document is to provide overview of the
testing, plus the techniques.
At SDEI, 3 levels of software testing is done at various SDLC phases:

26
  Unit Testing: In which each unit (basic component) of the software is tested to verify
thatthe detailed design for the unit has been correctly implemented
 Integration testing: In which progressively larger groups of tested software component
scorresponding to elements of the architectural design are integrated and tested until the
software works as a whole.
 System testing: In which the software is integrated to the overall product and tested
toshow that all requirements are met

A further level of testing is also done, in accordance with requirements:

 Acceptance testing: Upon which the acceptance of the complete software is based. The
clients often do this.
 Regression testing: Is used to refer the repetition of the earlier successful tests to
ensurethat changes made in the software have not introduced new bugs/side effects.

5.2 Test Cases:

 To make sure whether system meets user requirements or not.
 To make sure that during the operation, incorrect input, processing and output will be
detected.
 To see that when correct inputs are fed to the system the outputs are correct.
 To verify that the controls incorporated in the same system as intended.
5.2.1 Positive Test Cases:
 Positive testing determines the application works as expected. If an error is encountered
during positive testing, the test fails.
 The positive flow of the functionality must be considered.
 Must have the positive perception to verify whether the requirements are justified.
 By using this application, security can be easily provided in many sectors.
5.2.2 Negative Test Cases:
 Negative testing ensures the application can gracefully handle invalid input
orunexpected user behaviour.

27
  For example, if a user tries to type a letter in a numeric field, the correct behaviour in this
case would be to display the “Incorrect data type, please enter a number” message.
 The purpose of negative testing is to detect such situations and prevent applications from
crashing.
 Also, negative testing helps you improve the quality of your application and find its weak
points.

Test case: Detecting the drowsiness

Test case: capture the image from webcam.
Test Description: Detecting the drowsiness
Expected Results: Alert the user when he/she is in drowsy state
Result: Success.

28
 CHAPTER 6

RESULTS

After performing testing we get certain results. The results obtained from testing as
shown below. Results of the system can be expressed and evaluated in terms of output screen.
The output screen can be used to show the objectives set at the beginning are achieved at the end.

6.1 For drowsiness detection

Figure:6.1.1 Screenshot for Eye Detection

Figure 6.1.2: Screenshot for Drowsiness Detection

29
 Figure 6.1.3 Screenshot for Drowsiness Alert

6.2 For Yawn Detection

Figure:6.2.1 Screenshot for Mouth Detection

30
Figure 6.2.2 Screenshot for Yawn Detection

31
 CHAPTER 7

LIMITATIONS

 With poor lighting conditions even though face is easily detected, sometimes the system
is unable to detect the eyes.
 The distance between face and webcam is not at optimum range then certain problems are
arising.
 Even though the driver wears glasses the system fails to detect eyes which is the most
significant drawback of our system.

32
 CHAPTER 8
CONCLUSION

 The system proposed in this analysis provides accurate detection of driver fatigue.
 This application can be implemented in the real time to reduce traffic accidents rate due
to drowsy drivers and it can also help drivers to stay awake when driving by giving a
warning when the driver is sleepy
.

33
 CHAPTER 9

FUTURE SCOPE AND ENHANCEMENTS

We could enhance this application by adding features like..

 In the real time driver drowsiness detection system it is required to slow down a vehicle
automatically when drowsiness level crosses a certain limit.
 It monitors the level of drowsiness continuously and when this level exceeds a certain
value a signal is generated which controls the hydraulic braking system of the vehicle.

34
 BIBLIOGRAPHY
[1].Modern Control systems, Dorf&Bishop–Princeton Hall, ISBN: 0133-70-285, 2008

[2].A Practitioners Guide to Software, Roger S.Pressman8th Edition –McGraw-Hill, ISBN: 978-
0-07-8022128, 2014

[3].The Unified Modeling Language User Guide, Low Price Edition Grady Booch, James
Rumbaugh, IvarJacob, ISBN: 81-7808-769-5, 91, 1997

[4].The Elements of UMLTM 2.0, Scott Ambler-Cambridge University Press

Newyork@2005,ISBN: 978-0-07-52616-7-82, 2005

[5].Software Testing, CemKaner, James beach, BretPettiehord,ISBN: 978-0-471-120-940,2001

[6].A Practitioner’s Approach Roger S.Pressman, Software Engineering, 3rdEdition,ISBN: 978-
007-126782-3,482, 2014

[7]. Black_Box Testing: Techniques for Functional Testing of Software and Systems
Boris Beizer - Wiley Publications, ISBN: 978-0-471-120-940, 1995.

35
36