Emotion Recognition
Emotion Recognition
A Project Report
In partial fulfilment of the requirements for the award of the degree
Bachelor of Engineering
in
Computer science and
Engineering
(ISO9001:2015)
SDF Building, Module #132, Ground Floor, Salt Lake City, GP Block, Sector V,
Kolkata, West Bengal 700091
Title of the Project: Emotion
Recognition
Debpriya
Final santra Project Report 24th July,
2024
Date: Date:
For Office Use Only Mr. Joyjit Guha Biswas
Project Proposal Evaluator
Declaration
We hereby declare that the project work being presented in the project proposal
entitled “Emotion Recognition” in partial fulfilment of the requirements for the
award of the degree of Bachelor of Engineering at Ardent Computech PVT.
LTD, Saltlake, Kolkata, West Bengal, is an authentic work carried out under the
guidance of Mr. Joyjit Guha Biswas. The matter embodied in this project work
has not been submitted elsewhere for the award of any degree of our knowledge
and belief.
Date: 24/07/2024
Debpriya santra Ashutosh Rana Roumadeep Guchait Ishika suri shaw Falguni Roy
Signature
Ardent Computech Pvt. Ltd (An ISO 9001:2015 Certified)
SDF Building, Module #132, Ground Floor, Salt Lake City, GP Block, Sector V,
Kolkata, West Bengal 700091
Certificate
Guide / Supervisor
Project Engineer
Ardent Computech Pvt. Ltd (An ISO 9001:2015 Certified)
SDF Building, Module #132, Ground Floor, Salt Lake City, GP Block, Sector V,
Kolkata, West Bengal 700091
Acknowledgement
Overview
History of Python
Environment Setup
Basic Syntax
Variable Types
Functions
Modules
Packages
Artificial Intelligence
o Deep Learning
o Neural Networks
o Machine Learning
Machine Learning
o Supervised and Unsupervised Learning
o NumPy
o SciPy
o Scikit-learn
o Pandas
o Regression Analysis
o Matplotlib
o Clustering
o
Emotion Recognition
Overview:
Python is a high-level, interpreted, interactive and object-oriented scripting
language. Python is designed to be highly readable. It uses English keywords
frequently where as other languages use punctuation, and has fewer syntactical
constructions than other languages.
History of Python:
Python was developed by Guido van Rossum in the late eighties and early
nineties at the National Research Institute for Mathematics and Computer
Science in the Netherlands. Python is derived from many other languages,
including ABC, Modula-3, C, C++, Algol-68, Small Talk, UNIX shell, and
other scripting languages. Python is copyrighted. Like Perl, Python source
code is now available under the GNU General Public License (GPL). Python
is now maintained by a core development team at the institute, although Guido
van Rossum still holds a vital role in directing its progress.
Features of Python:
Easy-to-learn: Python has few Keywords, simple structure and clearly
defined syntax. This allows a student to pick up the language quickly.
Easy-to-Read: Python code is more clearly defined and visible to the
eyes. Easy -to-Maintain: Python's source code is fairly easy-to-maintain.
A broad standard library: Python's bulk of the library is very portable and
cross platform compatible on UNIX, Windows, and Macintosh.
Interactive Mode: Python has support for an interactive mode which allows
interactive testing and debugging of snippets of code.
Portable: Python can run on the wide variety of hardware platforms and has
the same interface on all platforms.
Extendable: You can add low level modules to the python interpreter. These
modules enables programmers to add to or customize their tools to be more
efficient.
Databases: Python provides interfaces to all major commercial databases.
GUI Programming: Python supports GUI applications that can be created and
ported to many system calls, libraries, and windows systems, such as Windows
MFC, Macintosh, and the X Window system of Unix.
Scalable: Python provides a better structure and support for large programs
than shell scripting.
Apart from the above-mentioned features, Python has a big list of good
features, few are listed below:
It support functional and structured programming methods as well as
OOP.
It can be used as a scripting language or can be compiled to byte code
for building large applications.
It provides very high level dynamic data types and supports dynamic
type checking.
It supports automatic garbage collections.
It can be easily integrated with C, C++, COM, Active X, CORBA and
JAVA.
Environment Setup:
To set up the environment for the Emotion Recognition project, the following tools and
libraries are required:
Python 3
OpenCV
NumPy
DeepFace
Type the following text at the Python prompt and press the Enter –
Hello, Python!
Python Identifiers:
A Python identifier is a name used to identify a variable, function, class, module
or other object. An identifier starts with a letter A to Z or a to z or an underscore
(_) followed by zero or more letters, underscores and digits (0 to 9).Python does
not allow punctuation characters such as @, $, and % within identifiers. Python
is a case sensitive proming language.
Python Keywords:
The following list shows the Python keywords. These are reserved words and
you cannot use them as constant or variable or any other identifier names. All
the Python keywords contain lowercase letters only.
For example:
And, exec, not Assert, finally, orBreak, for, pass Class, from, print ,continue,
global, raisedef, if, return, del, import, tryelif, in, while else, is, with ,except,
lambda, yield.
Lines & Indentation:
Python provides no braces to indicate blocks of code for class and function
definitions or flow control. Blocks of code are denoted by line indentation,
which is rigidly enforced.
The number of spaces in the indentation is variable, but all statements within
the block must be indented the same amount. For example –
if True: print "True"
else:
print "False"
Many programs can be run to provide you with some basic information
about how they should be run. Python enables you to do this with -h −
$ python-h
usage: python [option]...[-c cmd|-m mod | file |-][arg]...
Variable Types:
Variables are nothing but reserved memory locations to store values. This
means that when you create a variable you reserve some space in memory.
Python variables do not need explicit declaration to reserve memory space. The
declaration happens automatically when you assign a value to a variable. The
equal sign (=) is used to assign values to variables.
counter=10 # An integer assignment
weight=10.60 # A floating point
name="Ardent" # A string
Multiple Assignment:
The data stored in memory can be of many types. For example, a person's age is
stored as a numeric value and his or her address is stored as alphanumeric
characters. Python has five standard data types −
String
List
Tuple
Dictionary
Number
Sometimes, you may need to perform conversions between the built-in types.
To convert between types, you simply use the type name as a function.
There are several built-in functions to perform conversion from one data type to
another.
Functions:
Defining a Function:
def changeme(mylist):
"This changes a passed list into this function"
mylist.append([1,2,3,4]);
print"Values inside the function: ",mylist
return
mylist=[10,20,30];
changeme(mylist);
print"Values outside the function: ",mylist
Here, we are maintaining reference of the passed object and appending values in
the same object. So, this would produce the following result −
Variables that are defined inside a function body have a local scope, and those
defined outside have a global scope . For Example-
Modules:
A module allows you to logically organize your Python code. Grouping related
code into a module makes the code easier to understand and use. A module is a
Python object with arbitrarily named attributes that you can bind and reference .
The Python code for a module named aname normally resides in a file named aname.py.
Here's an example of a simple module, support.py
You can use any Python source file as a module by executing an import
statement in some other Python source file. The import has the following
syntax –
Consider a file Pots.py available in Phone directory. This file has following line
of source code −
def Pots():
print "I'm Pots Phone"
Similar way, we have another two files having different functions with the same
name as above –
To make all of your functions available when you've imported Phone, you need
to put explicit import statements in init .py as follows −
from Pots import Pots
from Isdn import Isdn
from G3 import
Numpy:
NumPy is a library for the Python programming language, adding support for
large, multi-dimensional arrays and matrices, along with a large collection of
high-level mathematical functions to operate on these arrays. The ancestor of
NumPy, Numeric, was originally created by Jim Hugunin.
NumPy targets the CPython reference implementation of Python, which is a
non-optimizing bytecode interpreter. Mathematical algorithms written for this
version of Python often run much slower than compiled equivalents.
Using NumPy in Python gives functionality comparable to MATLAB since they
are both interpreted, and they both allow the user to write fast programs as long
as most operations work on arrays or matrices instead of scalars.
Scipy:
Scientific computing tools for Python
SciPy refers to several related but distinct entities:
On this base, the SciPy ecosystem includes general and specialised tools for
data management and computation, productive experimentation, and high-
performance computing. Below, we overview some key packages, though there
are many more relevant packages.
IPython, a rich interactive interface, letting you quickly process data and test
ideas.
The Jupyter notebook provides IPython functionality and more in your web
browser, allowing you to document your computation in an easily
reproducible form.
Cython extends Python syntax so that you can conveniently build C
extensions, either to speed up critical code or to integrate with C/C++
libraries.
Dask, Joblib or IPyParallel for distributed processing with a focus on
numeric data.
Quality assurance:
nose, a framework for testing Python code, being phased out in preference
for pytest.
numpydoc, a standard and library for documenting Scientific Python
libraries.
Pandas:
In computer programming, pandas is a software library written for the Python
programming language for data manipulation and analysis. In particular, it offers
data structures and operations for manipulating numerical tables and timeseries.
It is free software released under the three-clause BSD license. "Panel data", an
econometrics term for multidimensional, structured data sets.
Library features:
You will need numpy and scipy to run these files. The code for this project is
available at https://github.com/jameslyons/python_speech_features .
Supported features:
(rate,sig) = wav.read("file.wav")
print(fbank_feat[1:3,:])
OS: Miscellaneous operating system interfaces
Extensions peculiar to a particular operating system are also available through the
os module, but using them is of course a threat to portability.
All functions accepting path or file names accept both bytes and string objects,
and result in an object of the same type, if a path or file name is returned.
The pickle module implements binary protocols for serializing and de-
serializing a Python object structure. “Pickling” is the process whereby a
Python object hierarchy is converted into a byte stream, and “unpickling” is the
inverse operation, whereby a byte stream (from a binary file or bytes-like object)
is converted back into an object hierarchy. Pickling (and unpickling) is
alternatively known as “serialization”, “marshalling,” 1 or “flattening”; however,
to avoid confusion, the terms used here are “pickling” and “unpickling”.
Operator: Standard operators as functions
The functions fall into categories that perform object comparisons, logical
operations, mathematical operations and sequence operations.
This module creates temporary files and directories. It works on all supported
platforms. TemporaryFile, NamedTemporaryFile, TemporaryDirectory, and
SpooledTemporaryFile are high-level interfaces which provide automatic cleanup
and can be used as context managers. mkstemp() and mkdtemp() are lower-level
functions which require manual cleanup.
All the user-callable functions and constructors take additional arguments which
allow direct control over the location and name of temporary files and directories.
Files names used by this module include a string of random characters which
allows those files to be securely created in shared temporary directories. To
maintain backward compatibility, the argument order is somewhat odd; it is
recommended to use keyword arguments for clarity.
Introduction to Machine Learnning:
Machine learning is a field of computer science that gives computers the ability
to learn without being explicitly programmed.
Evolved from the study of pattern recognition and computational learningtheory
in artificial intelligence, machine learning explores the study and construction of
algorithms that can learn from and make predictions on data.
Machine learning is a field of computer science that gives computers the ability
to learn without being explicitly programmed.
Machine learning tasks are typically classified into two broad categories,
depending on whether there is a learning "signal" or "feedback" available to a
learning system:-
Supervised Learning:
Supervised learning is the machine learning task of inferring a function from
labeled training data.[1] The training data consist of a set of training examples. In
supervised learning, each example is a pair consisting of an input object (typically
a vector) and a desired output value.
Unsupervised Learning:
Unsupervised learning is the machine learning task of inferring a function
to describe hidden structure from "unlabelled" data (a classification or
categorization is not included in the observations). Since the examples
given tothe learner are unlabelled, there is no evaluation of the accuracy
of the structurethat is output by the relevant algorithm—which is one way
of distinguishing unsupervised learning from supervised learning and
reinforcement learning.
A central case of unsupervised learning is the problem of density estimation
in statistics, though unsupervised learning encompasses many other
problems (andsolutions) involving summarizing and explaining key features
of the data.
y = b0 + b1 * x1 + b2 * x2 + ... + bn * xn
where:
Once the coefficients are determined, the model can be used to make
predictions on new data by simply plugging the feature values into the linear
equation.
Algorithm:
Data Collection
Data Formatting
Model Selection
Training
Testing
Emotion Recognition is a crucial and interesting task in the field of computer vision,
impacting various sectors such as healthcare, security, and human-computer interaction. In
this project, we aimed to create an Emotion Recognition model using a combination of
OpenCV for face detection and DeepFace for emotion analysis. This project serves as an
introductory exploration of applying machine learning to emotion recognition.
Emotion Recognition is a complex task that involves various facial features and
expressions. While using pre-trained models like those in DeepFace can provide effective
results, the challenge lies in accurately interpreting subtle differences in expressions and
emotions.
1. Environment Setup: Install the necessary libraries and tools, including OpenCV,
NumPy, and DeepFace.
2. Face Detection: Use OpenCV to capture video from the webcam and detect faces in
real-time using a pre-trained Haar Cascade classifier.
3. Emotion Prediction: Use the DeepFace library to analyze the detected faces and
predict the emotions.
4. Real-Time Processing: Integrate face detection and emotion prediction to process
video frames in real-time and display the results.
5. Model Evaluation: Evaluate the performance of the Emotion Recognition system by
testing it with various facial expressions and ensuring accurate emotion classification.
6. Result Visualization: Visualize the detected faces and predicted emotions by
drawing bounding boxes and emotion labels on the video frames.
7. Future Enhancements: Explore additional features and improvements, such as
handling multiple faces, improving prediction accuracy, and integrating with other
systems.
Actual codes for Emotion Recognition:
Doing the imports, initializing the camera and setting the window size:
Loading the model, capture frame by frame,converting frame to gray scale image , drawing
rectangle around the face, extracting the face from the image:
Future Scope:
The current Emotion Recognition project lays a solid foundation for real-time emotion
recognition using OpenCV and DeepFace. However, there are several areas for potential
improvement and expansion:
Improved Training Data: Utilize a larger and more diverse dataset for training to
improve the accuracy and robustness of Emotion Recognition across different
ethnicities, age groups, and lighting conditions.
Advanced Models: Incorporate more sophisticated deep learning models, such as
Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs), to
capture complex patterns and temporal dependencies in facial expressions.
Multiple Faces: Extend the system to handle multiple faces in a single frame,
ensuring accurate Emotion Recognition for each individual in group settings.
Interaction Analysis: Analyze interactions between individuals in a group to
understand collective emotional dynamics and social behaviors.
6. Application Integration
Data Privacy: Ensure the system adheres to data privacy regulations and ethical
guidelines, protecting user data and maintaining transparency in how the data is used.
Bias Mitigation: Address and mitigate biases in Emotion Recognition models to
ensure fair and equitable performance across different demographic groups.
8. Cross-Platform Deployment
References
OpenCV Documentation: https://opencv.org/
DeepFace Documentation: https://github.com/serengil/deepface
Python Documentation: https://www.python.org/doc/
THANK YOU