SUMMER INTERNSHIP PROJECT REPORT
On
“Mood detector – An AI Based Android Application”
Submitted in partial fulfilment of the requirements
For the award of the degree of
Bachelors of Computer
Applications
Submitted by:
HIMANSHU ACHARYA
Enrolment No. A50504821001
Under the guidance of
Dr. Shalini Bhaskar Bajaj
Head of Department
CSE department
Department of Computer Science & Engineering
Amity Institute of Information & Technology
AMITY UNIVERSITY GURGAON, HARYANA
November, 2022
i
Department of Computer Science and Engineering
Amity Institute of Information Technology
DECLARATION
I, Himanshu Acharya, student of Bachelor of Computer Applications hereby declare that the
project entitled ‘Mood detector – An AI Based Android Application’ which is submitted
by us to department of Computer Science & Engineering, Amity Institute of Information
Technology, Amity University Haryana, in partial fulfilment of the requirement for the award
of the degree of Bachelor of Computer Applications, has not been previously formed the
basis for the award of any degree, diploma or other similar title or recognition.
Haryana
Date: 2 NOVEMBER 2022 HIMANSHU ACHARYA
ii
Department of Computer Science and Engineering
Amity Institute of Information Technology
CERTIFICATE
This is to certify that Himanshu Acharya (Enrolment No: A50504821001), student of
Bachelor of Computer Applications (III Semester), Department of Computer Science &
Engineering, AIIT, Amity University Haryana, has done his internship project entitled
“Mood detector – An AI Based Android Applications” under my guidance and
supervision during ‘July 2022- August 2022’.
The work was satisfactory. He has shown complete dedication and devotion to the given
project.
Date: 2nd November 2022 (Dr. Shalini Bhaskar Bajaj)
Head of department
Computer Science & Engineering
AIIT, Amity University, Haryana
Head
Department of Computer Science & Engineering
Amity School of Engineering and Technology
Amity University Haryana, Gurgaon
iii
Date: 16th August 2022
TO WHOMSOEVER IT MAY CONCERN
This is to certify that Mr. Himanshu Acharya of BCA, of Amity University,
Gurugram has successfully completed his Internship during 29th June– 12th
August 2022, under the guidance of Mr. Maneesh Sharma and Mr. Yuvraj Singh
Shekhawat.
The project was undertaken by him in Android and Flask development, Mood
detector .
The Project on evaluation fulfils all the stated criteria and the student’s findings
are his original work.
We hereby certify his work good to the best of my knowledge.
For SunArc Technologies Pvt. Ltd.
Yours sincerely,
Authorized Signatory.
This is Computer Generated, Hence Signature is not required
ACKNOWLEDGEMENT
Acknowledgement is not a mere obligation but epitome of humility and ineptness to all those
who have helped in the completion of this project. I am thankful to Dr. Shalini Bhaskar Bajaj,
head of department, AIIT for her constant guidance and encouragement provided in this
endeavour. I also thank my parents for their continued support, understanding and patience
without whose support and understanding this endeavour would never been fruitful. I also
thank all my friends for helping me out in completing this project and helping me in solving
various problems encountered during the progress of this project.
Himanshu Acharya
Enrolment Number- A50504821001
v
ABSTRACT
Artificial intelligence is growing field and unlocks ease to daily life it is demand of new era
A.I. is used every ware today either for household or in industries. So, we worked on a topic
which is not so famous yet i.e., mood detection for industrial use. Example, many employees
work in industries and offices so it is necessary to track their emotion on daily basics this data
can help managers to keep check on employee behaviour example how much of work load is
fine for them are they overpressure or not, and many more. It can be very impractical and costly
to employee a human for this so, a powerful and accurate programme is need of hour and this
is it. The app not only provide statical data of emotion tracked but also recognises the person
to keep data of every employee separate and through face recognition it opens up with extra
feature of marking attendance on regular basics. A mood detector is an A.I. based application
which can detect your emotion through an image passed. Think of a virtual friend who always
knows your feelings and understand you how good it is. Application askes for your images to
train the database at initial run ones you are done with training you can go for emotion and face
recognition voila you are done with this. Android alone cannot perform such a complex task
that’s why we need a very powerful programming language i.e., python. The application works
on two different ends client and server. Data gathering and output of data is shown on client
side that is android application and image processing is done at server side.
vi
List of Figures
Figure No. Title Page No.
Figure 3.1 Application architecture 14
Figure 4. 1 Technologies used 15
Figure 5. 1 MTCNN Model working 20
Figure 5. 2 Resizing of image 21
Figure 5. 3 Flow Chart for Training the Model /Register A New Person 21
Figure 5. 4 Flow Chart for Training the Model /Register A New Person 22
(Backend)
Figure 5. 5 Flow Chart for Testing the Model / Checking the Identity 23
Figure 5. 6 Face net working 24
Figure 5. 7 Face net flow chart 24
Figure 5. 8 Face net data structure 25
Figure 5. 9 Analysing the facial expressions 26
Figure 5. 10 Classification in SVM Model 26
Figure 5. 11 Classification in KNN Model 27
Figure 6. 1 Admin panel 28
Figure 6. 2 Admin options 29
Figure 6. 3 Enter the details of employee 29
Figure 6. 4 Taking image with guidelines 30
Figure 6. 5 Guiding the user 30
Figure 6. 6 Validating the real person 31
vii
Figure 6. 7 Start processing the image 31
Figure 6. 8 Showing the output 32
Figure 6. 9 Taking 10 image continually 33
Figure 6. 10 Display the pie chart 33
Figure 6. 11 Connect to server 34
viii
CONTENTS
Declaration ii
Certificate iii
Internship certificate iv
Acknowledgement v
Abstract vi
List Of Figures vii
Chapters
1. INTRODUCTION 11
1.1 Project overview 11
1.2 Project Background 11
2. EMOTION ARTIFICIAL INTELIGENCE 12
3. DESIGN OF PROJECT 13
3.1 Hardware requirement 13
3.2 Software requirement 13
3.3 Application Architecture 14
4. TECHNOLOGIES USED 15
4.1 Server-side programming 15
4.1.1 Python 15
4.1.2 Face detection library 16
4.1.3 Plotly Library 16
4.1.4 Flask 16
4.1.5 Visual Studio code 16
4.2 Client-Side Programming 17
4.2.1 Kotlin 17
4.2.2 Volley 17
4.2.3 Google ML kit 18
4.2.4 Camerax API 18
4.2.5 Android Studio 18
5. METHODOLOGIES 19
5.1 Models used 19
ix
5.1.1 MTCNN 19
5.1.2 Face net 22
5.1.3 FER model 24
5.2 Machine learning algorithms 25
6. RESULT 27
6.1 Register new employee 27
6.2 Identify the person with emotion 30
6.3 Analyse emotion 32
6.4 Adding IP address 33
7. CONCLUSION AND FUTURE SCOPE 34
7.1 Conclusion 34
7.2 Future Scope 34
REFERENCES 35
x
Chapter 1
INTRODUCTION
1.1 Project overview
My project named mood detector -an A.I. based android application. The aim of this project
is to create a powerful android application that can detect non-verbal expressions of human
and can recognise their faces it can recognise different human behaviours such as happy, sad,
angry, surprised and neutral. It can also recognise different registered faces and announce
their names The application is built with vast models and libraries such as Google ml kit and
face net offered by Google and python libraries such as flask for API creation. Android plays
a major role in this project as the input data (images) captured through a mobile device and
then send to server through APIs
1.2 Project Background
Facial features and points play an important in emotion detection; the application captures
different landmarks of face to recognise the face and emotion. Training is the first phrase for
model creation, in this phase data is introduced to the model it takes twenty-five images of a
person from different angles so that model can be trained properly and predict the result more
accurately. Testing is the phrase where the accuracy of the model is checked it is always
recommended to check the accuracy of the model as the result is always dependent on it.
Final stage includes the recognition of face in this phase it is necessary to check that the face
against the camera is of a real person on a static photograph so the user is asked to blink his
eyes to validate. This is done using the Google ml kit -a ml library provided by the Google
for mobile devices. Once the validation is done the application starts processing the image
and shows the result obtained.
11
Chapter 2
EMOTION ARTIFICIAL INTELLIGENCE
Emotion AI, also known as Affective Computing or Artificial Emotional Intelligence, is one
of the emerging areas of Artificial Intelligence where machines study non-verbal cues of
humans like body language, facial expressions, gestures, and tonality of voice to detect their
emotional state. This technology has started to find tremendous use in areas like advertising,
customer service, healthcare, and others. The accuracy of emotion recognition is usually
improved when it combines the analysis of human expressions from multimodal forms such
as texts, physiology, audio, or video. Different emotion types are detected through the
integration of information from facial expressions, body movements and gestures, and
speech. Technology is said to contribute to the emergence of the so-called emotional or
emotive Internet. The existing approaches in emotion recognition to classify certain emotion
types can be generally classified into three main categories: knowledge-based techniques,
statistical methods, and hybrid approaches.
12
Chapter 3
DESIGN OF PROJECT
3.1 Hardware Requirement
3.1.1 Windows Requirement
• x86 64-bit CPU (Intel / AMD architecture)
• 4 GB RAM
• 40 MB free disk space
3.1.2 Mobile Requirement
• 1 GB ram
• 15 MB free disk space
3.2 Software Requirement
3.2.1 Windows Requirement
• Windows 7,10 or 11
• VS code
• Python plugins
3.2.2 Mobile Requirement
• Android operating system
• Minimum SDK 21
• Minimum android version 5.0 (Lollipop)
13
3.3 Application Architecture
The application uses powerful languages like python and vast datasets like Facenet and
MTCNN so it needs high computation power and recourses which is not possible on every
mobile device and hence needed a capable system in background so the application is divided
in two parts i.e., client-side and server-side
Figure 3. 1 Application Architecture
14
Chapter 4
TECHNOLOGIES USED
This project used several different technologies and models to accomplish the task at both
server and client side as shown in the flow chart below.
Figure 4. 1 Technologies used
4.1 Server-Side programming
4.1.1 Python
Python is a high-level, interpreted, general-purpose programming language. Its design
philosophy emphasizes code readability with the use of significant indentation. Python
is dynamically-typed and garbage-collected. It supports multiple programming paradigms,
including structured (particularly procedural), objectoriented and functional programming. It
is often described as a "batteries included" language due to its comprehensive standard
library. Guido van Rossum began working on Python in the late 1980s as a successor to
the ABC programming language and first released it in 1991 as Python 0.9.0. Python 2.0 was
released in 2000 and introduced new features such as list comprehensions, cycle-
detecting garbage collection, reference counting, and Unicode support. Python 3.0, released
in 2008, was a major revision that is not completely backward-compatible with earlier
versions. Python 2 was discontinued with version 2.7.18 in 2020. Python consistently ranks
as one of the most popular programming languages.
15
4.1.2 Face recognition library
Recognize and manipulate faces from Python or from the command line with the world’s
simplest face recognition library. Built using dlib’s state-of-the-art face recognition built with
deep learning. The model has an accuracy of 99.38% on the labelled Faces in the
Wild benchmark. This also provides a simple face recognition command line tool that lets
you do face recognition on a folder of images from the command line!
4.1.3 Plotly library
The Plotly Python library is an interactive, open-source plotting library that supports over 40
unique chart types covering a wide range of statistical, financial, geographic, scientific, and
3-dimensional use-cases. Built on top of the Plotly JavaScript library
(plotly.js), 16lotly enables Python users to create beautiful interactive web-based
visualizations that can be displayed in Jupyter notebooks, saved to standalone HTML files, or
served as part of pure Python-built web applications using Dash. The 16lotly Python library
is sometimes referred to as “16lotly.py” to differentiate it from the JavaScript library. Thanks
to deep integration with our Kaleido image export utility, 16lotly also provides great support
for non-web contexts including desktop editors (e.g., QtConsole, Spyder, PyCharm) and
static document publishing (e.g., exporting notebooks to PDF with high-quality vector
images).
4.1.4 Flask
Flask is a web framework; it’s a Python module that lets you develop web applications easily.
It’s having a small and easy-to-extend core: it’s a micro framework that doesn’t include an
ORM (Object Relational Manager) or such features. It does have many cool features like
URL routing, template engine. It is a WSGI web app framework. Flask is a web application
framework written in Python. It was developed by Armin Ronacher, who led a team of
international Python enthusiasts called Poocco. Flask is based on the Werkzeg WSGI toolkit
and the Jinja2 template engine. Both are Poocco projects.
4.1.5 Visual Studio code
Visual Studio Code, also commonly referred to as VS Code, is a source-code editor made
by Microsoft for Windows, Linux and macOS. Features include support
for debugging, syntax highlighting, intelligent code completion, snippets, code refactoring,
16
and embedded Git. Users can change the theme, keyboard shortcuts, preferences, and
install extensions that add additional functionality. In the Stack Overflow 2021 Developer
Survey, Visual Studio Code was ranked the most popular developer environment tool, with
70% of 82,000 respondents reporting that they use it. It provides a terminal to run system
queries so it is easy to use flask with vs code
4.2 Client-Side Programming
4.2.1 Kotlin
Kotlin is a cross platform, statically typed, general-purpose language programming language
with type inference. Kotlin is designed to interoperate fully with java; it allows its syntax to
be more concise. Kotlin mainly targets the JVM, but also compiles to JavaScript. It also
allows native development in android that’s why Google has officially declared it as official
language for android development Language development costs are borne by Jet Brains,
while the Kotlin Foundation protects the Kotlin trademark.
4.2.2 Volley
Volley is an HTTP library that makes networking for Android apps easier and most
importantly, faster. Volley is available on GitHub. Volley offers the following benefits such
as automatic scheduling of network requests. Multiple concurrent network connections.
Transparent disk and memory response caching with standard HTTP cache coherence.
Support for request prioritization. Cancellation request API. You can cancel a single request,
or you can set blocks or scopes of requests to cancel. Ease of customization, for example, for
retry and back off. Strong ordering that makes it easy to correctly populate your UI with data
fetched asynchronously from the network. Debugging and tracing tools. Volley excels at
RPC-type operations used to populate a UI, such as fetching a page of search results as
structured data. It integrates easily with any protocol and comes out of the box with support
for raw strings, images, and JSON. By providing built-in support for the features you need,
Volley frees you from writing boilerplate code and allows you to concentrate on the logic that
is specific to your app. Volley is not suitable for large download or streaming operations,
since Volley holds all responses in memory during parsing. For large download operations,
consider using an alternative like Download Manager.
17
4.2.3 Google ML kit
ML Kit is a mobile SDK that brings Google’s on-device machine learning expertise to
Android and iOS apps. Use our powerful yet easy to use Vision and Natural Language APIs
to solve common challenges in your apps or create brand-new user experiences. All are
powered by Google’s best-in-class ML models and offered to you at no cost. ML Kit’s APIs
all run on-device, allowing for real-time use cases where you want to process a live camera
stream for example. This also means that the functionality is available offline. It provides
several APIs such as barcode scanning, face detection, image labelling, object detection and
tracking, Text recognition and tracking, digital ink recognition, pose detection, Selfie
segmentation
4.2.4 Camerax API
CameraX is a Jetpack library, built to help make camera app development easier. For new
apps, we recommend starting with CameraX. It provides a consistent, easy-to-use API that
works across the vast majority of Android devices, with backward-compatibility to Android
5.0 (API level 21). CameraX emphasizes use cases, which allow you to focus on the task you,
need to get done instead of managing device-specific nuances. Most common camera use
cases are supported:
➢ Preview: View an image on the display.
➢ Image analysis: Access a buffer seamlessly for use in your algorithms, such as to pass to
ML Kit.
➢ Image capture: Save images.
➢ Video capture: Save video and audio.
4.2.5 Android studio
Android Studio is the official Integrated Development Environment (IDE) for Android app
development, based on IntelliJ IDEA. On top of IntelliJ's powerful code editor and developer
tools, Android Studio offers even more features that enhance your productivity when building
Android apps, such as a flexible Gradle-based build system, a fast and feature-rich emulator,
a unified environment where you can develop for all Android devices, apply Changes to push
code and resource changes to your running app without restarting your app, code templates
and GitHub integration to help you build common app features and import sample code,
extensive testing tools and frameworks, lint tools to catch performance, usability
18
Chapter 5
Methodologies
5.1 Models used
The Application includes different phrases and at each phrase it uses different models and
algorithms. The models and algorithms used are as follows
5.1.1 MTCNN
Multi-task Cascaded Convolutional Networks (MTCNN) is a framework developed as a
solution for both face detection and face alignment. The process consists of three stages of
convolutional networks that are able to recognize faces and landmark location such as eyes,
nose, and mouth. The paper proposes MTCNN as a way to integrate both tasks (recognition
and alignment) using multi-task learning. In the first stage it uses a shallow CNN to quickly
produce candidate windows. In the second stage it refines the proposed candidate windows
through a more complex CNN. And lastly, in the third stage it uses a third CNN, more
complex than the others, to further refine the result and output facial landmark positions.
Figure 5. 1 MTCNN Model working
19
Figure 5. 2 Resizing of image
Figure 5. 3 Flow Chart for Training the Model /Register A New Person
20
Figure 5. 4 Flow Chart for Training the Model /Register A New Person (Backend)
21
Figure 5. 5 Flow Chart for Testing the Model / Checking the Identity
5.1.2 Face Net
FaceNet is a deep neural network used for extracting features from an image of a person’s
face. It was published in 2015 by Google researchers Schroff et al. FaceNet takes an image of
the person’s face as input and outputs a vector of 128 numbers which represent the most
important features of a face. In machine learning, this vector is called embedding. Why
embedding? Because all the important information from an image is embedded into this
vector. Basically, FaceNet takes a person’s face and compresses it into a vector of 128
numbers. Ideally, embeddings of similar faces are also similar.
22
Figure 5. 6 Face net working
Figure 5. 7 Face net flow chart
23
Figure 5. 8 Face net data structure
5.1.3 FER Data set model
Facial emotion recognition (FER) is significant for human-computer interaction such as
clinical practice and behavioural description. Accurate and robust FER by computer models
remains challenging due to the heterogeneity of human faces and variations in images such as
different facial pose and lighting. Among all techniques for FER, deep learning models,
especially Convolutional Neural Networks (CNNs) have shown great potential due to their
powerful automatic feature extraction and computational efficiency. In this work, we achieve
the highest single-network classification accuracy on the FER2013 dataset. We adopt the
VGGNet architecture, rigorously fine-tune its hyperparameters, and experiment with various
optimization methods. To our best knowledge, our model achieves state-of-the-art single-
network accuracy of 73.28 % on FER2013 without using extra training data.
24
Figure 5. 9 Analysing the facial expressions
5.2 Machine Learning algorithms
Two different algorithms are used to train the model i.e., KNN and SVM
➢ SVM
Support Vector Machine (SVM) is a supervised machine learning algorithm used for both
classification and regression. Though we say regression problems as well its best suited
for classification. The objective of SVM algorithm is to find a hyper plane in an N-
dimensional space that distinctly classifies the data points
Figure 5. 10 Classification in SVM Model
25
➢ KNN
K-nearest neighbours (KNN) works on the principle that, the data points that have
minimum distance in feature space from our new data point. And K is the number of such
data points we consider in our implementation of the algorithm. Therefore, distance
metric and K value are two important considerations while using the KNN algorithm.
Euclidean distance is the most popular distance metric. You can also use Hamming
distance, Manhattan distance, Minkowski distance as per your need.
Figure 5. 11 Classification in KNN Model
26
Chapter 6
Results
Results are shown in procedural sequence
6.1 Register new employee
Figure 6. 1 Admin panel
27
Figure 6. 2 Admin options
Figure 6. 3 Enter details of employee
28
Figure 6. 4 Taking image with guidelines
Figure 6. 5 Guiding the user
29
6.2 Identify the person with emotion
Figure 6. 6 Validating the real person
Figure 6. 7 Start processing the image
30
Figure 6. 8 Showing the output
31
6.3 Analyse emotion
Figure 6. 9 Taking 10 image continually
Figure 6. 10 Display the pie chart
32
6.4 Adding IP address
Figure 6. 11 Connect to the server
33
Chapter 7
CONCLUSION AND FUTURE SCOPE
7.1 Conclusion
To conclude, this project has been made with Kotlin, Google libraries and Python libraries
the aim to create an android application which can recognise a person and at the same time
analyse his or her emotion. The android application presented on this paper is a very
fundamental model of the actual project with a few features however the additional features
and functions may be introduced in future work of this project, in this project API calling,
data transfer between server and client is determined. The project is made using different
open-source libraries which can easily accommodate changes in future. The object-oriented
programming approach used in project make it flexible to integrate new feature in future
without disturbing the current structure. It only works on human faces and hence provide
more sense of security.
7.2 Future Scope
This mood detector application has enormous scope in future, this application can be used as
an authentication at office entry, other use case is to analyse the behaviour or response of the
client in meeting or may be used as alert alarm at some restricted area. We can use this
application as attendance register so that it can automatically mark the attendance of an
employee when he or she enters the premises.
34
REFERENCES
• https://developer.android.com/training/volley?hl=es-419
• https://developer.android.com/training/camerax
• https://developers.google.com/ml-kit
• https://stackoverflow.com/questions/17571759/how-do-you-use-the-android-volley-api
• https://www.geeksforgeeks.org/volley-library-in-android/
• https://developers.google.com/ml-kit/vision/face-detection/android
35
