JARVIS
Project
submitted in partial fulfillment of the
requirements for the award of the degree of
BACHELOR OF TECHNOLOGY (B. TECH)
INFORMATION TECHNOLOGY
Submitted By:
Vasu Garg(180060103084)
Varun Kapruwan (180060103083)
Suryansh Sharma (180060103077)
Under the Supervision of
Mr. Kamal Kant Verma,Associate Professor
College of Engineering Roorkee, Roorkee
DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING
COLLEGE OF ENGINEERING ROORKEE
UTTRAKHAND TECHNICAL UNIVERSITY, DEHRADUN
May, 2022
Jarvis
Project
submitted in partial fulfillment of the
requirements for the award of the degree of
BACHELOR OF TECHNOLOGY (B. TECH)
Submitted by:
Varun Kapruwan 180060103083
Vasu Garg 180060103084
Suryansh Sharma 180060103077
Under the supervision of
Ms. Neha Arora Dr. Kamal Kant Verma
(Assistant Professor) (Associate Professor )
DEPARTMENT OF COMPUTER SCIENCE ENGINEERING
COLLEGE OF ENGINEERING ROORKEE, ROORKEE
ROORKEE-247667 (UTTARAKHAND) INDIA
MAY, 2022
DECLARATION
We hereby certify that the work which is being presented in the report entitles “Jarvis”, in the
partial fulfillment of the requirements for the award of the degree of Bachelor of Technology in
Information Technology, submitted to the department of Computer Science Engineering,
College of Engineering Roorkee, affiliated to Uttarakhand Technical University, Dehradun
(India), is an authentic record of our own work carried out during session 2021-2022, under the
supervision of Mr. Kamal Kant Verma, Associate Professor, Department of Computer Science
& Engineering Roorkee, Roorkee (247667).
The matter embodied in this project report has not been submitted by us for the award of any
other degree or diploma.
Dated: 25, may 2022
Vasu Garg(180060103084)
Varun Kapruwan (180060103083)
Suryansh Sharma (180060103077)
CERTIFICATE
This is to certify that the project work “ Jarvis”, is a Bonafide record of work done by Varun
Kapruwan, Vasu Garg & Suryansh Sharma under our guidance in partial fulfilment of the
requirement for the B.Tech project as per record of Uttarakhand Technical University,
Dehradun in the academic session 2021-22 at College of Engineering Roorkee,
Roorkee(247667), Uttarakhand(India).
Signature
Mr. Kamal Kant Verma ,Associate Professor,
Computer Sci. & Engineering Department
College of Engineering Roorkee, Roorkee
Date: 25, May 2022
ABSTRACT
As we know Python is an emerging language so it becomes easy to write a script for Voice
Assistant in Python. The instructions for the assistant can be handled as per the requirement
of user. Speech recognition is the process of converting speech into text. This is commonly
used in voice assistants like Alexa, Siri, etc. In Python there is an API called
SpeechRecognition which allows us to convert speech into text. It was an interesting task
to make my own assistant. It became easier to send emails without typing any word,
Searching on Google without opening the browser, and performing many other daily tasks
like playing music, opening your favorite IDE with the help of a single voice command. In
the current scenario, advancement in technologies are such that they can perform any task
with same effectiveness or can say more effectively than us. By making this project, I
realized that the concept of AI in every field is decreasing human effort and saving time.
Functionalities of this project include:
1. It can send emails.
2. It can read PDF.
3. It can send text on WhatsApp.
4. It can open command prompt, your favorite IDE, notepad etc.
5. It can play music.
6. It can do Wikipedia searches for you.
7. It can open websites like Google, YouTube, etc., in a web browser.
8. It can give weather forecast.
9. It can give desktop reminders of your choice.
10. It can have some basic conversation.
Now the basic question arises in mind that how it is an AI? The virtual assistant that I have
created is like if it is not an A.I, but it is the output of a bundle of the statement. But
fundamentally, the mail purpose of A.I machines is that it can perform human tasks with the
same efficiency or even more efficiently than humans. It is a fact that my virtual assistant is
not a very good example of A.I., but it is an A.I.
ACKNOWLEDGEMENT
It is our privilege to express our deep sense of gratitude and indebtedness to our guide
Mr. Kamal Kant Verma , Department of Computer Science & Engineering, College of
Engineering Roorkee, Roorkee for pointing us to the topic of “Jarvis”, as well as providing us
all the necessary guidance and inspirational support throughout the project.
We also thankfully acknowledge our Head of Department Information
Technology, Dr Taresh Singh who gave us their valuable suggestions from their busy
schedule.
Our heartiest gratitude to our parents, family members and mentors for their continuous
encouragement and blessings, best wishes and moral support which was a constant source of
assurance, guidance, strength and inspiration to us.
Date: 25, May 2022
Name of Students:
Vasu Garg(180060103084)
Varun Kapruwan (180060103083)
Suryansh Sharma (180060103077)
Declaration I
Certificate II
Abstract III
Acknowledgement IV
Contents V
Chapter 1
Introduction 2-3
1.1. Present System 3
1.2. Proposed system 3
Chapter 2
System Design 4-5
2.1. Data Flow 4
Chapter 3
Software details 5-7
3.1. Py Charm 5
3.2. PYQT5 for live GUI 5
3.3. Python Libraries 6
Chapter 4
Implementation work details 8-9
4.1. Real life application 8
4.2. Data Implementation and program execution 9
Chapter 5
Source code 10-11
Chapter 6
Input/Output screenshots 11-15
Chapter 7
System Testing 16-17
7.1. Functionality 16
7.2. Usability 17
7.3. Security 17
7.4. Stability 17
Chapter 8
Conclusion 18-19
8.1. Limitations 18
8.2. Future scope 18
JARVIS
1
Chapter 1: Introduction
1. INTRODUCTION
Artificial Intelligence when used with machines, it shows us the capability of
thinking like humans. In this, a computer system is designed in such a way that typically
requires interaction from human. As we know Python is an emerging language so it
becomes easy to write a script for Voice Assistant in Python. The instructions for the
assistant can be handled as per the requirement of user. Speech recognition is the Alexa,
Siri, etc. In Python there is an API called Speech Recognition which allows us to
convert speech into text. It was an interesting task to make my own assistant. It became
easier to send emails without typing any word, Searching on Google without opening
the browser, and performing many other daily tasks like playing music, opening your
favorite IDE with the help of a single voice command. In the current scenario,
advancement in technologies are such that they can perform any task with same
effectiveness or can say more effectively than us. By making this project, I realized that
the concept of AI in every field is decreasing human effort and saving time.
As the voice assistant is using Artificial Intelligence hence the result that it is
providing are highly accurate and efficient. The assistant can help to reduce human
effort and consumes time while performing any task, they removed the concept of
typing completely and behave as another individual to whom we are talking and asking
to perform task. The assistant is no less than a human assistant but we can say that this
is more effective and efficient to perform any task. The libraries and packages used to
make this assistant focuses on the time complexities and reduces time.
The functionalities include , It can send emails, It can read PDF, It can send text
on WhatsApp, It can open command prompt, your favorite IDE, notepad etc., It can play
music, It can do Wikipedia searches for you, It can open websites like Google,
YouTube, etc., in a web browser, It can give weather forecast, It can give desktop
reminders of your choice. It can have some basic conversation.
Tools and technologies used are PyCharm IDE for making this project, and I
created all py files in PyCharm. Along with this I used following modules and libraries
in my project. pyttsx3, SpeechRecognition, Datetime, Wikipedia, Smtplib, pywhatkit,
pyjokes, pyPDF2, pyautogui, pyQt etc. I have created a live GUI for interacting with
the JARVIS as it gives a design and interesting look while having the conversation.
2
1.1 PRESENT SYSTEM
We are familiar with many existing voice assistants like Alexa, Siri, Google
Assistant, Cortana which uses concept of language processing, and voice recognition.
They listens the command given by the user as per their requirements and performs that
specific function in a very efficient and effective manner.
As these voice assistants are using Artificial Intelligence hence the result that
they are providing are highly accurate and efficient. These assistants can help to reduce
human effort and consumes time while performing any task, they removed the concept
of typing completely and behave as another individual to whom we are talking and
asking to perform task. These assistants are no less than a human assistant but we can
say that they are more effective and efficient to perform any task. The algorithm used
to make these assistant focuses on the time complexities and reduces time.
But for using these assistants one should have an account (like Google account
for Google assistant, Microsoft account for Cortana) and can use it with internet
connection only because these assistants are going to work with internet connectivity.
They are integrated with many devices like, phones, laptops, and speakers etc.
1.2 PROPOSED SYSTEM
It was an interesting task to make my own assistant. It became easier to send
emails without typing any word, Searching on Google without opening the browser,
and performing many other daily tasks like playing music, opening your favorite IDE
with the help of a single voice command. Jarvis is different from other traditional voice
assistants in terms that it is specific to desktop and user does not need to make account
to use this, it does not require any internet connection while getting the instructions to
perform any specific task.
The IDE used in this project is PyCharm. All the python files were created in
PyCharm and all the necessary packages were easily installable in this IDE. For this
project following modules and libraries were used i.e. pyttsx3, SpeechRecognition,
Datetime, Wikipedia, Smtplib, pywhatkit, pyjokes, pyPDF2, pyautogui, pyQt etc. I
have created a live GUI for interacting with the JARVIS as it gives a design and
interesting look while having the conversation.
With the advancement JARVIS can perform any task with same effectiveness or
can say more effectively than us. By making this project, I realized that the concept of
AI in every field is decreasing human effort and saving time. Functionalities of this
project include, It can send emails, It can read PDF, It can send text on WhatsApp, It
can open command prompt, your favorite IDE, notepad etc., It can play music, It can
do Wikipedia searches for you, It can open websites like Google, YouTube, etc., in a
web browser, It can give weather forecast, It can give desktop reminders of your
choice. It can have some basic conversation.
3
Chapter 2: System Design
2.1. DATA FLOW
The data flow for JARVIS is as follow:
• Live GUI for interaction will appear on screen.
Start
• It will take input through voice commands related to
the task which is required to be done.
Input
• It will perform the required task for the user like
opening notepad, searching on browser, sending
Perform mails, playing songs etc.
• It keeps on asking for the command from user
until the user say "Quit". Once the user say
Exit "Quit", it exits.
Figure 2.1 Data flow for JARVIS
The system is designed using the concept of Artificial Intelligence and with
the help of necessary packages of Python. Python provides many libraries and
packages to perform the tasks, for example pyPDF2 can be used to read PDF. The
details of these packages are mentioned in Chapter 3 of this report.
The data in this project is nothing but user input, whatever the user says, the
assistant performs the task accordingly. The user input is nothing specific but the list
of tasks which a user wants to get performed in human language i.e. English.
4
Chapter 3: Software Details
The IDE used in this project is PyCharm. All the python files were created in
PyCharm and all the necessary packages were easily installable in this IDE. For this
project following modules and libraries were used i.e. pyttsx3, SpeechRecognition,
Datetime, Wikipedia, Smtplib, pywhatkit, pyjokes, pyPDF2, pyautogui, pyQt etc. I
have created a live GUI for interacting with the JARVIS as it gives a design and
interesting look while having the conversation.
3.1. PYCHARM
It is an IDE i.e. Integrated Development Environment which has many features
like it supports scientific tools(like matplotlib, numpy, scipy etc) web frameworks
(example Django,web2py and Flask) refactoring in Python, integrated python
debugger, code completion, code and project navigation etc. It also provides Data
Science when used with Anaconda.
Figure 3.1 PyCharm IDE
3.2. PYQT5 FOR LIVE GUI
PyQt5 is the most important python binding. It contains set of GUI widgets.
PyQt5 has some important python modules like QTWidgets, QtCore, QtGui, and
QtDesigner etc.
5
Figure 3.2 PyQT5
3.3. PYTHON LIBRARIES
In JARVIS following python libraries were used:
3.3.1. pyttsx3: It is a python library which converts text to speech.
3.3.2. SpeechRecognition: It is a python module which converts speech to
text.
3.3.3. pywhatkit: It is python library to send WhatsApp message at a
particular time with some additional features.
3.3.4. Datetime: This library provides us the actual date and time.
3.3.5. Wikipedia: It is a python module for searching anything on
Wikipedia.
3.3.6. Smtplib: Simple mail transfer protocol that allows us to send mails
and to route mails between mail servers.
3.3.7. pyPDF2: It is a python module which can read, split, merge any PDF.
3.3.8. Pyjokes: It is a python libararies which contains lots of interesting
jokes in it.
3.3.9. Webbrowser: It provides interface for displaying web-based
documents to users.
6
3.3.10. Pyautogui: It is a python libraries for graphical user interface.
3.3.11. os: It represents Operating System related functionality.
3.3.12. sys: It allows operating on the interpreter as it provides access to the
variables and functions that usually interact strongly with the interpreter.
Figure 3.3 Imported Modules
7
Chapter 4: Implementation Work Details
JARVIS, a desktop assistant is a voice assistant that can perform many daily tasks of
desktop like playing music, opening your favorite IDE with the help of a single voice
command. Jarvis is different from other traditional voice assistants in terms that it is
specific to desktop and user does not need to make account to use this, it does not require
any internet connection while getting the instructions to perform any specific task.
4.1. REAL LIFE APPLICATION
4.1.1. Saves time: JARVIS is a desktop voice assistant which works on the voice
command offered to it, it can do voice searching, voice-activated device control and
can let us complete a set of tasks.
4.1.2. Conversational interaction It makes it easier to complete any task as it
automatically do it by using the essential module or libraries of Python, in a
conversational interaction way. Hence any user when instruct any task to it, they feel
like giving task to a human assistant because of the conversational interaction for
giving input and getting the desired output in the form of task done.
4.1.3. Reactive nature: The desktop assistant is reactive which means it know human
language very well and understand the context that is provided by the user and gives
response in the same way, i.e. human understandable language, English. So user finds
its reaction in an informed and smart way.
4.1.4. Multitasking: The main application of it can be its multitasking ability. It can
ask for continuous instruction one after other until the user “QUIT” it.
4.1.5. No Trigger phase: It asks for the instruction and listen the response that is
given by user without needing any trigger phase and then only executes the task.
4.2. DATA IMPLEMENTATION AND PROGRAM EXECUTION
As the first step, install all the necessary packages and libraries. The command used
to install the libraries is “pip install” and then import it. The necessary packages included
are as follows:
4.2.1. LIBRARIES AND PACKAGES
4.2.2.1. pyttsx3: It is a python library which converts text to speech.
4.2.2.2. SpeechRecognition: It is a python module which converts speech to
text.
8
4.2.2.3. pywhatkit: It is python library to send WhatsApp message at a
particular time with some additional features.
4.2.2.4. Datetime: This library provides us the actual date and time.
4.2.2.5. Wikipedia: It is a python module for searching anything on
Wikipedia.
4.2.2.6. Smtplib: Simple mail transfer protocol that allows us to send mails
and to route mails between mail servers.
4.2.2.7. pyPDF2: It is a python module which can read, split, merge any
PDF.
4.2.2.8. Pyjokes: It is a python libraries which contains lots of interesting
jokes in it.
4.2.2.9. Webbrowser: It provides interface for displaying web-based
documents to users.
4.2. 2.10. Pyautogui: It is a python librariy for graphical user interface.
4.2.2.11. os: It represents Operating System related functionality.
4.2.2.12. sys: It allows operating on the interpreter as it provides access to
the variables and functions that usually interact strongly with the interpreter.
4.2.2. FUNCTIONS
4.2.2.1. takeCommand(): The function is used to take the command as input
through microphone of user and returns the output as string.
4.2.2.2. wishMe(): This function greets the user according to the time like
Good Morning, Good Afternoon and Good Evening.
4.2.2.3. taskExecution(): This is the function which contains all the
necessary task execution definition like sendEmail(), pdf_reader(), news()
and many conditions in if condition like “open google”, “open notepad”,
“search on Wikipedia” ,”play music” and “open command prompt” etc.
9
Chapter 5: Source Code
JARVIS source code is hosted on Github, the most widely used version management system in the world
.
There shall be proper Documentation available on its Official Github repository
for making further development easy and we aim to release our virtual assistant as
an Open Source Software where modifications and contributions by the
community are warmly welcomed.
Link to Github Repository:
https://github.com/varunkapruwan/Virtual-Assistant
10
Chapter 6: Input/Output Screenshot
Figure 6.1 Live GUI of JARVIS
Figure 6.2 Input for Google search
11
Figure 6.3 Output for Google search
Figure 6.4 Input to send Email
Figure 6.5 Output to send Email
12
Figure 6.6 Input for YouTube search
Figure 6.7 Output for YouTube search
Figure 6.8 Input to play music
13
Figure 6.9 Output to play music
Figure 6.10 Input to open cmd
Figure 6.11 Output to open cmd
14
Figure 6.12 Input and output for Wikipedia search
Figure 6.13 Input to open Microsoft Office
Figure 6.14 Output to open Microsoft Office
15
Chapter 7: System testing
The system testing is done on fully integrated system to check whether the
requirements are matching or not. The system testing for JARVIS desktop assistant
focuses on the following four parameters:
7.1. FUNCTIONALITY
In this we check the functionality of the system whether the system performs
the task which it was intended to do. To check the functionality each function was
checked and run, if it is able to execute the required task correctly then the system
passes in that particular functionality test. For example to check whether JARVIS
can search on Google or not, as we can see in the figure 7.1, user said “Open
Google”, then Jarvis asked, ”What should I search on Google?” then user said,
“What is Python”, Jarvis open Google and searched for the required input.
Figure 7.1 Input through voice commands
Figure 7.2 Output
16
7.2. USABILITY
Usability of a system is checked by measuring the easiness of the
software and how user friendly it is for the user to use, how it responses to
each query that is being asked by the user.
It makes it easier to complete any task as it automatically do it by using
the essential module or libraries of Python, in a conversational interaction
way. Hence any user when instruct any task to it, they feel like giving task to
a human assistant because of the conversational interaction for giving input
and getting the desired output in the form of task done.
The desktop assistant is reactive which means it know human
language very well and understand the context that is provided by the user
and gives response in the same way, i.e. human understandable language,
English. So user finds its reaction in an informed and smart way.
The main application of it can be its multitasking ability. It can ask for
continuous instruction one after other until the user “QUIT” it. It asks for the
instruction and listen the response that is given by user without needing any
trigger phase and then only executes the task.
7.3. SECURITY
The security testing mainly focuses on vulnerabilities and risks. As
JARVIS is a local desktop application, hence there is no risk of data
breaching through remote access. The software is dedicated to a specific
system so when the user logs in, it will be activated.
7.4. STABILITY
Stability of a system depends upon the output of the system, if the
output is bounded and specific to the bounded input then the system is said to
be stable. If the system works on all the poles of functionality then it is
stable.
17
Chapter 8: Conclusion
JARVIS is a very helpful voice assistant without any doubt as it saves time of the user
by conversational interactions, its effectiveness and efficiency. But while working on this
project, there were some limitations encountered and also realized some scope of
enhancement in the future which are mentioned below:
8.1. LIMITATIONS
8.1.1. Security is somewhere an issue, there is no voice command encryption in this
project.
8.1.2. Background voice can interfere
8.1.3. Misinterpretation because of accents and may cause inaccurate results.
8.1.4. JARVIS cannot be called externally anytime like other traditional assistants
like Google Assistant can be called just by saying, “Ok Google!”
8.2. SCOPE FOR FUTURE WORK
8.2.1. Integerating a Machine Learning based Software developer salary predictor application.
8.2.2. JARVIS android app can also be developed.
8.2.3. Integerating a crypto currency trend analyser with Jarvis
8.2.4. Adding functionality to predict winner of IPL match befiore the match starts.
18
19
