2017Third International Conference on Science Technology Engineering & Management (ICONSTEM)

VOICE CONTROLLED HOME AUTOMATION

SYSTEM USING NATURAL LANGUAGE PROCESSING
(NLP) AND INTERNET OF THINGS (IoT)
Mrs. Paul Jasmin Rani1*, Jason Bakthakumar2, Praveen Kumaar.B3, Praveen Kumaar.U4 and
Santhosh Kumar5
1
Asst. Prof, Department of Computer Science, Rajalakshmi Institute of Technology, India; pauljasminrani.l@ritchennai.edu.in
2
UG, Department of Computer Science, Rajalakshmi Institute of Technology, India; jasonbakthakumar@gmail.com
3
UG, Department of Computer Science, Rajalakshmi Institute of Technology, India; praveenkumaarb@gmail.com
4
UG, Department of Computer Science, Rajalakshmi Institute of Technology, India; dashingcoder@gmail.com
5
UG, Department of Computer Science, Rajalakshmi Institute of Technology, India; santhoshsandy1295@gmail.com

Abstract: The primary objective of our project is to

construct a fully functional voice based Home
automation system that uses Internet of Things, Artificial Introduction:
Intelligence and Natural Language Processing (NLP) to
provide a cost-effective, efficient way to work together Our project is built upon the on the concept of
with home appliances. There are many smart home Internet of Things. We have determined to
solutions in the market that aim to automate the basic implement seamless integration of all the
operations of these home appliances using various appliances in the home via a central console. The
technologies such as GSM (Global System for Mobile), project greatly changes the way in which we
NFC (Near-Field Communication) etc. However, most of communicate with our home appliances. It also
these systems focus on mimicking the basic operation of
reduces the need for any personal contact with any
the electrical switch. Our project aims at providing a
of them as it delivers a wholesome experience of
fully automated voice based solution that our users can
rely on, to perform more than just switching on/off the wireless, voice controlled system. We plan to
appliances. The user sends a command through speech to implement this project with the aid of Arduino
the mobile device, which interprets the message and boards with the capability to perform IoT (Internet
sends the appropriate command to the specific appliance. of Things) operations. Installing the boards in every
We plan on implementing four basic home appliances as appliance of the house will allow us to establish
a “Proof-of-Concept” for this project which includes Fan, real-time communication with them via Wi-Fi. We
Light, Coffee Machine and Door Alarms. The voice plan on using Arduino MK1000, known as
command given by the user is interpreted by the mobile
Genuino MK1000 outside of the United States to
device using Natural Language processing. The mobile
implement this project.
device acts as a central console; it determines what
operation must be completed by which appliance to
Arduino MKR1000 was primarily designed for
fulfill the user’s request. The central console might
likewise be either a desktop application, web application
practical and economic solution for innovators
or a smart phone application as nearly all of the data seeking to create Wi-Fi enabled projects with
transferred can be processed by the cloud. However, for minimal pre-requisite knowledge in the field. It is
the convenience of the user and increased mobile built upon the Atmel ATSAMW25 SoC (System on
capabilities we will be using a smart phone in this Chip), which is a member of the SmartConnect
project. The appliances are associated with the mobile family of Atmel Wireless devices, that specializes
device through an Arduino Board that establishes the in IoT (Internet of Things) projects and devices.
concept of Internet of Things. The Arduino Boards are
interfaced with the appliances and programmed in a The ATSAMW25 is primarily composed of three
manner that they respond to mobile inputs. distinct blocks
Keywords: Home Automation System, Arduino Boards,
• SAMD21 Cortex-M0+ 32bit low power ARM
Natural Language Processing (NLP), Artificial
MCU
Intelligence (AI), Internet of Things (IoT).
• WINC1500 low power 2.4GHz IEEE® 802.11

978-1-5090-4855-7/17/$31.00 ©2017 IEEE ƒ‰‡͵͸ͺ

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY TIRUCHIRAPALLI. Downloaded on June 04,2024 at 06:31:53 UTC from IEEE Xplore. Restrictions apply.
 2017Third International Conference on Science Technology Engineering & Management (ICONSTEM)

b/g/n Wi-Fi limited power consumption. The Wi-Fi module of

• ECC508 Crypto Authentication the MKR1000 supports certificate SHA-256. Most
of the Arduino sketches can be developed in any
The ATSAMW25 comprises of a single 1*1 stream programming language of choice with a compiler
PCB Antenna. The design of the board entails a that yields binary code. The Atmel Corporation
Lithium Polymer charging circuit which helps the provides a standard development environment for
Arduino/Genuino MKR1000 to operate on battery its microcontrollers, AVR Studio and the latest
power or peripheral battery, charging the Lithium version of Atmel Studio, which can be used for
Polymer battery when it is running on a peripheral programming Arduino.
source. The switching of a single source to another
is done instantaneously and automatically. Our project uses a mobile application to interact
with these Wi-Fi signals emitted from these
appliances. The application sends out instructions
It has a proper 32 bit computational power similar to the Arduino boards and also respond to the
to Arduino Zero board, the usual array of I/O feedback from them. When any command is sent
interfaces, power efficient Wi-Fi along with a from a remote place, the Arduino boards receive
Crypto chip for reliable data transmission, and the them from the Wi-Fi router which they are
simplicity of utilization of the Arduino Software connected to when they are connected through to
(IDE) for code programming and development. All the mobile device.
of the above listed features makes this board the
most preferred option for the emerging IoT battery- The Internet of Things (IoT) has immense potential
powered projects such as ours. Its condensed form to change many of our daily activities, routines and
factor makes it even simpler to operate and aids in behaviors. The persistent characteristic of the
design. internet sources means that a large amount of data
relating to almost every facet of human activity,
both communal and private will be produced,
transmitted, collected and stored. Consequently the
integrity and confidentiality of the communicated
data as well as the authentication of the services
offering that particular data is crucial. Hence
security is a critical functionality of the IoT. So we
provide an authentication process in the mobile
application that involves the use of a username and
a password.

The natural language processing (NLP) is a field of

computer science that helps us to infer what the
user is trying to say through his voice commands.
The NLP in our project gives the user the freedom
to interact with the home appliances with his/her
own voice and normal language rather than
complicated computer commands.

Fig: Arduino/Genuino MKR1000 Natural Language Processing (NLP) is a field

of computer science, artificial intelligence, and
It has a USB port that can be used to provide computational linguistics that helps computer
power (5V) to the board. The Arduino/Genuino system understand and respond to commands given
MKR1000 has the capability to run with or without in natural (human) language. NLP is can be
the Lithium Polymer battery involved and has very categorized in the field of human-computer

978-1-5090-4855-7/17/$31.00 ©2017 IEEE ƒ‰‡͵͸ͻ

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY TIRUCHIRAPALLI. Downloaded on June 04,2024 at 06:31:53 UTC from IEEE Xplore. Restrictions apply.
 2017Third International Conference on Science Technology Engineering & Management (ICONSTEM)

interaction. Most of the challenges in NLP are does not add any functionality to the existing
listed as follows: human language understanding, proposed system, rather increase the complexity of
enabling computers to derive meaning from the implementation. The high dependence on wireless
voice commands given to it through human sensors may also lead to erroneous results. [1]
(natural) language, and others involve natural
language interaction between computers and Other home automation systems insist on using
humans. Most of the latest NLP algorithms are Arduino boards as the only solution. Most of these
built upon machine learning, especially statistical systems can also be replaced by either ZigBee,
machine learning. Raspberry Pi or any other programmable board as
their functionalities are not altered. These systems
The primary concept of machine learning is also have a heavy reliance towards using mobile
different from that of most prior techniques at device as their central console alone, giving very
language processing. Former implementations of less flexibility of changing this. This is a concern
the language-processing jobs usually involved the for the user, because most of the time mobile
straight forward coding of a very large set of rules. devices are carried along with the user everywhere
The principle of the machine-learning standard they go rather than statically staying in the house.
calls instead for utilizing universal learning They also do not support voice commands from the
algorithm – mostly based on statistical inference – user. [2]
to routinely learn the most likely rules by the
analysis of a large sample (corpora) of typical real- The most noble causes of the home automation is
world examples. A corpus (plural, "corpora") is a to help make the lives of paralyzed people much
large sample of documents (or sometimes, discrete easier. These home automation systems built upon
sentences) that have been hand-annotated with the voice recognition are for the differently abled
accurate values to be learned. people suffering from quadriplegia or paraplegia
(who cannot shift their limbs but can speak and
Related Works: listen) and it enables them to control the various
home appliances and can actuate the bed elevation
The industry and the Academicians have worked just by the voice commands according to their need
together in perfect harmony to make great advances and comfort. The proposed system comprises of a
in the field of Home Automation Systems. Many voice recognition module, Arduino Uno
technology giants including Apple, Amazon and microcontroller, relay circuit to and a modifiable
Google have been keen on making viable consumer bed. The voice recognition unit needs to be skilled
products for the common man. Most of these in advance to be used to recognize commands.
gadgets have been released during the course of However all of the listed functionalities are limited
this year, some of them include: Amazon Echo, to adjustable beds. The proposed concept is
Apple HomeKit and Google Home. However none extremely clichéd and lacks sound design
of these systems aim to possess omnipresence or principles. [3]
include artificial intelligence to predict the user’s
commands. Most of these advancements have come Most of the other home automation systems use
only in the past year (2015) urging both companies Bluetooth in their mobile device for
and academicians to make further advancements in implementation. This has various demerits in every
this particular field. aspect. Primarily the Bluetooth has a very small
range and the proposed system could only work in
Certain home automation systems have suggested small homes, excluding homes with larger area.
complete dependence on wireless sensors and Bluetooth does not support remote operations, (i.e.)
nothing else. These systems also use very primal the appliances cannot be controlled by Bluetooth
technology such as HTML web pages to co- from a remote geographical area.
ordinate between these Arduino boards. They have
also indicated the use of mobile platform which

978-1-5090-4855-7/17/$31.00 ©2017 IEEE ƒ‰‡͵͹Ͳ

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY TIRUCHIRAPALLI. Downloaded on June 04,2024 at 06:31:53 UTC from IEEE Xplore. Restrictions apply.
 2017Third International Conference on Science Technology Engineering & Management (ICONSTEM)

Many appliances within the same range may pick Proposed System:
up the commands given to particular device
resulting in erroneous behavior. They also do not Our project aims to provide the most easy and
use Natural Language Processing and artificial efficient way to interact with home appliances by
intelligence to understand what the user is trying to giving voice commands in human (natural)
perform. [4] language. We plan on eliminating the tedious
process of clicking through various application
Most of the other home automation systems purely screens with just one voice command. The natural
depend on GSM technology for implementation. language processing in the project provides a
Even though their remote capabilities are personal connection with our system. Primarily the
tremendous, the overhead of operating it in a small user is authenticated by entering the specified
home environment is huge. The boards are tough to username and password in the mobile device. The
be compliant to GSM technology and increase the user sends a voice command to the mobile device,
cost of implementation exponentially. [5] which interprets the message and sends the
appropriate command to the specific appliance. The
Existing System: voice command given by the user is interpreted by
the mobile device using Natural Language
The existing technologies, in these regard are far
processing.
outdated than what we have proposed. Most of the
smart home automation systems that are existing The mobile device acts as a central console; it
only automate the basic process of changing the determines what operation must be done by which
state of the appliances to ON/OFF. There are many appliance to fulfill the user’s request. The central
smart home mechanization systems in the market console can also be either a desktop application,
that aim to automate the basic operations of these web application or a smart phone application as all
home appliances using various technologies such as of the data transferred can be processed by the
GSM (Global System for Mobile), NFC (Near- cloud. However, for the convenience of the user
Field Communication) and Wi-Fi. and increased mobile capabilities we will be using
a smart phone in this project.
The existing smart home systems that have either
been implemented or proposed have an elaborate The appliances are connected to the mobile device
procedure to interact with the home appliances. through an Arduino Board that establishes the
Some include pressing a button in a static location concept of Internet of Things. The Arduino Boards
while some others include giving commands are interfaced with the appliances and programmed
through a mobile device. Various technology in such a way that they respond to mobile inputs.
companies have been trying to create amazing Our project automates the operation of every single
products in the department of home automation appliance in the house, which greatly reduces the
system since a decade ago. However, the Internet power consumption due to excess use/wastage of
of things has become a briskly growing field only the appliance’s services.
in the recent past.
Architecture Diagram:
Many viable consumer products are being sold by
various technology giants in order to bring the
technology to the common man. Most of these
gadgets have been released during the course of
this year, some of them include: Amazon Echo,
Apple HomeKit and Google Home. However none
of these systems aim to possess omnipresence or
include artificial intelligence to predict the user’s
commands.

978-1-5090-4855-7/17/$31.00 ©2017 IEEE ƒ‰‡͵͹ͳ

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY TIRUCHIRAPALLI. Downloaded on June 04,2024 at 06:31:53 UTC from IEEE Xplore. Restrictions apply.
 2017Third International Conference on Science Technology Engineering & Management (ICONSTEM)

Conclusion:

Most of the existing automation systems follow a

specific set of commands or procedures in order to
interact with their home appliances. These
cumbersome procedures distance the users away
from the technology. To overcome all of the
existing issues and address the solution our project
proposes voice commands to interact with home
appliances using Arduino and a mobile device.
These voice commands which are processed by
Natural language processing helps the users to
build a better connection with the technology and
encourages them to use it more. It also eradicates
the mundane task of operating home appliances
manually.

References:

[1] Subhajit Dey, “Web based real-time home automation and

security system”, International Journal of Electrical and
Electronic Engineering & Telecommunications, Volume 4, No.
3, July 2015.

[2] Nathan David, Abafor Chima, Aronu Ugochukwu, “Design

of a Home Automation System Using Arduino”, International
Journal of Scientific & Engineering Research, Volume 6, Issue
6, June-2015.

[3] Mukesh Kumar, Shimi S.L,” Voice Recognition Based

Home Automation System for Paralyzed People”, International
Journal of Advanced Research in Electronics and
Fig: Architecture diagram using Arduino Communication Engineering (IJARECE) Volume 4, Issue 10,
MKR1000 October 2015.

[4] Akbar Satria, Muhammad Luthfi Priadi, Lili Ayu

Wulandhari, Widodo Budiharto, “The Framework of Home
Remote Automation System Based on Smartphone”,
International Journal of Smart Home Vol. 9, No. 1 (2015).

[5] Sonali Sen, Shamik Chakrabarty, Raghav Toshniwal, Ankita

Bhaumik, “Design of an Intelligent Voice Controlled Home
Automation System”, International Journal of Computer
Applications (0975 – 8887) Volume 121 – No.15, July 2015.

[6] Mahesh N. Jivani, “GSM Based Home Automation System

Using App-Inventor for Android Mobile Phone”, International
Journal of Advanced Research in Electrical, Electronics and
Instrumentation Engineering, Vol. 3, Issue 9, September 2014.

[7] Erik Brynjolfsson and Andrew McAfee, “The Second

Machine Age: Work, Progress and Prosperity in a Time of
Brilliant Technologies”, 2014.

[8] Rozita Teymourzadeh, Salah Addin Ahmed, Kok Wai Chan,

Fig: Screenshot of the Mobile Application and Mok Vee Hoong, “Smart GSM Based Home Automation
System”, 2013 IEEE Conference on Systems, Process & Control
(ICSPC2013).

978-1-5090-4855-7/17/$31.00 ©2017 IEEE ƒ‰‡͵͹ʹ

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY TIRUCHIRAPALLI. Downloaded on June 04,2024 at 06:31:53 UTC from IEEE Xplore. Restrictions apply.
 2017Third International Conference on Science Technology Engineering & Management (ICONSTEM)

[9] S. P. Kumar and O. Pandithurai, “Sixth sense technology,” in

Proc. of 2013 Int. Conf. on Information Communication and
Embedded Systems (ICICES), pp. 947-953, February 21-22,
2013

[10] O. Pandithurai, B Preethi, M Meena, V Nivedha, V Ramya,”

A framework for efficacy in lane discipline management using
hybrid approach”2016 International Conference on Science
Technology Engineering and Management (ICONSTEM), DOI:
10.1109/ICONSTEM.2016.7560927

978-1-5090-4855-7/17/$31.00 ©2017 IEEE ƒ‰‡͵͹͵

Authorized licensed use limited to: NATIONAL INSTITUTE OF TECHNOLOGY TIRUCHIRAPALLI. Downloaded on June 04,2024 at 06:31:53 UTC from IEEE Xplore. Restrictions apply.