Annexure A

A Project Report
On

SMART SYSTEM FOR BLIND AND MUTE

Submitted in partial fulfillment of the requirement for the degree of

Bachelor of Technology
In
Electronics and Communication Engineering

Submitted By: Under the Supervision of:

HARSH GUPTA (9916102120) DR. JITENDRA MOHAN
DHANANJAY BHARDWAJ (9916102036)
HARSH VATS (9916102075)

Department of Electronics and Communication Engineering

JAYPEE INSTITUTE OF INFORMATION TECHNOLOGY, NOIDA
December, 2019
 Annexure B

CERTIFICAT
E

It is ensured that the work contained in the venture report titled "Smart System For Blind and Mute"
put together by Dhananjay Bhardwaj, Harsh Gupta and Harsh Vats, has been completed under my
watch and this work has not been submitted somewhere else for a degree.

Signature of Supervisor:
Dr. Jitendra Mohan
ECE Department,
JIIT NOIDA

December, 2019

ii
 Annexure C

DECLARATION

We announce this composed accommodation speaks to our thoughts in our own words and where
others' thoughts or words have been incorporated, we have satisfactorily referred to and referenced the
first sources. We likewise announce that we have clung to all standards of scholastic trustworthiness
and uprightness and have not distorted or manufactured or adulterated any
thought/information/certainty/source in our accommodation. We understand that any infringement of
the above will be cause for disciplinary activity by the Institute and can likewise inspire correctional
activity from the sources which have in this way not been appropriately referred to or from whom
legitimate authorization has not been taken when required.

(Signature)

Name: Harsh Gupta

Enrollment:9916102120

Name: Dhananjay Bhardwaj

Enrollment: 9916102036

Name: Harsh Vats

Enrollment: 9916102075

Date: 16 December, 2019

iii
 ABSTRACT

The project aims to provide a convenient and safe method for blinds to overcome their
difficulties in daily life. The Smart System for Blind is application based prototype that can
help visually impaired facilitate movements and perform daily activities without relying too
much on others. It outlines a better navigation and gesture sensing tool for the visually
impaired and makes them more independent. The glove is equipped with sensors to give
information about the environment. The integration of ultrasonic sensor HC-SR04 will help
blind to facilitate movement and give alert to user if there is an obstacle in front of them in
the range 2 cm to 300 cm. In order to control the various functionalities of a media player
like increasing/decreasing volume, simple hand gesture recognition is used using OpenCV.
For conveying navigation information to the care taker, a more secure and efficient
technology called Li-Fi is used. Smart System for Blind will help blind people walk and
estimate the distance from obstacles and perform daily routine tasks autonomously with
simple hand gestures and even navigate quite easily and securely with Li-Fi
communication. Throughout the project, several hardware tools are employed in order to
implement the final prototype.

iv
 ACKNOWLEDGMENT

We place on record and heartily recognize the ceaseless support, significant oversight, convenient
recommendations and motivated direction offered by our project mentor Dr. Jitendra Mohan,
Department of Electronics and Communication Engineering at Jaypee Institute of Information
Technology Sector-128 in bringing this report to a successful completion. We are appreciative of our
Major project coordinator Dr. Sajaivir Singh to make the offices accessible in the Department to
complete the project effectively. To wrap things up we express our earnest gratitude to every one of
our companions who have calmly broaden a wide range of help for achieving this endeavor.

Name: Harsh Gupta

Enrollment:9916102120

Name: Dhananjay Bhardwaj

Enrollment:9916102036

Name: Harsh Vats

Enrollment:9916102075

December 2019

v
 CONTENTS

Certificate.................................................................................................................ii

Declaration…..........................................................................................................iii

Abstract....................................................................................................................iv

Acknowledgement....................................................................................................v

List of Figures…..................................................................................................viii

1 Introduction and Literature Survey 9

Background 9
Global Data 9
Indian Data 9
Motivation......................................................................................................10

2 Components 11
Detail of Components..............................................................................11
2.1.1 LED…..................................................................................11
2.1.2 LDR......................................................................................11
2.1.3 LM 741.................................................................................12
Keypad 12
LCD 13
Buzzer 13
Microcontroller..........................................................................................14

3 Implementation 15
Hardware.........................................................................................................15
LED 15
LDR 16
3.1.3 LM 741......................................................................................16
Keypad 16
LCD 16
Buzzer 16
Results…...................................................................................................17

vi
 Software...........................................................................................................18
Introduction 18
OpenCV 18
Block diagram of system 19
Segment the hand region 19
Motion detection and thresholding.........................................................................................20
Contour extraction 20
Process flowchart 20
Simulation results 21

4 Application 22
Ease of work 22
Faster Communication............................................................................................................22
Secure Communication...........................................................................................................23
Environmental Impact.............................................................................................................23

5 Conclusion and Future Work 24

Text to speech..................................................................................................24
Increase range of transmission........................................................................24

References

Appendix I: Arduino Nano Pin-

Out Appendix II: Sensors

Schematic

vii
 LIST OF FIGURES

Global population of blind persons....................................................................................10

Visually impaired dependent on care taker........................................................................10
LED 11
LDR 11 .
LM 741 Opamp 12 ..
4x4 Keypad 12
2.5 16x2 LCD....................................................................................13
.
Buzzer 13
..
Arduino Uno 14
.
Arduino Nano 14
System Architecture ...
15
Hardware Setup… 17 .
Number on keypad vs time...................................................................................................17
Voltage level of LDR vs time.............................................................................................17
...
OpenCV with Python 18 .
Block diagram of system....................................................................................................19
.
Image capturing algorithm.................................................................................................20
..
Detection and counting of fingers...................................................................................... 21
.
Recognition of simple hand gesture................................................................................... 21
..
Distribution of blind and low vision people.......................................................................22
Increase in data rate with different led technologies.......................................................... 22 ..

Increasing demand for high speed data.............................................................................. 23 .

.

viii
 CHAPTER 1

INTRODUCTION
AND
LITERATURE SURVEY

Background

Over the years blindness caused by illness has decreased. This is wholly due to the success of
public health systems. Despite success of healthcare systems blindness in people over 60 years
is increasing by 2 million per decade. Unfortunately all these numbers are estimated to be
doubled by 2020 [1] .The need for assistance through devices for navigation has increased. The
most famous, used and easy to use tools which are present are trained dogs and white cane.
Despite of being famous and easy to use these tools fail to provide a blind person with all the
information which is available to a person with sight [1]. Figure 1.1 shows a graph representing
this information.

Global Data

The World Health Organization (WHO) announced that there are 285 million outwardly disabled
individuals around the world. Among these people, there are 39 million who are partially visually
impaired [2]. More than 1.3 million are totally visually impaired and around 8.7 million are outwardly
disabled in the USA [1]. Unfortunately of these, 100,000 are students..

Indian Data

India is home to nearly 12 million blind of the 39 million around the globe, almost 1/3 of total
blind people. NPCB (National Programme for Control of Blindness) defines blindness as
vision of 6/60 or less and a visual field loss of 20 degrees or less in the better eye after
spectacle correction.

9
 Figure 1.1: Global population of blind persons
Source: Global Prevalence of Vision Impairment and Blindness: Magnitude and Temporal Trends, 1990–2010.

Motivation

In our societies, on a daily life basis we usually adopt various measures to simplify things for
our elders, either we implement some technical solutions or provide them an ease of life
manually. But these solutions are way simpler for those who are not handicapped. In most of
the cases people fail to provide a proper assistance to blind people. This biggest challenge is to
provide them a 24x7 assistance which is next to impossible and most of the times impractical to
implement. How can they compete with the other people of this society? How can they navigate
from one place to another and that too without anyone’s help? To answer these questions, we
have come up with a solution for this section of our society. it will be now easier for them to
detect an object. It will also help them to navigate from one place to another. Figure 1.2 depicts
the need of a care taker in simple day to day activities of a visually impaired.

Figure 1.2: Visually impaired dependent on care taker

Source: https://www.japantimes.co.jp/newS/2018/01/22/national/social- issues/dialogue-verbal explanations-
tactile-methods-help-japans-visually- impaired-deepen-appreciation-art/#.XfUy5-hKjIU

10
 CHAPTER 2

COMPONENTS

Detail of Components

LED

A light-emitting diode (LED) is a simple semiconductor device that has a property of

emitting energy when electric current is passed through it. It is a simple LED, used as the
transmission source by frequently changing its light intensity [3]. Figure 2.1 shows a set of
LED’s of different colors.

Figure 2.1: LED

Source: https://www.reichelt.com/de/en/led-3-mm-low-current-
green-led-3mm-2MA- gn-P21624.html

LDR

A Light Dependent Resistor (LDR) is a device having resistivity as a function of the incident
EM (electromagnetic) radiation, therefore its behavior shows light sensitivity. They are also
termed as photo conductors, photo conductive cells or simply photocells. Figure 2.2 shows a
LDR sensor.

Figure 2.2: LDR

Source: https://potentiallabs.com/cart/ldr-big

11
2.1.3 LM 741

An operational amplifier or simply op-amp is an integrated circuit primarily designed for

performing analogue computations. It is a high-gain electronic voltage amplifier with a
differential input and usually a single-ended output [4]. Figure 2.3 shows a dual Opamp IC
LM 741.

Figure 2.3: LM 741 Opamp

Source: https://www.tandyonline.com/741-operational-amplifier.html

Keypad

The keypad used has 12 keys with 3 columns and 4 rows[5]. The row and column pins are
directly connected with Arduino. Figure 2.4 displays a keypad with 4 rows and 4 columns.

Figure 2.4: 4x4 Keypad

Source: https://www.makerlab-electronics.com/product/4x4-matrix-membrane- keypad/

12
 LCD

The 16x2 LCD display is a module commonly used in prototyping of circuits. The 16x2
LCD translates a display of 16 characters per line in 2 rows. In this type of LCD’s every
character is displayed in a format of a 5x7 pixel matrix [6]. Figure 2.5 displays a 16x2 LCD.

Figure 2.5: 16x2 LCD

Source: https://www.seeedstudio.com/LCD-16x2-Character s-White-Text-Blue-Background-p-1612.H T ml

Buzzer

The buzzer is an alerting device with two pins to attach one at VCC and other at Ground
(GND). When current is applied to the buzzer it leads to a frequent expansion and
contraction of the ceramic plate. Figure 2.6 shows a buzzer having two polarterminals.

Figure 2.6: Buzzer

Source: https://www.pcboard.ca/minipiezo-buzzer

13
 Microcontroller

Arduino UNO

Arduino UNO as shown in Figure 2.7 is a prototyping board with ATMEGA 328
functioning as the microcontroller. It has 14 digital input/output pins, 6 analog inputs, a 16
MHz quartz crystal, a USB connection and other on board components [7]. In the
proposed system, Arduino UNO is directly connected to the transmitting LED and the
keypad. The digital and the analog pins of the keypad are used to receive the data from the
keypad, i.e. the number pressed on the key. The running program is responsible fetch the
key pressed and accordingly change.

Figure 2.7: Arduino UNO.

Source: https://en.wikipedia.org/wiki/Arduino

Arduino Nano

The Arduino Nano as appeared in Figure 2.8 is a little, complete board dependent on the ATmega328P
(Arduino Nano 3.x). It has pretty much a similar use of the Arduino Duemilanove however in an
alternate bundle. It needs just a DC power jack and works with a Mini-B USB link rather than a
standard one [8]. Figure 2.8 shows an Arduino Nano.

Figure 2.8: Arduino Nano

Source: https://diygeeks.org/shop/arduino-boards/arduino-nano/

14
 CHAPTER 3

IMPLEMENTATION
Hardware

The following section describes the working of the hardware interfaced. Figure 3.1
shows the hardware setup of the project. The following is the proposed system
architecture for the project. The architecture is completed by interfacing basic
components for example, LED, LDR and a buzzer. It is divided into two parts, one is the
receiver circuit and the other is transmitter circuit. Figure 3.2 demonstrates the actual
hardware setup of the project.

Figure 3.1: System Architecture

LED
It is a simple LED, used as the transmission source by frequently changing its light intensity.
The reason for using led is because of its high flickering rate. A normal human eye cannot
detect its fast flickering and its nature of getting obstructed with an object provides security.

15
 LDR

In the proposed system, LDR acts as the receiving element as it senses the changing intensity.
One of the terminals is grounded and the other acts as the inverting input terminal for the
Opamp. The changing voltage from the LDR is an input at the Opamp terminals

3.1.3 LM 741

In the proposed system, single op-amp LM- 741 IC works as comparator. The Op-amp
comparator compares an analogue voltage level at one of the pin with another voltage level, or
some preset reference voltage, that is, VREF and produces an output signal based on this
voltage comparison. In the system, the positive/non- inverting terminal is connected to a 10k
potentiometer which is connected to the VCC, while the negative/inverting terminal is directly
connected to the LDR.

Keypad

The row and column pins are directly connected with Arduino. In the proposed system,
keypad is the crucial element as it acts as a connecting interface between the blind person and
the care taker. Each numeric on the keypad is assigned a corresponding function that the blind
person wishes to perform.

LCD

In the model, LCD is interfaced with Arduino NANO at the recipient circuit which shows the
numeric key squeezed by the individual at the transmission side of the framework.

Buzzer

In the proposed system, the buzzer is interfaced with the receiver circuit. Whenever the
transmitted data is received, the buzzer beeps for a short duration of time. It functions as an
alarm that helps the care taker to stay aware whenever the data is received.

14

16
 Figure 3.2 : Hardware Setup
Results
With the Li-Fi communication system ready and set, few tests are performed to check whether
the program is able to communicate via visible light. Following graphs are plotted as a result
of instructions sent from transmitter to receiver side. Figure 3.3 displays the number in the
keypad pressed vs time at the transmitter side and Figure 3.4 displays the voltage level of
LDR vs time at the receiver side after pressing of a certain key in the keypad.

Figure 3.3: Display of key pressed vs time at transmitter side

Figure 3.4: Instructions received and the corresponding LDR voltage vs time at receiver side

17
Software

Computer Vision and Image Processing have been used in a number of tasks involving
automatic detection and monitoring. In this system/prototype Open CV is used to produce
hand gesture recognition. Through hand gesture recognition we controlled VLC media
players function like Play/Pause and Volume Up/Volume Down thus making it easy for a
visually impaired person to control the player.

Introduction

Gesture recognition has been a very interesting area of development and problematic in
Computer Vision community for a long time. One of the key reasons for this is the sheer
difficulty of separating the foreground from a clustered background in real time. The most
evident explanation is a result of the semantic hole included when a human glances at a
picture and a PC taking a gander at a similar picture. People can without much of a stretch
make sense of what's in a picture however for a PC, pictures are only 3-dimensional
frameworks.

OpenCV

One of the most useful and easily available tools to perform computer vision is OpenCV.
"An open source PC vision library" which comprises of in excess of 2500 calculations,
equipped for filling in as a far reaching set of both PC vision and AI calculations and can
be utilized to identify and perceive faces, identify objects, track movements etc. Figure 3.5
explains the most common use of OpenCV with Python.

Figure 3.5: OpenCV with Python

Source: https://opencv.org/about.html

18
Block Diagram of System

This project actualizes PC vision and motion acknowledgment procedures and builds up a
dream based minimal effort input gadget for controlling the VLC player through signals.
In this firstly, video from webcam is captured known as image acquisition and then image
pre-processing is done using convex hull algorithm. By using Python programming,
interfacing with computer is done to control VLC Media Player. The procedure of the
picture processing depends on the treatment of digitalized pictures, so as to investigate
their substance or control them. This hand gesture recognition technique will not only
replace the use of mouse to control the VLC player but also provide an easy and efficient
way for visually impaired to listen music [9]. The whole proposed software system is
explained in Figure 3.6.

Figure 3.6: Block diagram of system.

Segment the Hand region

The first task at was to identify hand region and eliminate all the unwanted background.
The method implemented in our project was called running averages. In this method the
system looks over 30 frames. After figuring out the background model, the current frame
(which contains both foreground and background) is also used. The absolute difference
between current frame (hand) and the background frame (updated over time) will give us a
new image with newly added foreground. Running average was calculated from the
formula given in the equation (1). [10]

(1)

18

19
 Motion Detection and Thresholding

Subsequent stage was to distinguish the hand region from this difference picture. For this thresholding
of the difference, picture is required, with the goal that solitary the hand region becomes obvious and
the various undesirable regions are painted as dark. Thresholding is fundamentally the task of pixel
forces to 0's and 1 depends on a specific limit level with the goal that the object of intrigue alone is
caught from a picture. In the event that x(n) speaks to the pixel force of an information picture at a
specific pixel arrange, at that point limit chooses how pleasantly the picture will be divided into a
twofold picture as clarified in condition (2) [10].

(2)

Contour Extraction

Contours are obtained after performing thresholding on the difference image. The contour with the
biggest region is thought to be the hand. Contour is the diagram limit of an item situated in a picture.

Process Flowchart

As shown in Figure 3.7, there are four intermediate steps to count the fingers, given a
segmented hand region [11].

Figure 3.7: Block diagram of system

20
Simulation Results

With the image processing tool is ready and set few tests are performed to check whether
the program is able to detect hand gesture and being able to control various functionalities
of media player.
After observing the results we found that the program was able to detect and count the
number of fingers and correspondingly was able to produce the contour and thresholding
image from the original image as well as shown in Figure 3.8.

Figure 3.8: Detection and counting of fingers

After testing the program to count the number of fingers, the program was checked to see
if the functionalities of the media player can be controlled with the recognition of simple
hand gestures. The program was successfully able to detect and control the functionalities
of media player like Play/Pause and Volume Up/Down. This was done with the help of
convex hull algorithm and counting of the convexity defects in the image. Figure 3.9
shows the recognition of simple hand gesture and hence, in this case, lowering down of
volume.

Figure 3.9: Recognition of simple hand gesture

21
 CHAPTER 4

APPLICATIONS

Ease of Work
The world has over 285 million blind people in the world and 36 million in India according to
2015 census as shown in Figure 4.1. Our project aims to create a safe environment for those
people . Through the implementation of our project travelling and maneuvering around short
areas will become comfortably easy for blind people.

Figure 4.1: Distribution of blind and people with low vision around the globe.
Source: https://www.slideshare.net/InternationalCentreforEyeHealth/epidemiologyandvisualimp

Faster Communication

The mode used is Li-Fi which uses visible light spectrum ranging from 430 THz to 750 THz.
This spectrum ensures high data rates, more bandwidth and less latency. Figure 4.2 depicts an
increase in data rate with different LED technologies.

Figure 4.2: Increase in data rate with different LED technologies.

Source: https://www.lifi.eng.ed.ac.uk/lifi-newS/2015-11-28-1320/how-fast-can-lifi-be

22
Secure Communication

As Li-Fi works on VLC (Visible Light Communication), it is one of the most secured
modes of transmission. As light cannot pass an opaque object hence one can control the
area of transmission using opaque objects.

Environmental Impact

In the race for less latency, more bandwidth and better use of spectrum RF spectrum has far
exceeded safe frequencies region. The current 5G technology is expected to release small
amounts of radiation and can’t be used near nuclear power plants because of radiation it
emits. On the other hand Li-Fi which uses a simple LED for its transmitter which even act
as a source of light. In near future every light source can act as a light source and a Li-Fi
transmitter. Figure 4.3 depicts an increase trend in demand for high speed data over the
years.

Figure 4.3: Increasing demand for high speed data

Source: IEEE 5G Tech Focus: Volume 2, Number 2, May 2018

23
 CHAPTER 5

CONCLUSION AND
FUTURE WORK

This project emerges as a promising system in terms of number of applications provided

to the visually impaired. All the applications mentioned in the project can be performed
by them with minimal need of any external support of a care taker. This has been made
possible by integrating two of the most emerging fields of technology i.e. OpenCV and
VLC. So far, the work has been conducted independently in these fields of technology. In
this project, VLC is being implemented as a system that will communicate any instruction
to be sent to the care taker via pressing a single key of keypad next to the person. The
system simplifies the problem that usually occurs with the visually impaired while
listening to music from a device such as laptop. The proposed system makes use of hand
gestures in order to control the music. Alexa, too can fail in situations when the
surrounding is too noisy but the device can easily control the music with a simple
recognition of hand gesture.

Text to speech

The future work of this project would be centered around the conversion of data received
on the receiver side via visible light communication into an audio based output. This will
improve the system in such a way that if a care taker misses to read the text to be
displayed on the receiving end, the voice instructions will help the person in reminding
the instructions received from a visually impaired person.

Increasing range of transmission

Currently, the prototype made in this project consists of 3 led’s through which the visible
light communication has been made possible but this is has resulted into a short range of
transmission. Using a large grid of led matrix can be useful for longer range of
communication and even increase in number of applications like transmission of images.

24
 REFERENCES

[1] “American Foundation for the Blind (AFB).” The Grants Register 2019, 2018, 55–56. [Online]. Available:
https://doi.org/10.1007/978-1-349-95810-8_72. [Accessed Dec. 15, 2019].

[2] “Vision Impairment and Blindness.” World Health Organization. [Online]. Available: https://www.who.int/news-
room/fact-sheets/detail/blindness-and-visual-impairment. [Accessed Dec. 15, 2019].

[3] Sandip Das, Ankan Chakraborty, Debjani Chakraborty, Sumanjit Moshat, "PC to PC Data Transmission using
Visible Light Communication", 2017 International Conference on Computer Communication and Informatics
(ICCCI -2017), Jan. 05 – 07, 2017, Coimbatore, INDIA.

[4] [Online]. Available: https://www.elprocus.com/ic-741-op-amp-tutorial-and-characteristics/ [Accessed Dec. 5,

2019].

[5] [Online]. Available: http://www.circuitbasics.com/how-to-set-up-a-keypad-on-an-arduino/ [Accessed Nov. 22,

2019].

[6] [Online]. Available: https://electronicsforu.com/resources/learn-electronics/16x2-lcd-pinout-diagram [Accessed

Nov. 25, 2019].

[7] [Online]. Available: https://www.tomsonelectronics.com/blogs/news/arduino-uno-specification [Accessed Nov.

22, 2019].

[8] [Online]. Available: https://www.tomsonelectronics.com/blogs/news/arduino-nano-v3-0-specification [Accessed

Nov. 26, 2019].

[9] Aekta Patel, “VLC Media Player Controlling Using Hand Gesture” International Journal of Advanced Scientific
and Technical Research. [Online]. Available: http://www.rspublication.com/ijst/index.html. [Accessed Dec. 12,
2019].

[10] “Hand Gesture Recognition using Python and OpenCV - Part 1” [Online]. Available:
https://gogul.dev/software/hand-gesture-recognition-p1 [Accessed Dec. 3, 2019].

[11] “Hand Gesture Recognition using Python and OpenCV - Part 2” [Online]. Available:
https://gogul.dev/software/hand-gesture-recognition-p2 [Accessed Dec. 3, 2019].

25
 APPENDIX I

ARDUINO NANO PIN-OUT

26
 APPENDIX II

SENSOR SCHEMATIC

LDR SENSOR

OPAMP IC LM 741 PIN OUT

27