FACE DETECTION BASED
ATTENDANCE SYSTEM USING IOT
Saravanan D, Rangarajan M, Vikram D
College of Engineering Guindy, Chennai.
ABSTRACT:
The proposal is mainly about the face detection-based attendance system which overcomes manual
method of taking attendance, reduces the time to take attendance and avoids proxy. Unlike other
Biometric attendance systems like fingerprint recognition, DNA matching, face recognition, Iris
recognition and voice recognition, this face detection-based attendance system does not require
any person to be in contact with the fingerprint reader module, which reduces a lot of time,
improves security levels and low fraud rate. This project involves the student, faculty or employee
attendance by face recognition. The face detection and recognition were carried out by using
ESP32 cam module which captures the images of the available students in the classroom. Using
this image, it recognizes and compares with the images in the database and marks their attendance
Keywords: Face Recognition, Face Detection, ESP32 camera module.
INTRODUCTION:
In today's fast-paced world, managing attendance efficiently is a crucial task for organizations,
institutions, and businesses. Traditional attendance systems, such as manual registers or swipe
cards, can be prone to errors and time-consuming. With advancements in technology, automated
face detection attendance systems have emerged as a reliable and convenient solution.
Attendance management is an integral part of any institution, be it educational, corporate, or
organizational, as it directly influences productivity, resource utilization, and decision-making.
Traditional methods, such as manual roll-calling or sign-in sheets, are laborious, time-consuming,
and susceptible to errors.
The advantages of employing a face detection-based attendance system are multifaceted. Firstly,
it offers seamless and frictionless user experience, with individuals being marked present simply
by standing in front of a camera, without the need for any physical interaction. Secondly, the
system mitigates issues of "buddy punching" or proxy attendance, as it uniquely identifies
individuals based on their facial features, ensuring accountability and data accuracy.
1
Implementing a face detection-based attendance system poses certain challenges. These may
include concerns regarding privacy and data security, as facial information is sensitive and requires
stringent protection. Additionally, factors like varying lighting conditions, occlusions, and pose
variations may affect the system's performance, necessitating careful consideration during design
and deployment.
The Face Detection Attendance System using ESP32 is a cutting-edge application of Internet of
Things (IoT) and computer vision technologies. It combines the power of ESP32, a versatile
microcontroller, and sophisticated facial recognition algorithms to create a seamless and accurate
attendance management system.
LITERATURE SURVEY:
Taking attendance manually is a time-consuming process that frequently results in human
mistakes. This paper describes the use of Open-Source Software to create a Student Attendance
Management System in a multi-user setting. For reading tags, the system employs Python as a
backend. To authenticate lecturers, run the Python code, and record tags in a sheets file. Finally,
the sheet file is posted to the server, where it will be processed and interpreted. The web portal can
be used to check attendance.
It has recognized that there are several technologies at our disposal, yet it's still doing things the
old way. When it came to the attendance system at universities and schools, professors had to do
everything by hand. Lecturers took attendance and manually put it into the database. It has
identified a wealth of technological instruments that may be used to ease the tension of lectures.
One example of this is the use of IOT (Internet of Things). It can accomplish it automatically,
obviating the necessity for lectures.
The Eigenfaces algorithm was developed by Sirovich and Kirby (2013) and used by Matthew Turk
and Alex Pentland in face classification. This algorithm captures the variation in the collected faces
and compares it with individual faces by generating a set of eigenvectors from the image by the
principal component analysis. The eigenvectors are derived based on the covariance matrix of the
probability distribution over the high-dimensional vector space of face images [12].
Fisher faces algorithm is derived from an idea suggested by R.A.Fisher in 2014. This algorithm is
an enhancement to Eigen faces that uses linear discriminant analysis. The linear discriminant
analysis finds the linear combination of features that characterizes or separates two or more classes
of faces. The resulting combination is used for the reduction of dimensionality. This algorithm is
used when there is a large variation in the illumination and facial expressions [13].
Local binary pattern histogram algorithm is described in the year 2016 and was a powerful
technique for feature extraction. In this algorithm, the grayscale image is divided into cells of the
3X3 matrix. The obtained matrix is represented with the range of pixel intensities. The center value
2
of the matrix is considered as the threshold and each pixel value is compared with the threshold.
If the pixel value is greater than the threshold it is given 1 otherwise 0. An 8-digit binary value is
converted into decimal value for convenience. Then the histogram is computed over the cells and
frequency of each number and concatenate the histogram of all cells to obtain the feature vector.
This feature vector is processed to classify the images [14], [15].
Scale-invariant feature transform algorithm was published by David Lowe in 2017. This algorithm
detects and describes the local features in the images. Whenever the image is transformed, like the
size of the image is altered or when it is rotated the features of the image are invariable. These
invariable features are used to detect the image [16].
Patil et al. in applied face recognition for classroom attendance in 2017. They used Eigenface for
the recognition, but the overall accuracy of the system was not mentioned in the paper. Similar
approach using Eigenface for the face recognition based attendance tracking system was proposed
in. They achieved an overall recognition accuracy of 85% for unveiled faces. Tharanga et al. used
Principal Component Analysis (PCA) method for the face recognition for their attendance system,
achieving the accuracy of 68% [18].
Speed up robust feature algorithm was presented by Herbert Bay, Tuytelaars, T. and Van Gool in
the year 2019. This algorithm is a patented local feature detector and descriptor. SURF descriptors
are used to locate and recognize objects, people or faces, to reconstruct 3D scenes, to track objects
and to extract points of interest. This algorithm uses a multiresolution pyramid technique where
the image is transformed into coordinates to copy the original image with Pyramidal Gaussian
shape to obtain an image with the same size but with decreased bandwidth [17].
The above complexities are overcome by a simplified face detection algorithm provided by ESP -
32 cam server and with a Arduino uno communication card which provides an easily accessible
user interface for enrollment and detection of multiple faces
PROPOSED METHODOLOGY:
The project aims to develop a face detection attendance system utilizing the ESP32-CAM module
as the core hardware platform. The methodology involves thorough planning and requirements
gathering to define the system's scope and desired outcomes. The hardware setup entails
assembling the ESP32-CAM module, USB to Serial TTL converter, power supply, SD card, and
camera mounting. The software setup involves configuring the development environment,
including the Arduino IDE or PlatformIO, and incorporating essential libraries for ESP32, camera,
and face detection, such as OpenCV. An appropriate face detection algorithm, like Haar cascades
or Single Shot Multibox Detector (SSD), is selected for efficient embedded performance. A dataset
of faces is collected and labeled for training the face detection model, and the model is then
3
exported to the ESP32-CAM. The code is developed to capture video frames from the camera and
apply the face detection algorithm in real-time, recording timestamps and individual IDs upon
detection. Data storage and management systems are established, using an SD card to store face
data and attendance records, organized in a well-structured format. Optionally, face recognition
can be implemented to further enhance accuracy and individual identification. Testing is conducted
in diverse environmental conditions to optimize the system's accuracy and performance. The
ESP32-CAM system, along with any optional user interface, is deployed for attendance tracking,
and comprehensive documentation is created to facilitate future maintenance and iterations.
Regular monitoring and maintenance are performed to ensure the system's reliability and
efficiency, providing an effective and automated face detection attendance solution.
SCHEMATIC DIAGRAM:
Fig 1: Schematic diagram
1. 5V to 5V on the board (or 3.3V to 3.3V).
2. GND to GND.
3. TXD orange cable to UOT on the board.
4. RXD yellow cable to UOR on the board.
5. A jumper between GPIO0 and GND (before uploading the code).
BLOCK DIAGRAM:
4
Fig 2: Face Detection Attendance System
COMMUNICATION CARD:
To ensure the uploading of an operating program of such an application to the ESP32-CAM card,
there are several cards such as FTDI, ARDUINO, MODULE TTL. We chose the ARDUINO board
as a converter compatible with the control board because it can play both a control board and a
converter.
CAMERA: ESP - 32 CAM:
It is a relatively small integrated module consisting of an OV2640 camera and an ESP3-S processor
that enables the creation of IP camera projects for video streaming with various resolutions and
visualization in real time via a wifi network. It has three 3.3V or 5V GND power pins.
For night photos, for example, a high-brightness LED can be bought. In this situation, a
photoresistor can be connected to the board. It does, in fact, provide some GPIO ports for
connecting various sensors and actuators.
The uses of this camera are :
It has high data transfer speed, storage capacity, efficient transmission without data loss or noise
intrusion, effective receipt of transmitted signals (such as images; sensor responses), and
responsiveness to order reception and execution.
The ESP-32CAM is suitable for a wide range of IoT applications. It is appropriate for smart home
appliances, industrial wireless controls, wireless monitoring, QR wireless identification, wireless
5
positioning system signals, and other Internet of Things (IoT) applications. It is an excellent choice
for IoT applications.
Fig. 3. : Internal composition of the ESP32-CAM board
FACE DETECTION:
In this section, network connection to the ESP32-CAM board is established. The router functions
as an access point, while the ESP32-CAM board functions as a station. It is connected to a local
network to operate and access the ESP32-CAM web server. The ESP32-Cam module's network
access mechanism, which includes WiFi connectivity, enables it to send movies in real time. The
module is essentially composed of a microcontroller that operates similarly to an Arduino,
allowing it to be programmed using the Arduino software.
To upload the program from the computer to the ESP32-Cam, the first step is to connect the module
to a PC via a programmer. Hosted by a web server, the ESP32-Cam module can create its own
Wifi network which allows it the autonomy to generate a connection and send videos to any
location.
The face recognition algorithm by ESP32-CAM is given as follows:
6
Step 1: Access via web browser
The serial monitor displays the IP address that will allow the opening of the camera according to
the URL reserved in a web browser
Fig. 4. Displaying data on the serial monitor
Copy the address given by Fig.8 and then paste it into the address bar of a web browser. The web
browser needs to be on the same network as the ESP32-CAM is connected to.
Fig. 5. : Camera feature
Step 2 : Validation of the authentication phase
Each time the camera succeeds in detecting a face, an identification is made after a real-time
comparison between the detected face and those registered in our database.
Once the comparison is made, the application can provide two results: the name of the person if
the face is well selected in the database or an unknown person since his face does not exist in the
7
database. Demonstrate the proper operation of the phenomena of facial recognition by using
numerous instances with grimaces and varied positions indicating the percentage of confirmation
of the requested person.
SENDING ATTENDANCE DATA TO GOOGLE SHEETS:
As previously stated, attendance data from the ESP32-CAM will be provided to Google Sheets.
We'll be using IFTTT applets for this. This applet will use webhooks to retrieve data from the
ESP32-CAM and then update it in Google Sheets.
Fig 6:Choosing a service to receive web requests from esp-32 cam
When it gets the request from the webhooks it will initiate a request to google sheets and it will
store the attendance in the excel sheets by adding rows to each person
8
Fig 7:Google sheets for accessing attendance data
FINAL CIRCUIT:
Fig 8: Project image
9
RESULTS AND DISCUSSIONS:
Each pupil must first register their faces. After each student has been enrolled, the camera will
begin identifying faces in the frame whenever the student walks in front of it. After detecting, the
system will automatically mark the attendance of the specific student in the attendance list in the
Excel sheet of a certain topic with their name and time. Students' attendance is recorded in an excel
sheet along with the time it was recorded.
Fig 9: Face Detection of Person 1
10
Fig 10: Face Detection of Person 2
Fig 11: Face Detection of Person 3
11
Fig 12 : Employee Attendance Sheet
CONCLUSION:
Different classrooms may have different lighting, seating layouts, and atmosphere. There may also
be students with diverse facial expressions, hair styles, beards, spectacles, and so on. This approach
can be adopted to improve the management of attendance and leaves. The system will save time,
reduce the amount of work that the administration needs to do, replace stationery with electronic
equipment, and minimize the quantity of human resource required for the purpose.
REFERENCES:
1. Mahesh Sutar, Mahesh Patil, Sachin Waghmare, “Smart Attendance System
Using RFID In IOT”, International Journal of Advanced Research in Computer
Engineering &Technology (IJARCET)
12
2. T. Lim, S. Sim, and M. Mansor,” RFID based attendance system”, in Industrial
Electronics and Applications, 2009. ISIEA 2009. IEEE Symposium on, vol. 2.
IEEE, 2009, pp.778782.
3. Beymer, D. and Poggio, T. (1995) Face Recognition from One Example View, A.I.
Memo No. 1536, C.B.C.L. Paper No. 121. MIT
4. Viola, Paul, and Michael J. Jones, “Robust real-time face detection”, International
journal of computer vision 57.2(2004):137-154.
5. R. Girshick, “Fast R-CNN,” in IEEE International Conference on Computer Vision
(ICCV), 2015
6. Yang B, Yan J, Lei Z, et al. Convolutional Channel Features for Pedestrian, Face
and Edge Detection [J]. Computer Science, 2015:82-90.
7. Michał Dolecki, Paweł Karczmarek, Adam Kiersztyn, Witold Pedrycz, "Face
recognition by humans performed on basis of linguistic descriptors and neural
networks", Neural Networks (IJCNN) 2016 International Joint Conference on, pp.
5135-5140, 2016, ISSN 2161-4407
8. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep
convolutional neural networks,” in Advances in Neural Information Processing
Systems 25: 26th Annual Conference on Neural Information
Processing Systems 2012. Proceedings of a meeting held on December 3-6,
2012, (Lake Tahoe, Nevada, United States), pp. 1106–1114, 2012
9. Soniya, Paul S, Singh L. A review on advances in deep learning[C]// IEEE
Workshop on Computational Intelligence: Theories, Applications and Future
Directions. IEEE, 2015:1-6.
10. R. Sokullu, O. Akin, and E. Aslan, "Face Recognition Based Multifunction Smart
Mailbox," 7th International Conference on Energy Efficiency and Agricultural
Engineering (EE&AE), pp. 1-4, (2020),
11. Ming Du, Aswin C. Sankaranarayanan, and Rama Chellappa, Robust Face
Recognition from Multi-View Videos, IEEE TRANSACTIONS ON IMAGE
PROCESSING, VOL. 23, NO. 3, MARCH 2014.
12. M. Turk and A. Pentland, “Eigenfaces for recognition,” Journal of
cognitive neuroscience, vol. 3, no. 1, pp. 71–86, 1991.
13. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, “Eigenfaces vs.
fisherfaces: Recognition using class specific linear projection,” tech. rep.,
Yale University New Haven United States, 1997.
14. T. Ahonen, A. Hadid, and M. Pietikainen, “Face description with local
binary patterns: Application to face recognition,” IEEE Transactions on
Pattern Analysis and Machine Intelligence, vol. 28, pp. 2037–2041, Dec
2006.
13
15. T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution gray-scale
and rotation invariant texture classification with local binary patterns,”
IEEE Transactions on pattern analysis and machine intelligence, vol. 24,
no. 7, pp. 971–987, 2002.
16. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,”
International journal of computer vision, vol. 60, no. 2, pp. 91–110,
2004.
17. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, “Speeded-up robust
features (surf),” Computer vision and image understanding, vol. 110,
no. 3, pp. 346–359, 2008.
18. R.S., D. (2020, March 31). Attendance Authentication System Using Face
Recognition. Journal of Advanced Research in Dynamical and Control Systems,
12(SP4), 1235–1248.
14
