An Intelligent Attendance Tracking System Using

Masked Face Recognition Technology

Dilshad Begum Nisarga R Jeevika M
Department of Computer Department of Computer Department of Computer
Science and Engineering Science and Engineering Science and Engineering
Ghousia College Engineering Ghousia College Ghousia College Engineering
Engineering

Keerthi M Harshitha S
Department of Computer Department of Computer
Science and Engineering Science and Engineering
Ghousia College Engineering Ghousia College Engineering

Abstract— The COVID-19 pandemic has altered lifestyle images making CNN applicable for facial recognition applications.
patterns remarkably, like making it a habit to use face masks in The researchers developed better recognition algorithms that enable
public settings to control viral spread. Nonetheless, this one to recognize faces even while they are partial by training these
situation has presented challenges for face recognition systems models with huge datasets with masked and, unmasked facial
that identify and authenticate individuals. To alleviate the images.
problem, a deep learning-based model using Convolutional The field where this technology is primarily applied is education as
Neural Networks is developed to detect and recognize faces with taking attendance is one of the most important activities carried out
masks. The system even tries to assess whether the mask is within an academic setting. The traditional way of documenting
correctly worn.This system operates with several image datasets attendance through calling out names by gazing through the
and implements a binary classification model that discriminates attendance list or using biometric systems like fingerprint scanning
between masked and unmasked faces with an excellent is time-consuming and subject to a big percentage of error, along
achievement of 99.77% accuracy. The next multi-class model with proxy attendance. Although other options such as RFID and iris
further classifies the masked faces as correctly masked, scanning have been tried, they have from time to time proved
incorrectly masked, and non-masked with an accuracy of complicated due to the need for toting along external apparatus and
99.5%. Furthermore, on subject identification, it achieves a have tended to prove intrusive. The attendance management system
mean accuracy of 97.98%, proving efficient in face localization based on facial recognition allows methods that are non-intrusive,
automated, and effective. It detects students present within an
and recognition tasks.
outdoor hall or classroom using a live soda stream and automatically
registers them within an attendance record.
Studies show that face recognition technology faces new
challenges due to the wearing of masks that conceal a major
part of the face. Some studies reported high accuracy in II. RELATED WORKS
recognizing only the top half of the face. This method relies on
the visible portion of the masked face for facial recognition Face detection and recognition have witnessed considerable
purposes. The research involved recording videos of 125 improvement over the time, with researchers employing various
subjects and deploying YOLOv4 for face detection.. techniques so as to enhance accuracy, speed, and robustness. Due to
the presence of facial features, many have scrutinized different
methodologies in order to improve facial recognition under critical
Keywords— Face recognition, deep learning, masked face conditions like occlusions, pose, and low-light situations.
detection, attendance system, InceptionResNetV1.
The first stage consisted of a texture-based approach for face
detection with pixel-space methods being dominated by domain
compression techniques. This was undertaken in light of many
aspects that could affect their effectiveness: block quantization,
I. INTRODUCTION feature selection, and detection block size. Other hybrid approaches
Following the onset of the COVID-19 pandemic, important proved more efficient, as they combined texture-based detection
challenges were posed in various sectors that require identity with other features, like face color information. To ascertain
verification. Social distancing mandates and applicable safety differences in human facial structures, progressive image-based
protocols weakened the plausibility of traditional methods of detection techniques took on the challenge affecting the robustness
verification, which include fingerprint scans and a manual of computer face recognition. In parallel, the introduction of
attendance record, since they all posed a serious risk of infection machine learning and deep learning models increased the accuracy
transmission. The face masks-worn that became a mandatory of detection, allowing the automation of processes such as face and
recommended protective measure by the health authorities all over tagging identification.
the world-additionally worsened face identification. Due to the
relatively large part of the face hidden, conventional facial Viola and Jones proposed a real-time face detection algorithm that
recognition systems had difficulties providing accurate could process hundreds of frames per second, reducing computation
identification metrics. time and improving detection accuracy. Large datasets like WIDER
FACE assisted in extensive training during deep learning
Artificial intelligence heat and deep learning have been utilized to implementation, adding diversity through various face annotations.
address the issues faced by masked face recognition systems. Deep In subsequent research studies, convolutional neural networks
learning techniques, self-embodied most notably in the teaching of (CNNs) and feature quantization techniques have improved the
deep convolutional neural networks (CNNs), have proven to be classification accuracy of Multilayer Perceptron (MLP) classifiers.
quite effective in a number of applications, including classification,
identification, and face image segmentation. The abstract of neural The advent of deep learning brought about the combination of
processing mechanisms based on biological concepts in CNN multiple algorithms into hybrid face recognition systems in order to
architecture will extract hierarchical features from the original establish robustness against variations. Some of the techniques
implemented pretrained CNNs, such as ResNet50, for feature
extraction, followed by classifiers like Support Vector Machines
(SMVs) and different ensemble techniques to improve accuracy.
Additional deep learning architectures such as Inception-v3 and
Xception were applied to masked face detection and accomplished
high rates of recognition. Among other interventions, studies
proposed augmenting datasets to quasi-generate real-life scenarios,
so as to enhance generalization of the model.
Real-time face recognition in constrained situations have also been
the focus of attention in recent studies. These systems either
integrate some kind of hardware component, such as Raspberry Pi,
for edge-based face detection or send an alert upon detecting a
person who is not wearing a mask. Emotion recognition and Fig 1 Simplified Recognition Flow-Chart
behavior analysis are a few other backgrounds pursued by
developing multilevel models based on Adaboost and SVM 1. Data Acquisition and Pre-processing phase
classifiers. Generative models, such as GAN, have been worked on
At first, students were asked to register during which multiple
for the reconstruction of occluded facial features, which works to
the improvement of recognition accuracy in masked facial images of their faces were captured under various lighting
recognition. conditions with different angles and expressions. These multiple
images were systematically pre-processed so that the Region of
Graph-based deep learning models have also been introduced for Interest (ROI) would reside well within the defined parameters
face clustering and recognition. A method using residual graph retaining characteristics of facial features only. These processes
convolutional networks go onto automate facial clustering in a way provide the following:
that's proved a lot better than classical clustering methods.
Techniques such as RetinaFace and VGGFace2 further optimized • All images shared a standardized dimension, such that there
masked face detection with state-of-the-art accuracy. would be uniformity across the set.
With regard to attendance management, different automation-on- • In terms of color space, while in RGB modeling, the images are
automated attendance systems using facial recognition have indeed transformed to grayscale for fast computations, removing
been developed. Some of them involve a combined form of face redundancy.
recognition along with RFID technology for secure and reliable
attendance tracking. There are also variations in which iris • Augmentation of data through techniques such as flipping,
biometrics is said to receive individual identification during shifting, modification in color can strengthen the model against
recognition through the utilization of unique characteristics of the generalization and robustness.
eye. Face recognition-based attendance systems have adopted
different algorithms including Viola-Jones, Histogram of Oriented • Face region extraction along with a feature-based classifier to get
Gradients, and support vector machine classifiers for increasing rid of unwanted background details.
accuracy in diverse environmental conditions. The combination of
Discrete Wavelet Transforms and Discrete Cosine Transforms has The Haar cascade classifier was used for face detection in the KYC
also been found useful for extracting facial features efficiently. process, which has been trained on multiple face structures, thus
enabling effective recognition of frontal faces.
Other techniques involve two-stage object detection
methodologies for improved accuracy in facial recognition. Region-
based RNNs coupled with Fast RNN and Faster RNN have been
more extensively used for refining face detection. Though Mask 2. Masked Face Classification
R_CNN extends this capability to include segmentation masks and
improves recognition performance, single-stage detectors like After face detection, the next step is to classify whether the person
YOLO and SSD are getting a lot popular nowadays owing to their is wearing the mask correctly, incorrectly, or not at all. This
faster, more efficient functionality in real time. classification is achieved through Convolutional Neural Networks
(CNN), which are deep learning-based architectures. The
In recent times, two methods were developed to improve face classification employs a pre-trained feature extractor for effective
detection accuracy: dataset preprocessing and feature extraction feature extraction.
through improved techniques and the application of transfer
learning. Pre-trained models based on ResNet, Inception, and Learning in a proficient manner. The classification process is
EfficientNet architectures exhibit excellent performance in the divided into the following steps:
domain of object recognition. The introduction of deep learning
models leveraging surveillance systems allows, for real-time face • Feature vectors are obtained using deep learning models like
tracking and monitoring, attendance-taking. VGG-16 that are already trained and work well for classification
Overall, advances in deep learning and computer vision have problems involving images.
further improved the accuracy and efficiency of such systems, so • The CNN processes those feature vectors so that it can
that all the different models have been studied from classical differentiate between the masked, unmasked, and improperly
feature-based approaches to modern CNN architectures and hybrid masked faces.
models; this led to the creation of systems that work better in real
time. The incorporation of these techniques into attendance • The Stochastic Gradient Descent (SGD) optimizer is employed to
monitoring systems ensures full automation in identifying students fine-tune the network weights for improved classification
and further cements the viability of these solutions within today's performance with faster convergence.
security and monitoring application.

III. PROPOSED APPROACH

In this phase, we integrate masked face detection and recognition

with automated attendance tracking in a three-fold approach. It
consists of all four stages to improve the accuracy, efficiency, and
robustness of real-world classroom applications.

Fig 2 classification whether the person is wearing the mask correctly.l.

 3. Face Recognition for Identity Verification 2) Analysis of Masked Face Recognition
The ability of the system to recognize individuals with and
Once the system determines if a student is wearing a mask, the next
without masks was assessed through an experimental model based
phase is to recognize the person. The recognition module includes:
on deep learning and implemented on a hybrid dataset with both
• Feature Extraction: The system extracts the critical features of masked and unmasked subjects’ images. The results showed that
the face from the pre-processed images through a convolutional recognition accuracy increased significantly on large training
neural network (CNN) based algorithm. datasets.To refine performance, various approaches for pre-training
architectures were tested. Optimization strategies such as
• Database creation: A database of the feature vectors of registered regularization techniques and adjustments to batch size were carried
students is created against which identification would be out to augment recognition. Some architectures performed better
performed in real-time. than others as explorers in stronger generalization, especially with
respect to that faced properly separating their masked faces from
• Matching Identity: While taking attendance, the facial features faces.A comparative study on the various architectures of the
detected are matched against the records stored in the database to models was designed in consideration of hyper-parameters such as
determine the identity of the student. epoch number, batch size, and optimization algorithms.
The results emphasized that deep networks fine-tuned reach
• Validation Mechanisms: The system accommodates approaches better accuracy, especially in masked face identification.
of identification (who is this person?) and verification (is this the
correct person?), ensuring high-level accuracy in the presence of
masks. 3) Identification of Incorrect Mask Usage
The Local Binary Pattern Histogram (LBP) is deployed for facial To ensure thorough monitoring of compliance with mask-
recognition to enhance recognition performance by representing an wearing protocols, the object was further extended to cover cases of
image in terms of texture patterns. incorrect mask usage. Two separate datasets were used to train and
validate the model. The model achieved excellent accuracy when
4. Attendance Automation trained as a binary classifier (mask vs. no mask); there was a slight
decrease in accuracy when extended to a three-class classification
After the identification of a student, their attendance is (mask, no mask, improper mask usage).Moreover, testing on an
automatically marked on an attendance register. The attendance external dataset revealed that the differences in designs and patterns
procedure encompasses: in the use of masks affect recognition accuracy. Some other types of
protection equipment brought further challenges, thus suggesting
• The student’s ID is marked present in an Excel sheet or a that the system can further upgrade itself by an enhancement in
database. feature extraction for complicated classifications of mask styles.
• If a student’s face is detected but does not match any student in
the stored database, the student is marked absent.
The system prepares the attendance reports and automatically
emails the faculty members about students who had been absent.
Thus, this contactless attendance management system will reduce
human efforts by reducing errors and speeding up the process of
student record maintenance.This methodology promises distinct
representation of the concepts while staying approximately
technically faithful.

Fig 3 Accuracy across models

IV. EXPERIMENTS B. Mobile-Based Model Optimization and Deployment
After performance evaluation, the optimized model was
deployed into the mobile environment to enable real-time
An indirect narration of the experimental results is presented recognition. This was done by integrating Lightweight Deep
here, which does not lose any significant findings of such results: Learning Models into an Android-Based Application. The system
Experimental Results was made to work on real-time video frames with as low latency as
possible and with high accuracy.
Various experiments showed the effectiveness of the proposed
system by assessing mask detection and masked face recognition The app presents real-time facial detection and identity
capabilities. Varying architectures and techniques were considered recognition, with the output being visibly marked by color codes for
to enhance performance under real-world constraints such as various mask status. Furthermore, to ensure the model worked well
diverse dataset sizes and complexities of the images as well as the on mobile devices, model compression techniques helped in
deployment on mobile devices. reducing computational constraints without sacrificing performance.
The system was thus able to deliver good speed and accuracy and
could suit large-scale deployment in educational and public
A. Selection and Evaluation of Base Model institutions.
1) Partial Face Recognition Performance
The first experiment was designed to recognize individuals on
the basis of the eye-region only, considering problems of face
occlusion arising from masks. For the experiments, cropped eye-
region images of the subjects were built into the training dataset.
The resolution was lowered, causing a loss in performance
concerning accuracy. Super-resolution techniques were applied to
boost recognition.Additionally, the influence of dataset size on
performance was analyzed. After reducing the number of subjects,
slight variances in accuracy resulted. Findings show that the
method was light-weight, efficient, and worked well when a
diverse dataset was trained.

Fig 4 Face recognized,mask not detected.

 Fig 8 Admin log-in to track attendance
Fig 5 Masked face recognized.

Fig 9 The tracked attendance data

V. CONCLUSION
Fig 6 Mask detected face not recognized The study describes an intelligent tracking system based on
the face mask. The system tackles the issue of identity verification in
an environment of face mask-wearing rightly and accurately without
any human interference. It is based on several advanced
architectures combined with transfer learning to perform
exceedingly well while being efficient in real-time applications. It
advances the idea that using conventional networks in tandem with
transmission learning is a step ahead in face recognition accuracy
and efficiency on real-time applications. Performance is improved
because of its debut-determined models and homogeneous dataset,
again showing traits of facing reliable identification even under
partial facial occlusion.
Instead, the system can find application beyond attendance tracking
by integration into high-resolution surveillance systems for
extensive security applications. Future enhancements may bring in
improving facial landmark detection under masks, computational
speed for mobile applications, and support for other biometric
characteristics like emotion and behavior detection. This work meets
the real-time goals of working against data and time constraints and
develops potential solutions for performance modeling toward
Fig 7 Mask not detected face not recognized scalable and cost-effective masked face recognition in educational,
healthcare, and public safety applications.
 REFERENCES
[1] G. J. Chowdary, N. S. Punn, S. K. Sonbhadra, and S. Agarwal, ‘‘Face
mask detection using transfer learning of Inceptionv3,’’ in Proc. Int.
Conf. Big Data Anal. Cham, Switzerland: Springer, 2020, pp. 81–90.
[2] C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A.
Acosta,
A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi, ‘‘Photo-realistic
single image super-resolution using a generative adversarial
network,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.
(CVPR), Jul. 2017,
pp. 4681–4690.
[3] B. Batagelj, P. Peer, V. Štruc, and S. Dobrišek, ‘‘How to correctly
detect face-masks for COVID-19 from visual information?’’ Appl.
Sci., vol. 11, no. 5, p. 2070, Feb. 2021.
[4] S. Ge, J. Li, Q. Ye, and Z. Luo, ‘‘Detecting masked faces in the wild
with LLE-CNNs,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.
(CVPR), Jul. 2017, pp. 2682–2690.
[5] S. Yang, P. Luo, C. C. Loy, and X. Tang, ‘‘WIDER FACE: A face
detection benchmark,’’ in Proc. IEEE Conf. Comput. Vis. Pattern
Recognit. (CVPR), Jun. 2016, pp. 5525–5533.
[6] Siswanto, Adrian Rhesa Septian, Anto Satriyo Nugroho, and
Maulahikmah Galinium. "Implementation of face recognition
algorithm for biometrics based time attendance system." 2014
International Conference on ICT For Smart Society (ICISS). IEEE,
2014.
[7] Lukas, Samuel, et al. "Student attendance system in classroom using
face recognition technique." 2016 International Conference on
Information and Communication Technology Convergence (ICTC).
IEEE, 2016.
[8] https://becominghuman.ai/face-detection-using-opencv-with-haar-
cascade-classifiers-941dbb25177
[9] https://www.superdatascience.com/blogs/opencv-face-recognition

[10] Salim, Omar Abdul Rhman, Rashidah Funke Olanrewaju, and Wasiu
Adebayo Balogun. "Class attendance management system using face
recognition." 2018 7th International Conference on Computer and
Communication Engineering (ICCCE). IEEE, 2018
[11] [1]Recognition with Alarm System using Deep Learning," 2020 11th
IEEE Control and System Graduate Research Colloquium (ICSGRC),
pp. 106-110, 2020.
[12] [2]K. Suresh, M.B. Palangappa, S. Bhuvan, "Face Mask Detection by
using Optimistic Convolutional Neural Network," 6th International
Conference on Inventive Computation Technologies (ICICT), pp.
1084- 1089, 2021.
[13] [3]P. Nagrath, R. Jain, A. Madan, R. Arora, P. Kataria, J. Hemanth,
"SSDMNV2: A real-time DNN-based face mask detection system
using single shot multibox detector and MobileNetV2," Sustainable
cities and society, vol. 66, pp. 102692, 2021.
[14] [4]G.J. Chowdary, N.S. Punn, S.K. Sonbhadra, S. Agarwal, "Face
mask detection using transfer learning of inceptionv3," International
Conference on Big Data Analytics, pp. 81-90, 2020.
[15] [5]M. Loey, G. Manogaran, M.H.N. Taha, N.E.M. Khalifa, "Fighting
against COVID-19: A novel deep learning model based on YOLO-v2
with ResNet-50 for medical face mask detection," Sustainable cities
and society, vol. 65, pp. 102600, 2021.
[16] [6]M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, L.C. Chen,
"Mobilenetv2: Inverted residuals and linear bottlenecks," IEEE
Conference on Computer Vision and Pattern Recognition, pp. 4510-
4520, 2018.