Mini Project Front Page-1
Mini Project Front Page-1
                                        A
                                    REPORT
                                       On
“Face Recognition-Based Authenticated Voice Assistant
                     System”
    Submitted in Partial Fulfillment of the requirements for the Fifth semester
                   BACHELOR OF ENGINEERING
                              IN
               COMPUTER SCIENCE AND ENGINEERING
                                  Submitted By
   BABU G V                                       1SJ22CS020
   DEEPAK S N                                     1SJ22CS047
   POORNA CHANDRA TEJASWI P                       1SJ22CS118
   SHASHANK S R                                   1SJ22CS146
                                 Carried out at
                                 Project Lab,
                                 Dept of CSE,
                                    SJCIT
CERTIFICATE
This is to certify that the project work entitled “Face Recognition-Based Authenticated
Voice Assistant System”is a Bonafide work carried out by BABU G V(1SJ22CS020),
DEEPAK.S.N(1SJ22CS047),POORNACHANDRATEJASWI.P(1SJ22CS118),
SHASHANK.S.R(1SJ22CS146)                        in partial fulfillment for the award of Bachelor of
Engineering in Computer Science and Engineering of Visvesvaraya Technological
University, Belagavi during the year 2024- 2025. It is certified that all
corrections/suggestions indicated for internal assessment have been incorporated in the
report. The Mini project report has been approved as it satisfies the academic
requirements with respect to Mini project work prescribed for the Bachelor of
Engineering degree in Fifth Semester.
                                                  DEEPAK.S.N
                                                  1SJ22CS032
                                                  SHASHANK.S.R
                                                   1SJ22CS061
                                                 i
                                                ABSTRACT
The primary objective of this project is to develop a system that ensures high accuracy in face recognition
while maintaining low latency for seamless real-time use. The system addresses the growing demand for
secure access to sensitive information and resources across personal, corporate, and smart home environments.
Additionally, it emphasizes safeguarding user privacy by implementing advanced data security measures.
The methodology involves a structured approach comprising requirement analysis, system design,
implementation of facial recognition algorithms, virtual assistant integration, and rigorous testing. The project
also incorporates state-of-the-art machine learning models to improve recognition accuracy under varying
conditions such as lighting, pose, and expression.
This system not only improves security but also enhances convenience, making it a viable solution for modern
authentication challenges. The results demonstrate its effectiveness in delivering a high-performance and
reliable authentication mechanism. The proposed system has significant potential for scalability and future
enhancements, such as incorporating multimodal biometrics and extending functionality to diverse platforms.
                                                          ii
                                     ACKNOWLEDGEMENT
With reverential pranam, we express my sincere gratitude and salutations to the feet of his holiness Paramapoojya
Jagadguru Byravaikya Padmabhushana Sri Sri Sri Dr. BalagangadharanathaMaha Swamiji, his holiness
ParamapoojyaJagadguru Sri Sri Sri Dr. Nirmalanandanatha Maha Swamiji, andSri SriMangalnath Swamiji,
Sri Adichunchanagiri Mutt for their unlimited blessings.
        First and foremost, we wish to express our deep sincere feeling and gratitude to our institution, Sri
Jagadguru Chandrashekaranatha Swamiji Institute of Technology, for providing us an opportunity for
completing the Mini-Project Work successfully.
        We extend deep sense of sincere gratitude to Dr. G T Raju, Principal, SJC Institute of Technology,
Chickballapur, for providing an opportunity to complete the Mini-Project Work.
        We extend special in-depth, heartfelt, and sincere gratitude to HOD Dr. Manjunatha Kumar B H, Head of
the Department, Computer Science and Engineering, SJC Institute of Technology, Chickballapur, for his
constant support and valuable guidance of the Mini-Project Work.
        We convey our sincere thanks to Project Guide Deepthi N, Assistant Professor, Department of
Computer Science and Engineering, SJC Institute of Technology, for his/her constant support, valuable guidance
and suggestions of the Mini-Project Work.
        We also feel immense pleasure to express deep and profound gratitude to Mini-Project Coordinators
Mrs. Bhavya R A, Assistant Professors, Department of Computer Science and Engineering, SJC Institute of
Technology, for their guidance and suggestions of the Mini-Project Work.
        Finally, we would like to thank all faculty members of Department of Computer Science and Engineering,
SJC Institute of Technology, Chickaballapur for their support
We also thank all those who extended their support and co-operation while bringing out this Mini-Project work.
                                       BABU.G.V                 1SJ22CS020
                                       DEEPAK.S.N               1SJ22CS032
                                       POORNA CHANDRA TEJASWI.P 1SJ22CS118
                                       SHASHANK.S.R         1SJ22CS061
                                                           i
                                        CONTENTS
Declaration                                                    i
Abstract                                                       ii
Acknowledgement                                                iii
Contents                                                       iv
                                                 ii
            Proposed System
          Advantages
  5                               SYSTEM DESIGN   15-20
          5.1Project Modules
          5.2Activity Diagram
          5.3UseCase Diagram
          5.4Dataflow Diagram
          5.5Sequence Diagram
  6                             IMPLEMENTATION    21-23
         Algorithm/Pseudo-code module wise
                                   TESTING        24-26
Methods of Testing
Unit Testing
7.1.2Validation Testing
7.1.3Functional Testing
7.1.4Integration Testing
7.1.5User Acceptance
 Testing Test Cases
   8                       PERFORMANCEANALYSIS    27-28
9 CONCLUSION &FUTUREENHANCEMENT 29
BIBLIOGRAPHY 30
                                    APPENDIX       31
       Appendix A: Abbreviation
                                         v
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               Chapter 1: Introduction
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Overview
Problem Statement
The growing reliance on digital systems for critical applications has underscored the
limitations of conventional authentication methods, such as passwords and PINs. These
methods are susceptible to threats like phishing, brute force attacks, and poor password
practices. A need exists for a reliable, efficient, and user-friendly authentication
mechanism that ensures the security of sensitive data without compromising ease of
access.
Integrating face recognition into virtual assistant systems holds immense potential in
transforming user interactions across a variety of contexts, including:
This technology not only enhances security but also provides a seamless and efficient
user experience.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               4
............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
                                      Objectives
Methodology
                                               5
                          Chapter 2: Literature Survey
                                       Introduction
The literature survey examines existing research and technologies in face recognition,
biometric authentication, and virtual assistant systems. This analysis identifies the
strengths and limitations of current systems, providing a foundation for the development
of the proposed solution.
1. Traditional Approaches
           ⚫ Early face recognition methods like Principal Component Analysis (PCA) and
               Linear Discriminant Analysis (LDA) were effective for small datasets but
               struggled with variations in lighting, pose, and expression.
           ⚫ These methods were computationally efficient but lacked the robustness
               required for real-world applications.
           ⚫ Algorithms like YOLO and MTCNN enable efficient face detection and
               recognition in real-time scenarios.
           ⚫ These advancements are widely adopted in surveillance systems and consumer
               devices.
                                               6
                           Biometric Authentication Systems
           ⚫ Assistants like Siri, Alexa, and Google Assistant rely on voice commands for
               interaction but lack robust security mechanisms.
           ⚫ Integrating face recognition into virtual assistants can enhance both security and
               personalization.
2. Challenges in Integration
Identified Gaps
   1. Integrating advanced face recognition algorithms with virtual assistants for secure and
       personalized authentication.
                                               7
  2. Employing optimized models to ensure real-time performance without compromising
     accuracy.
  3. Implementing privacy-preserving measures to enhance user trust and data security.
  Hardware Requirements
        1. Processor: High-performance CPU, such as Intel Core i5 or AMD Ryzen 5
                                                  (or higher).
            2. Memory (RAM): Minimum 8 GB; recommended 16 GB for smoother
                                                  operation.
            3. Storage: SSD with at least 256 GB for faster read/write operations.
     4. GPU: Dedicated GPU (e.g., NVIDIA GTX 1050 or higher) for running deep
                                  learning models efficiently.
             5. Camera: High-resolution webcam for real-time facial data input.
  Software Requirements
      1. Operating System: Windows 10 or Linux-based OS.
      2.     Programming Language: Python (primary) with relevant libraries like
            TensorFlow, Keras, OpenCV, and Numpy.
      3. Frameworks:
                         - TensorFlow or PyTorch for machine learning.
                           - Flask/Django for the application backend.
           4. Database: MySQL or SQLite for managing user data and embeddings.
                5. **IDE/Editor**: PyCharm, VSCode, or Jupyter Notebook.
  Functional Requirements
      1. Facial recognition using real-time video feed.
      2.     Secure authentication based on face recognition.
      3.     Virtual assistant interaction through voice or text.
      4. User-friendly interface for easy navigation and interaction.
      5. Data encryption to protect sensitive user information.
Non-Functional Requirements
                                              8
          1. Performance: High accuracy (target >95%) with low latency (target
              <1s per operation).
          2. Scalability: Support multiple user authentications concurrently.
          3. Reliability: Uptime of >99% under typical operating conditions.
          4. Privacy and Security: Implement advanced data encryption and
              ensure compliance with data protection standards.
Performance Metrics
   1. Accuracy: Minimum target 95% for face recognition under various lighting
                                    and pose conditions.
   2.   Latency: Average response time of <1000 ms.
   3. Error Rates:
                 - False Positive Rate: <2%.
                 - False Negative Rate: <5%.
                                       9
                          Chapter 4: System Analysis
                                 Existing System
Overview
1. Security Vulnerabilities:
3. Scalability Issues:
                                              10
                               Proposed System
Overview
The proposed system integrates face recognition technology with virtual assistant
capabilities to create a secure, efficient, and user-friendly authentication mechanism.
This system addresses the limitations of existing methods by leveraging biometric data,
which is unique, non-transferable, and difficult to forge.
1. Enhanced Security:
3. Scalability:
                                             11
        2. High Accuracy: Advanced deep learning models ensure reliable recognition
        even under varying conditions.
        3. Real-Time Performance: Optimized algorithms enable rapid detection and
        authentication.
        4. Integration with Virtual Assistant: Combines security with personalized
        functionality, enhancing usability.
        5. Data Privacy: Ensures compliance with industry standards for user privacy
        and data protection.
        ⚫   Captures real-time video input and detects faces using advanced algorithms
            like MTCNN or Haar cascades.
        ⚫   Recognizes faces using deep learning models such as FaceNet or DeepFace.
2. Authentication Module:
        ⚫   Verifies user identity by comparing the detected face with the stored facial
            data.
        ⚫   Ensures secure and efficient data handling during the authentication process.
        ⚫   Manages storage and retrieval of user data, including facial features and
            authentication logs.
        ⚫   Implements encryption to ensure data security and privacy.
                                              12
 5. System Interface Module:
                               Activity Diagram
The activity diagram represents the overall workflow of the system from face detection
to authentication and user interaction:
                                             13
                                Activity Diagram
Level 0 DFD
                                            14
                               Level 1 DFD
Sequence Diagram
The sequence diagram illustrates the Flowchart of event scheduling and updating
database:
                                          15
             Chapter 6: Implementation
Sequence Diagram
                       16
The implementation of the "Face Recognition-Based Authentication Virtual Assistant
System" follows a systematic approach, illustrated in the sequence diagram. This
highlights interactions between various system components, including the user, the face
recognition module, the authentication system, and the virtual assistant.
                Algorithm/Pseudo-code (Module-wise)
Face Detection and Recognition Module
Algorithm:
                        Authentication Module
     Pseudo-code:
     Input: Captured facial features, Database of registered users
     Output: Authentication result
          1. Start
          2. For each captured facial feature:
                 a. Calculate similarity with stored embeddings using
                        cosine similarity.
                  b. If similarity > threshold:
                          Authenticate user.
                               Allow access to system.
                      Else:
                              Deny access.
           3. End
Algorithm:
                                             17
1. Receive input (voice or text) from the user.
2. Authenticate the user using the face recognition module.
3. Fetch user-specific data or preferences from the database.
4. Generate a personalized response based on the input.
5. Output the response via text or speech.
Pseudo-code:
1.   Start
2.   Encrypt all user data before storage.
3.   Store facial embeddings with associated user IDs.
4.   Provide secure retrieval methods for authentication and logs.
5.   End
                         CHAPTER 7: TESTING
                           Methods of Testing
                                            18
Unit Testing
Validation Testing
Test the system against known user data to validate accuracy and reliability of
authentication.
Functional Testing
Verify that all functionalities, including face recognition, authentication, and virtual
assistant integration, work as expected.
Integration Testing
Ensure smooth interaction between different modules, such as face recognition and the
virtual assistant.
Engage real users to evaluate the system's usability, response times, and overall
experience.
Test Case
                                              19
Overview
Performance Metrics
                              Accuracy Evaluation
Testing Dataset
Results:                                                         OverallAccuracy:95.1%
                          Test Scenario           Accuracy (%)
                                 Latency Analysis
                                             20
       Measurement
⚫ Average time measured for face detection, recognition, and authentication combined.
                              Scalability Testing
Methodology
10 950 30
100 1200 50
500 2000 80
Observations
The system performs optimally for up to 100 users with minimal impact on response
time. Performance degrades gradually beyond 500 users, indicating the need for resource
scaling.
Observations
⚫ The system exhibits a low error rate, ensuring reliability in user authentication.
System Uptime
Performance Summary
The system demonstrates high accuracy and low latency under normal operating
conditions. While scalability is effective for small to medium user bases, additional
resources may be required for larger deployments. Error rates are minimal, ensuring
robust security and reliability.
                                               22
                Chapter 8: Conclusion and Future Work
                                    Conclusion
The system’s performance metrics confirm its viability for real-world applications in
personal devices, smart homes, corporate environments, and beyond. By combining
cutting-edge technology with a focus on usability and security, the project sets a strong
foundation for modern authentication systems.
                                             23
                                  Future Work
While the system meets its primary objectives, there are opportunities for further
improvement and expansion. Key areas for future work include:
1. Multimodal Authentication:
2. Cloud Integration:
5. Performance Optimization:
                                             24
                                    Bibliography
1.   Abate, A. F., Nappi, M., Riccio, D., & Sabatino, G. (2007). 2D and 3D face
     recognition: A survey. Pattern Recognition Letters, 28(14), 1885-1906.
     https://doi.org/10.1016/j.patrec.2007.04.018
2.   Schroff, F., Kalenichenko, D., & Philbin, J. (2015). FaceNet: A unified embedding
     for face recognition and clustering. Proceedings of the IEEE Conference on
     Computer Vision and Pattern Recognition (CVPR), 815-823.
     https://doi.org/10.1109/CVPR.2015.7298682
3.   Taigman, Y., Yang, M., Ranzato, M. A., & Wolf, L. (2014). DeepFace: Closing the
     gap to human-level performance in face verification. Proceedings of the IEEE
     Conference on Computer Vision and Pattern Recognition (CVPR), 1701-1708.
     https://doi.org/10.1109/CVPR.2014.220
4.   Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep Face Recognition. British
     Machine Vision Conference (BMVC), 41.1-41.12. https://doi.org/10.5244/C.29.41
5.   Dalal, N., & Triggs, B. (2005). Histograms of Oriented Gradients for Human
     Detection. Proceedings of the IEEE Computer Society Conference on Computer
     Vision and Pattern Recognition (CVPR), 886-893.
     https://doi.org/10.1109/CVPR.2005.177
6.   Viola, P., & Jones, M. J. (2001). Rapid object detection using a boosted cascade of
     simple features. Proceedings of the IEEE Computer Society Conference on
     Computer Vision and Pattern Recognition (CVPR), I-511-I-518.
     https://doi.org/10.1109/CVPR.2001.990517
7.   Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
     https://www.deeplearningbook.org/
8.   He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image
     Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern
     Recognition (CVPR), 770-778. https://doi.org/10.1109/CVPR.2016.90
                                            25
9.   Google Cloud AI. (n.d.). Using face recognition APIs for personalized applications.
     Retrieved from https://cloud.google.com/ai
11. OpenCV. (n.d.). Open Source Computer Vision Library. Retrieved from
    https://opencv.org
                                           26
                                   Appendix
                     Appendix A: Abbreviations
Abbreviation                   Full Form
AI             Artificial Intelligence
CNN            Convolutional Neural Network
DFD            Data Flow Diagram
FNR            False Negative Rate
FPR            False Positive Rate
GPU            Graphics Processing Unit
MTCNN          Multi-Task Cascaded Convolutional Networks
PCA            Principal Component Analysis
RAM            Random Access Memory
SSD            Solid State Drive
27