St.
Vincent Pallotti College of Engineering and Technology
DEPARTMENT OF ARTIFICIAL INTELLIGENCE
Academic Year 2024-25
Project Seminar
On
“AI-Powered Medical Scribe”
Name of Guide: Project Members:
Prof. Snehal Awachat Vaishnavi Amle (23010043)
Ayush Nair (23010054)
Rishi Tapase (23010028)
Maithilee Bansod (23010044)
Table of Content
• Introduction
• Problem Definition
• Literature Survey
• Proposed System Architecture
• Project Modules
• Techniques Used
• Screenshots of work done
• References
Introduction
AI-powered medical scribe is a smart system that helps doctors by
automatically writing down patient details during a consultation. It listens to the
doctor-patient conversation, understands medical terms, and creates accurate
notes. This saves time and allows doctors to focus on patient care instead of
paperwork.
Problem Definition
Doctors spend a lot of time writing patient records, which reduces the
time they can spend with patients. Manual note-taking can also lead
to errors. An AI-powered medical scribe can solve this problem by
automatically generating medical notes, reducing the workload on
doctors and improving accuracy.
Literature Survey
Sr.
no
Title of Paper Author Name Year Findings
Artificial Intelligence-Driven AI scribes improve dermatologist workflow and patient interaction. They
1 Digital Scribes in Clinical David Y. Cao 2023 reduce manual note-taking, allowing doctors to focus on diagnosis. This
Documentation leads to better patient satisfaction and time efficiency.
Ambient Artificial
AI scribes reduce documentation workload for clinicians. Ambient
Intelligence Scribes to Brian Hoberman,
2 2023 listening supports passive transcription. Clinicians can review and edit
Alleviate the Burden of MD
auto-generated notes with ease.
Documentation
The Utility and Implications AI can generate accurate medical notes from provider–patient
Puneet Seth,
3 of Ambient Scribes in 2024 conversations. It enhances real-time note creation. The study also notes
Romina Carretas
Primary Care increased accuracy and reduced admin burden.
Challenges of Developing a Identifies challenges in speech-based documentation systems. Highlights
Digital Scribe to Reduce Juan C. Quiroz, include data privacy concerns, transcription accuracy, and voice
4 2019
Clinical Documentation Liliana Laranjo differentiation. Recommends careful model training and clinical
Burden validation.
AI can assist in structuring and writing medical notes efficiently. Focuses
Learning to Write Notes in
5 Peter J. Liu 2018 on learning-based systems. Notes improved learning curve for new
Electronic Health Records
doctors using EHRs with AI guidance.
Proposed System Architecture
Project Modules
Project Modules User Input
Module
1. User Input Module:- Collect doctor and patient names using text
input.Start/stop recording via button.
Audio Recording
2. Audio Recording Module:- Uses sounddevice to capture audio input Module
from mic.Saves to .wav or .mp3.
Transcription
3. Transcription Module:- Uses OpenAI's Whisper model for speech-to- Module
text.Model options: "tiny", "base", "small", "medium", "large“
4. Prescription Module:- According to the diagnosis it prescribes the Prescription
medicines Module
5. Note Structuring Module:- Formats output into readable
Note Structuring
dialogue.Displays on-screen or allows download as text/PDF.
Module
6. Frontend Web Interface (Streamlit):- Simple web interface to collect
names, show notes, and run the app. Frontend Web
Interface
Techniques Used
Speech Recognition
OpenAI Whisper: Converts spoken language into text.
Model loads: whisper.load_model("base")
Handles multiple languages & noisy environments well.
Summarization (Text Processing)
Using Hugging Face Transformers to summarize long transcripts into short, meaningful notes.
Audio Recording
sounddevice records real-time audio
scipy.io.wavfile to save and load .wav format
PDF Generation
•Using FPDF library to automatically generate structured prescription PDFs from conversation data.
Web Framework
Streamlit: Python-based lightweight web framework for data apps
Super easy to build UI
Tools & Libraries Used
Tool/Library Purpose
Streamlit UI Framework
sounddevice Audio Recording
whisper (OpenAI) Transcription
transformers (Hugging Face) Summarization
FPDF PDF Generation
os / time / datetime File Handling
Screenshots of work done
References
1 D. Y. Cao, "Artificial Intelligence-Driven Digital Scribes in Clinical Documentation," Int. J. Med. Inform.,
2023. Available: [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10988030/].
2 B. Hoberman, "Ambient Artificial Intelligence Scribes to Alleviate the Burden of Documentation," NEJM
Catalyst, 2023. Available: [https://catalyst.nejm.org/doi/full/10.1056/CAT.23.0404].
3 P. Seth, R. Carretas, "The Utility and Implications of Ambient Scribes in Primary Care," JMIR AI, 2024.
Available: [https://ai.jmir.org/2024/1/e57673].
4 J. C. Quiroz, L. Laranjo, "Challenges of Developing a Digital Scribe to Reduce Clinical Documentation
Burden," npj Digit. Med., 2019. Available: [https://www.nature.com/articles/s41746-019-0190-1].
5 P. J. Liu, "Learning to Write Notes in Electronic Health Records," arXiv preprint, 2018. Available:
[https://arxiv.org/abs/1808.02622].
09-03-2024 8
Thank you!
09-03-2024 9
