AI Mirror for Video Self-Modeling Therapy
SBoard is a mobile app that helps children practice communication skills through live conversations with an AI companion. It combines video self-modeling (recording the child in real time) with voice-based AI interaction so parents and therapists can capture meaningful moments, track progress, and revisit clips later.
SBoard supports children who benefit from practicing speaking in low-pressure, scenario-based conversations. The app:
- Encourages communication — A warm AI friend (“Mirror”) talks with the child in real time, listens, and responds naturally.
- Captures progress — Each session is recorded as video; the AI marks “positive moments” (e.g., new words, clear sentences) as bookmarks so parents can review highlights.
- Offers guided missions — Structured roleplay scenarios (e.g., “Say Hello,” “Order at a Toy Shop”) with clear objectives and scores.
- Tracks milestones — Sessions and clips are stored in the cloud so parents can see streaks, session counts, and favorite moments.
- Parent signs in and subscribes to Circle Space (RevenueCat).
- Parent adds one or more children (name, age, avatar).
- Parent taps CHAT in the bottom nav → chooses Free Talk or a Mission → starts an AI Mirror session.
- Child talks with the AI while the front camera records them. The AI speaks, listens via the mic, and celebrates communication attempts.
- Session ends → parent can save a full clip or individual “positive moment” clips.
- Memories screen shows recent sessions and clips so parents can rewatch and track progress.
| Screen | Purpose |
|---|---|
| Auth | Sign in / sign up (Firebase Auth) |
| Onboarding | First-launch intro to AI Mirror, Memories, Missions |
| Paywall | Subscription gate (RevenueCat) — must subscribe to use the app |
| Shell | Main layout: Memories tab + Profile tab; CHAT opens AI Mirror |
| Mission Picker | Modal to choose a guided mission or Free Talk |
| AI Mirror | Live AI chat + selfie camera; records session; saves clips |
| Memories | Recent sessions, video clips, progress stats, favorites |
| Profile | Parent account, subscription, add/manage children |
Cold start
│
├─► Not signed in ──► Auth Screen (Sign In / Sign Up)
│
└─► Signed in ──► Entitlement check (RevenueCat)
│
├─► No subscription ──► Paywall (must subscribe)
│
└─► Has subscription ──► First time?
│
├─► Yes ──► Onboarding (3 screens)
│
└─► No ──► Shell (Memories / Profile)
│
├─► No children ──► "Add Your Child" CTA
│
└─► Has children ──► Tabs: Memories | Profile
│
CHAT (bottom nav)
│
└─► Mission Picker (Free Talk or Mission)
│
└─► AI Mirror Screen
│
├─► Connect Gemini
├─► Start camera + mic
├─► Stream audio to AI
├─► Receive AI speech + text
├─► Record video, bookmark positive moments
└─► End session → save clip → back to Shell
- Google Gemini Live API (via
firebase_ai) — bidirectional, real-time multimodal model (gemini-live-2.5-flash-preview). - Modalities: Text + audio in/out (voice conversation).
- Streaming: Microphone PCM → Gemini; Gemini → text transcript + audio.
┌─────────────────────────────────────────────────────────────────┐
│ AI Mirror Screen │
├─────────────────────────────────────────────────────────────────┤
│ GeminiLiveService RecordingService │
│ - connect() - initCamera() │
│ - startMic() / stopMic() - startRecording() │
│ - receive() loop - addBookmark() │
│ - streams: - stopRecording() → save clip │
│ • transcripts │
│ • positiveMoments │
│ • missionResults │
│ • objectivesHit │
└─────────────────────────────────────────────────────────────────┘
│ │
▼ ▼
Firebase AI (Gemini Live) Camera + Storage
- Live session - Firestore sessions
- System prompt - Storage clips
- Audio PCM 16kHz mono - ClipService
The AI is driven by structured system prompts that define its role, tone, and behavior.
- Role: Warm friend “Mirror” for video self-modeling.
- Behavior: Short replies (1–2 sentences), age-appropriate, encouraging.
- Task: Celebrate communication attempts and mark them with
[POSITIVE_MOMENT: description]so the app can bookmark and toast them. - Rules: Never mention markers to the child; gently re-engage if the child is quiet.
- Role: Character from a scenario (e.g., Friendly Neighbor, Toy Shop Owner).
- Behavior: Stay in character, guide toward objectives, use simple vocabulary.
- Markers:
[POSITIVE_MOMENT: description]— progress in communication.[OBJECTIVE_HIT: objective text]— child completed an objective.[MISSION_COMPLETE: score | feedback]— mission done (score 1–3).
- App parses these and shows mission summary, completed objectives, and positive-moment toasts.
The model emits hidden markers in its text output. The app parses them with regex and reacts:
| Marker | Purpose |
|---|---|
[POSITIVE_MOMENT: ...] |
Triggers toast + bookmark in video |
[OBJECTIVE_HIT: ...] |
Marks objective complete in mission UI |
[MISSION_COMPLETE: score | feedback] |
Ends mission, shows summary sheet with score |
Markers are stripped before the text is shown to the user, so the child never sees them.
- Input:
recordpackage streams 16-bit PCM, 16 kHz, mono. - Output: Gemini returns audio; app receives it and can route it to
just_audiofor playback (or show text transcription).
- Sessions: Stored in Firestore
sessions(childId, start/end time, missionId, score, positiveMomentsCount, objectivesHit). - Clips: Uploaded to Firebase Storage, metadata in Firestore
clips. - Children: Firestore
childrensubcollection per parent.
Predefined roleplay scenarios with objectives and difficulty levels:
| Mission | Persona | Example Objectives |
|---|---|---|
| Say Hello | Friendly Neighbor | Say hello, tell your name, ask how they are |
| How Do You Feel? | Kind School Counselor | Name a feeling, explain why, ask how they feel |
| I Want... | Cheerful Toy Shop Owner | Say what you want, use “please,” say “thank you” |
| My Morning | Breakfast Buddy | Describe morning activities, use time words |
| At the Playground | New Friend | Introduce yourself, ask what to play, suggest a game |
| Snack Time | Cafeteria Helper | Ask for snack, ask for drink, say please/thank you |
| Show and Tell | Curious Classmate | Name favorite thing, describe it, explain why |
| When I Feel Scared | Caring Teacher | Name what scares you, describe feelings, ask for help |
- Flutter — UI
- Firebase — Auth, Firestore, Storage, Crashlytics, AI (Gemini)
- RevenueCat — Subscriptions (Circle Space entitlement)
- record — Microphone streaming
- camera — Selfie video recording
- shared_preferences — Onboarding and entitlement cache
cd sboard
flutter pub get
flutter runRequires:
- Firebase project with Auth, Firestore, Storage, AI (Gemini) enabled
- RevenueCat project with “Circle Space” entitlement
- Camera and microphone permissions (iOS Info.plist configured)