Voice-first AI cooking assistant for iOS. Plan your weekly meals and cook along step-by-step as Chomp gets to know your cooking and taste preferences.
Chomp (previously named weat) is a SwiftUI iOS app that uses OpenAI's Realtime API (gpt-realtime-2) over WebRTC for low-latency voice conversations. Uses a Patch of the swift-realtime-openai library. Uses Supabase backend for memory, meal plans, and cooking history. Remembers what you like and don't like over time through extracting facts from natural conversation.
Chomp introduces itself and prompts the user to enter their name. It then asks the user a series of questions to extract and populate initial facts (dietary restrictions, allergies, equipment, cuisines, skill level, time) to build the meal plan.
The MealPlanView is the Home view that displays the meal plan that Chomp creates. The user can modify the meal plan conversationally by starting a Plan with Chomp session. Chomp edits the meal plan in real time through tool calls.
When the user taps Cook Now on any meal card, Chomp will guide the user through the recipe conversationally. Chomp can receive and show images (currently just through querying Pexel) and set timers through AlarmKit. The user can rate and add notes about the recipe when they end the session.
The user can view their past recipes by opening their History view on the top-left icon of the Home view.
The user can view their extracted facts by opening their Profile view on the top-right icon of the Home view.
The three conversational user journeys — Onboarding, Planning, Cooking — all use the same (ConversationViewModel + WebRTC + function tools). What differs is the prompt, the tool set, the hosting screen, and the end-of-session hook.
┌─────────────────────────────────────────┐
│ iOS App (weat) │
│ │
┌───────────┐ │ ┌───────────────────────────────────┐ │
│ Apple ID │◀──┼──│ AuthService (Sign in with Apple) │ │
└───────────┘ │ └───────────────────────────────────┘ │
│ │
│ ContentView ── routes by auth + │
│ ── onboarding state │
│ │ │
│ ├─ SignInView │
│ ├─ OnboardingView │
│ └─ HomeView → MealPlanView │
│ │ │
│ └─ ConversationView│
│ │
│ ConversationViewModel (one per session)│
│ ├─ Conversation (patched WebRTC)│
│ ├─ LiveActivityManager │
│ ├─ MealPlanService │
│ ├─ MemoryService │
│ └─ AlarmService (cook only) │
└─────────────────────────────────────────┘
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌───────────┐ ┌──────────────┐
│ Supabase │ │ OpenAI │ │ Pexels │
│ Auth (SIWA) │ │ Realtime │ │ (proxied │
│ Postgres + RLS │ │ API │ │ via Edge │
│ Edge Functions: │ │ (WebRTC) │ │ Function) │
│ realtime-token │ │ │ └──────────────┘
│ pexels-search │ │ gpt- │
│ extract-facts │ │ realtime-2│
└──────────────────┘ └───────────┘
Realtime session lifecycle:
- Mint ephemeral key via
realtime-tokenEdge Function. - Open WebRTC peer connection to
api.openai.com/v1/realtime/calls?model=gpt-realtime-2. - On
session.created, pushsession.updatewith instructions + tools + turn-detection config. - Audio + events stream both ways over an SCTP data channel for the rest of the session.
endSession()cancels observation, disconnects WebRTC, runsextract-factsin the background, tears down the Live Activity.
greeting → nameInput → askingQuestions → buildingPlan → (ContentView → HomeView)
Entry: first sign-in. OnboardingCoordinator.check() finds zero facts → .needsOnboarding → ContentView mounts OnboardingView.
Mode: SessionMode.onboarding(state:) — facts written live as Chomp learns them.
Tools: add_fact, display_options, begin_planning, add_meal, finish_onboarding.
Kickoff: yes — Chomp speaks first ("Hi! Nice to meet you — what should I call you?").
USER OnboardingView ConversationVM OpenAI Supabase
│ │ │ │ │
│ signs in │ │ │ │
│ │ create VM ─────────────▶│ │ │
│ │ │ mint token ──────────────────────────▶ realtime-token
│ │ │◀──────────────────────────────────── │
│ │ │ WebRTC handshake ─▶ │
│ │ │◀── session.created │ │
│ │ │ session.update │ │
│ │ │ (onboarding prompt + tools) ─────▶ │
│ │ │ response.create (kickoff) ────────▶ │
│ │ │◀── "Hi! Nice to meet you…" │
│ hears greeting │ chompHasSpoken = true │ │ │
│ │ phase: nameInput │ │ │
│ types name │ │ │ │
│ ────────────────▶│ submitName ───────────▶ │ updateDisplayName ──────────────────▶│ auth.users
│ │ │ send(text:"Please call me X") ───▶ │
│ │ │◀── ack + Q1 audio + display_options │
│ │ │ dispatch display_options │
│ │ phase: askingQuestions │ (cards appear in UI + Live Activity)│
│ │ option cards animate │ │ │
│ │ │ │ │
│ answers (voice) │ │ ── audio ─────────▶│ │
│ │ │◀── add_fact(category, value) │
│ │ │ dispatch ─────────────────────────── │ INSERT facts
│ │ fact card appears │ │
│ │ (cards loop ~6×) │ │
│ │ │◀── begin_planning │
│ │ isPlanningStarted=true │ (UI flag flip; no DB write) │
│ │ phase: buildingPlan │ │
│ │ meal-plan animation │ │
│ │ │◀── add_meal × 3-5 │
│ │ │ dispatch ──────────────────────────▶ │ INSERT meals
│ │ meals appear in stack │ │
│ │ │◀── finish_onboarding │
│ │ isComplete = true │ │
│ │ finishUp(): wait for │ ── farewell audio ◀──────────────────│
│ │ ~2s silence │ │
│ │ endSession ────────────▶│ disconnect WebRTC │
│ │ │ extract-facts (bg) ────────────────▶ │ facts merge
│ │ coordinator.markComplete│ │
│ sees meal plan │ (ContentView swaps) │ │
Entry: user taps "Plan with Chomp" on MealPlanView.
Mode: SessionMode.mealPlan — facts + upcoming meals pre-baked into the prompt so the model doesn't need to call get_meals for routine context.
Tools: add_meal, update_meal, delete_meal, mark_meal_made, suggest_meals, get_meals.
Kickoff: no — user speaks first. Planning is user-driven.
USER MealPlanView ConversationVM OpenAI Supabase
│ │ │ │ │
│ taps Plan │ │ │ │
│ ────────────────▶│ startSession ─────────▶ │ loadActiveFacts ─────────────────────▶ facts
│ │ │ mealPlan.load() ─────────────────────▶ meals (today+)
│ │ │ mint token, WebRTC, session.update │
│ │ │ (plan prompt embeds facts + meals) │
│ │ PlanningSessionBar │ │
│ │ appears │ │
│ "show me three │ │ ── audio ─────────▶ │
│ weeknight │ │ │ │
│ dinners" │ │◀── suggest_meals([3]) │
│ │ │ dispatch: │
│ │ carousel shows 3 cards │ ├─ mealPlan.setSuggestions(...) │
│ │ Live Activity chips │ └─ liveActivityShowChips(...) │
│ │ on lock screen │ │
│ "yes, pad thai │ │ ── audio ─────────▶ │
│ tomorrow" │ │ │
│ │ │◀── add_meal(...) │
│ │ │ dispatch ──────────────────────────▶ │ INSERT meals
│ │ new meal card on plan │ ├─ suggestions cleared │
│ │ Live Activity chips │ └─ liveActivityShowChips(empty) │
│ │ clear │ │
│ taps End │ │ │
│ ────────────────▶│ endSession ───────────▶ │ disconnect │
│ │ │ extract-facts (bg) ────────────────▶ │ facts merge
│ │ reload meals │ │
│ back on plan │ │ │
Drag-and-drop on MealPlanView (re-scheduling meals) and the servings stepper on MealDetailView both bypass the realtime session — they call MealPlanService directly. The session is only for voice-driven editing.
Entry: user taps "Cook Now" on a planned meal card.
Mode: SessionMode.cookRecipe(meal:) — full recipe + user's facts pre-baked into the prompt.
Tools: show_reference, start_timer, cancel_timer.
Kickoff: yes — Chomp opens by naming the recipe and going straight into the first step.
Turn detection: semanticVad(eagerness: .low, interruptResponse: false) — kitchens are loud, Chomp can't be cut off by clatter.
Any active planning session is torn down before the cook cover presents (no two realtime sessions at once).
USER MealPlanView ConversationVM OpenAI Pexels AlarmKit Supabase
│ │ │ │ │ │ │
│ "Cook Now" │ │ │ │ │ │
│───────────▶│ endSession() │ │ │ │ │
│ │ (plan, if │ │ │ │ │
│ │ running) │ │ │ │ │
│ │ cookingMeal= │ │ │ │ │
│ │ meal │ │ │ │ │
│ │ fullScreen │ │ │ │ │
│ │ Cover ─────▶│ loadActiveFacts ────────────────────────────────────▶│ facts
│ │ │ mint token, WebRTC, session.update │
│ │ │ (cook prompt embeds recipe + facts) │
│ │ │ response.create (kickoff) ─▶ │
│ │ │◀── "Pad Thai. Grab your wok…" │
│ hears step │ │ │ │ │ │
│ "what does │ │ ── audio ─────────▶ │ │ │
│ it look │ │ │ │ │ │
│ like?" │ │◀── show_reference("caramelized onions") │
│ │ │ dispatch: │
│ │ │ ImageReference.search ───▶ pexels-search (Edge Fn) │
│ │ │ │ ────▶ Pexels API │
│ │ │◀────────────── imageURL ─────│ │
│ │ │ injectAssistantImage: │
│ │ image bubble │ ├─ chat transcript │
│ │ in chat │ └─ Live Activity (cache to App Group container) │
│ │ │ │
│ "set 10min │ │ ── audio ─────────▶ │
│ timer" │ │ │
│ │ │◀── start_timer(seconds:600, label:"Pasta") │
│ │ │ AlarmService.scheduleTimer ─────────────▶│ │
│ │ timer LA on │ (CookingTimerLiveActivity widget) │ │
│ │ lock screen │ │
│ taps │ │ │
│ "All Done" │ │ │
│───────────▶│ endSession ─▶│ disconnect │
│ │ │ extract-facts (bg) ─────────────────────────────────▶│ facts merge
│ │ phase=.rating│ │
│ │ MealRatingView │
│ rates 5⭐ │ │ │
│───────────▶│ markMade ───────────────────────────────────────────────────────────▶│ UPDATE meals
│ │ cover dismiss │
│ back on │ │
│ plan │ │
The CookingTimerLiveActivity is independent of the session Live Activity — AlarmKit owns its lifecycle, so the countdown survives endSession() and keeps ticking on the lock screen / Dynamic Island until it fires.
Runs after every session regardless of mode. Each conversation is a chance to learn more about the user's tastes; we don't restrict extraction to onboarding.
endSession()
│
├─ POST /functions/v1/extract-facts ─────▶ Edge Function (Deno)
│ │
│ ├─ load existing active facts (1 query)
│ ├─ ask gpt-4o-mini to extract + revoke
│ ├─ in-memory merge plan:
│ │ • exact match → reinforce confidence
│ │ • new value → insert
│ │ • single-value → supersede + insert
│ │ • deactivate_ids → mark inactive
│ └─ apply via Promise.all ─▶ Postgres facts
├─ disconnect WebRTC peer connection
├─ liveActivity.end()
└─ clear cached reference photo from App Group container
One LiveActivityManager per realtime session; local-only (no APNs push). Updates fire from the observation loop AND from individual tool dispatchers.
| Source | What pushes |
|---|---|
| Observation loop (every tick) | status — connecting / listening / speaking / muted / ended |
suggest_meals (planning) |
chips = suggestion titles, chipsTitle = "Suggestions" |
display_options (onboarding) |
chips = option labels, chipsTitle = question prompt |
add_meal / add_fact |
clears chips once the user has answered |
show_reference (cooking) |
imageFilename = path in App Group container |
endSession() |
status = .ended, then immediate dismissal |
Reference photos download to the App Group container (group.com.fasai.weat) so the widget extension can render them via local file I/O. (AsyncImage is unreliable inside a widget snapshot.)
Claude Opus 4.6 with CLAUDE.md.
npx skills add dpearson2699/swift-ios-skills \
--skill activitykit \
--skill widgetkit \
--skill alarmkit \
--skill authentication \
--skill permissionkit \
--skill swiftui-patterns \
--skill swift-concurrency
npx skills add supabase/agent-skills
- Less Coding, More Discussion: Don't rush to coding - always confirm your understanding, think critically about the suggested approach, ask questions, and make suggestions on how to improve the design.
- Web Search Documentation understanding: Read through the documentation here and reiterate your understanding. Write down step by step what you plan to do and what information you need from me.
- Delegate To Me: Don't try to read, search, or execute on everything yourself. If it's easier for me to do it, please give me instructions and I can acquire the information or perform the action for you.
- Break Down Tasks & Test: That's too big of a task. Break it down more so we can test in between.
- Generate Documentation: Create documentation for ____.
- For a really big project, I ask Claude to summarize and save architecture details not apparent in code to memory. I start a new session when I switch to a new area of the codebase. For this prototype, I only used a single session with two /compact runs.
- I always use direct file and folder references (@) so Claude knows exactly where I expect it to read / write.
- I use /btw to explore alternative options a lot.
- I help it course correct the larger design instead of going down a rabbit hole. If it's struggling to debug something, it's likely that the APIs / libraries were misused.
- Polish conversation prompts manually for more natural conversation
- Conversation Reliability & Latency Hillcimb
- Set up Remote Notification Server to push Live Activities to notify the user to order groceries, to prepare for cooking time, or when new meals have been generated and planned.
- Automatic meal planning based on cooking patterns.
- Integration with Instacart / Ubereats / etc. for Grocery Ordering