Daydream

Daydream is a local-first desktop recall assistant for answering one question: what did I do today?

It records screen, audio, activity, and window context on hardware you control, then turns that evidence into a timeline you can review and process locally.

• Multi-monitor capture through the Linux desktop portal and PipeWire.
• A scene editor for arranging selected displays on one recording canvas.
• Separate desktop and microphone audio streams in local MKV segments.
• Explicit start, pause, resume, and stop controls.
• Local Library playback with thumbnails, waveforms, track lanes, and source timestamps.
• Timeline redactions that exclude or invalidate overlapping derived results.
• Protected-window rules that can pause capture or create automatic redactions.
• Mouse, keyboard, frame-change, and window activity lanes without recording typed text or pointer coordinates.
• Resumable processing jobs for transcription, OCR, visual observations, and activity analysis.
• Local transcription through vendored whisper.cpp.
• Tesseract OCR using system or managed language data.
• Optional OpenAI-compatible remote transcription and vision endpoints.
• Rust-owned SQLite storage with embedded migrations.

The demonstration below was recorded by Daydream itself. The animation is silent; the original desktop and microphone tracks were not added to this repository.

Daydream handles screen recordings, audio, OCR, transcripts, window titles, activity patterns, and generated observations. The default architecture keeps that data on the local machine.

• Captured content is not uploaded by default.
• Remote AI providers require explicit configuration.
• API keys are stored through the backend rather than exposed to React.
• Raw frames, audio, OCR text, transcripts, prompts, and window titles are not written to normal logs.
• Redacted ranges are excluded from later processing and hidden or invalidated in derived results.
• Input activity records timing and state, not key values, typed text, pointer coordinates, or clicked content.
• Recording state stays visible and user-controlled.

The codebase treats deletion and redaction as data-model concerns, not visual filters. Work is still ongoing to make physical media deletion and every downstream cleanup path equally complete.

• Development and real capture testing currently focus on Linux, especially KDE Plasma on Wayland. Windows and macOS capture adapters are not implemented.
• The searchable day index, semantic search, daily digest, task extraction, and cited question answering are planned work.
• There is no stable installer, automatic updater, or supported release channel.
• Some Linux capabilities depend on desktop portal, PipeWire, GStreamer, Tesseract, input-device permissions, and compositor support.
• Full GPU composition and hardware encoding remain future work.
• Optional remote model endpoints can receive private content when the user enables and configures them.

Daydream is a Yarn 4 monorepo:

• apps/desktop: Tauri 2, React, Vite, Tailwind, shadcn components, TanStack Router, and Zustand.
• apps/desktop/src-tauri: Rust capture, playback, processing, settings, diagnostics, and SQLite ownership.
• packages/shared: TypeScript contracts shared across the desktop boundary.
• packages/ffmpeg: development FFmpeg resolution through ffmpeg-static.
• packages/eslint-config and packages/tsconfig: shared project tooling.

The Linux recorder uses xdg-desktop-portal and PipeWire for screen sources, GStreamer for segmented media writing, and separate desktop and microphone audio tracks. Processing runs after capture in deterministic, resumable chunks.

capture
  -> local MKV segments and activity metadata
  -> redaction-aware chunk planning
  -> transcription, OCR, activity, and optional vision processing
  -> SQLite-backed timeline results
  -> local playback and review

React does not access SQLite, native capture APIs, media writers, or model runtimes directly. Typed Tauri commands own those boundaries.

• Node.js 22 or newer
• Corepack
• Rust and Cargo
• Linux Tauri development libraries for WebKitGTK 4.1
• PipeWire and xdg-desktop-portal
• GStreamer development headers and the base, good, bad, ugly, and PulseAudio plugin sets
• CMake, Clang, and libclang for the vendored whisper.cpp build
• Tesseract for local OCR

On Debian or Ubuntu, the native development packages include:

sudo apt install \
  libwebkit2gtk-4.1-dev \
  build-essential \
  curl \
  wget \
  file \
  libxdo-dev \
  libssl-dev \
  libayatana-appindicator3-dev \
  librsvg2-dev \
  libpipewire-0.3-dev \
  libpulse-dev \
  libgstreamer1.0-dev \
  libgstreamer-plugins-base1.0-dev \
  libgstreamer-plugins-bad1.0-dev \
  gstreamer1.0-plugins-base \
  gstreamer1.0-plugins-good \
  gstreamer1.0-plugins-bad \
  gstreamer1.0-plugins-ugly \
  gstreamer1.0-libav \
  gstreamer1.0-tools \
  gstreamer1.0-pulseaudio \
  cmake \
  clang \
  libclang-dev \
  tesseract-ocr

Package names vary by distribution.

COREPACK_HOME=/tmp/corepack corepack yarn install
COREPACK_HOME=/tmp/corepack corepack yarn dev

On Linux Wayland, the development launcher sets WEBKIT_DISABLE_DMABUF_RENDERER=1 by default to avoid known WebKitGTK DMA-BUF protocol failures.

COREPACK_HOME=/tmp/corepack corepack yarn lint:fix
COREPACK_HOME=/tmp/corepack corepack yarn check
cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml

Some Rust transcription tests bind temporary loopback ports.

Detailed design decisions and implementation history live under docs/milestones:

• Recorder foundation
• AI processing foundation
• Track timeline editor
• Day index MVP plan
• Protected-window recording rules

Read CONTRIBUTING.md before opening a change.

Do not attach real recordings, screenshots, transcripts, OCR output, window titles, or unredacted logs to public issues. Report security problems privately as described in SECURITY.md.

Daydream is licensed under the GNU Affero General Public License v3.0 or later.

Copyright © 2026 Felix Faust.

Third-party code, tools, models, and runtime components retain their own licenses. See THIRD_PARTY_NOTICES.md.