KDB-Flow

Auto KDP ebook generator from YouTube videos. Lightweight, bulk, fast.

Status: MVP pipeline online. The current implementation can enqueue a job, run the filesystem-backed stage pipeline, generate stub/mock book artifacts, expose job APIs, and serve a Vite/React dashboard shell. Real production integrations are being added stage by stage.

• Backend: Python 3.12, FastAPI, asyncio, SQLite, uv
• Frontend: Vite + React 19 + TypeScript + Tailwind v4 + shadcn/ui
• Pipeline: youtube-transcript-api, yt-dlp, ffmpeg, OpenAI / Gemini, Flux.1 [pro], Pillow, pandoc, weasyprint

Xem chi tiết: docs/architecture.md.

• Python 3.12+
• Node 20+, Yarn 4 (qua corepack)
• uv (install)
• ffmpeg for frame capture, pandoc for ebook output (system binaries)

# Backend deps
uv sync
cp .env.example .env             # điền API keys

# Terminal 1 — backend (port 8000)
make dev-api

# Terminal 2 — optional queue worker for CLI/API queued jobs
make dev-worker

# Terminal 3 — frontend
make install-fe
make dev-fe                      # vite :5173

Mở http://localhost:5173.

Implemented:

• SQLite-backed job queue and async worker runner.
• CLI commands for run, batch, status, artifact listing, logs, retry/cancel, rerun-from-stage, and a local queue worker.
• FastAPI JSON endpoints for job create/list/detail/retry/cancel.
• FastAPI stats endpoint backed by SQLite queue/cost/stage aggregates.
• YouTube ingest metadata validation with duration checks and report-only license/source-quality warnings by default.
• Filesystem-backed stages: ingest, transcribe, ground, estimate, orchestrate, plan, write, verify, illustrate, format, and package.
• Offline stub pipeline that produces meta.json, transcript.json, outline.json, chapter markdown, book.md, book.html, book.epub, book.pdf, cover_kindle.jpg, manifest.json, kdp_submission.json, and release.zip.
• Optional OpenAI- or Gemini-backed outline/chapter mode when configured.
• LLM usage, disk cache, JSON outline repair, and budget tracking for OpenAI/Gemini planning/writing jobs.
• Disk-backed transcript and LLM response caching to avoid repeated caption provider/API calls for identical inputs.
• Artifact-backed expansion planning with expansion_plan.json so thin caption sources can become story-mode ebooks without bypassing factual gates.
• Queue-only batch submission, resumable duplicate skipping, and stale running job recovery through kdb-flow worker --recover-stale-minutes.
• Baseline KDP readiness packaging with deterministic Kindle cover generation, WeasyPrint PDF export, release bundle inclusion, and readiness warnings for upload preparation.
• React/Vite dashboard MVP for submitting jobs, viewing recent jobs, opening job detail, checking stages, and inspecting artifact metadata.

Still in progress:

• Broader real-world caption source-quality reporting.
• LLM editing/review passes beyond the current planning/writing flow.
• Figure selection, AI cover/background generation, paperback cover output, and richer KDP review workflows.
• Authentication, SSE progress streaming in the UI, and generated frontend OpenAPI types.

src/kdb_flow/        # Python package (CLI + pipeline + FastAPI JSON API)
frontend/            # React SPA (Vite + TS + shadcn)
docs/                # architecture & design docs
tests/               # pytest
scripts/             # ops helpers

• docs/architecture.md — system architecture
• docs/web-ui.md — frontend stack & pages
• docs/cover-generation.md — KDP cover pipeline
• docs/frame-selection-strategy.md — figure selection

Proprietary. All rights reserved. Contact owner for usage.