AI-powered semantic video editing tool that automatically inserts relevant B-roll clips into your main video based on spoken content.

• Automatic Transcription - Transcribes A-roll audio using Groq Whisper
• Automated B-Roll Analysis - Uses Llama 4 Scout (Vision) to auto-describe B-roll clips
• Semantic Embeddings - Generates semantic concept embeddings for matching
• LLM Matching - Uses Llama 3.3 70B for intelligent B-roll selection
• Confidence Scoring - AI-generated confidence scores (0.0-1.0) for each match
• Hybrid Approach - Combines embedding similarity with LLM reasoning
• Timeline Generation - Creates JSON timeline with insertion points
• Video Rendering - FFmpeg-based video composition with B-roll overlays
• Modern UI - Glassmorphism design with Transcript Viewer and Video Preview
• Auto-Regeneration - Ability to regenerate descriptions for existing B-rolls

• Node.js v18+
• MongoDB (local or Atlas)
• FFmpeg (auto-installed via @ffmpeg-installer/ffmpeg)

git clone https://github.com/YOUR_USERNAME/smart-broll-inserter.git
cd smart-broll-inserter

Backend:

npm install

Frontend:

cd frontend
npm install
cd ..

Create a .env file in the root directory:

# Server
PORT=8080

# MongoDB
MONGO_URI=mongodb+srv://<username>:<password>@cluster.mongodb.net/broll-inserter

# Groq API (Required for transcription & matching)
GROQ_API_KEY=gsk_your_groq_api_key_here

# Optional Configuration
SIMILARITY_THRESHOLD=0.45        # Minimum confidence for B-roll matches
BROLL_DURATION_SEC=5.0           # Default B-roll duration in seconds

npm run dev

Backend runs at: http://localhost:8080

cd frontend
npm run dev

Frontend runs at: http://localhost:5173

Use two terminal windows:

# Terminal 1 - Backend
npm run dev

# Terminal 2 - Frontend
cd frontend && npm run dev

• Click "Upload A-Roll" and select your main video file
• Supported formats: MP4, MOV, WAV

• Click "Upload B-Roll" and select a clip
• Add a description (e.g., "busy street food stall with cooking")
• Repeat for all B-roll clips you want in your library

• Click "Run Full Pipeline"
• This will:
  • Transcribe your A-roll
  • Segment the transcript
  • Match B-rolls semantically using AI
  • Generate a timeline

• Select a timeline from the dropdown
• Review matched insertions and confidence scores
• Click "Render Final Video"
• Download the rendered video

Located at: GET /api/pipeline/timelines

Example structure:

{
  "aroll_id": "aroll_4ceaa7ca",
  "aroll_duration_sec": 40.02,
  "insertions": [
    {
      "start_sec": 1.28,
      "duration_sec": 5.0,
      "broll_id": "broll_39099266",
      "confidence": 0.85,
      "reason": "Semantic match: speaker discussing street food..."
    }
  ]
}

• Location: exports/final_XXXXXXXX.mp4
• Auto-deleted after 5 minutes (configurable)
• Download via frontend video player

smart-broll-inserter/
├── frontend/               # React frontend
│   ├── src/
│   │   ├── App.jsx         # Main UI component
│   │   ├── index.css       # Glassmorphism styles
│   │   └── services/       # API service layer
│   └── package.json
├── services/
│   ├── transcription/      # Groq Whisper integration
│   ├── matching/           # LLM-based B-roll matching
│   └── output/             # Video rendering & timeline
├── router/
│   ├── transcribe.route.js # Upload endpoints
│   └── pipeline.route.js   # Pipeline & render endpoints
├── model/                  # MongoDB schemas
├── uploads/                # Uploaded media files
├── exports/                # Rendered videos (temp)
├── index.js                # Express server entry
└── .env                    # Environment variables

1. Fork the repository
2. Create feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open a Pull Request

This project is licensed under the MIT License.

• Groq - Ultra-fast LLM inference
• FFmpeg - Video processing
• MongoDB - Database