Making the web more accessible, one video at a time.

SignTube is an intelligent accessibility tool that automatically generates sign language animations for YouTube videos in real-time. By combining the power of OpenAI's Whisper for transcription and spaCy for NLP, we translate spoken English into grammatically correct American Sign Language (ASL), displayed via seamless overlays.

• 📺 Real-Time Translation: Instantly translates YouTube video audio into sign language.
• 🧠 Smart ASL Grammar: Not just word-for-word! Uses spaCy to reorder sentences into proper ASL structure (Time → Subject → Verb → Object).
• 🗣️ AI Transcription: Leverages OpenAI Whisper for state-of-the-art speech-to-text accuracy.
• 🔌 Browser Extension: A plug-and-play Chrome extension that integrates directly into the YouTube player.
• 🚀 Modern Web App: A beautiful Next.js frontend to explore and manage the platform.
• 🎭 3D & GIF Support: Uses a rich library of GIFs and 3D avatar fallbacks to ensure every word is signed.

• Framework: Next.js 16 & React 19
• Styling: Tailwind CSS v4
• Animations: Framer Motion & Canvas Confetti
• Extension: Manifest V3 (Chrome/Edge)

• Server: Python Flask
• AI/ML:
  • OpenAI Whisper (Speech-to-Text)
  • spaCy (NLP & Grammar Logic)
• Media Processing: yt-dlp & FFmpeg (Audio extraction)

The backend handles the heavy lifting of downloading audio, transcribing it, and processing the NLP.

cd backend
# Install dependencies
pip install -r requirements.txt
# Download the spacy model
python -m spacy download en_core_web_trf
# Run the server
python app.py

Server runs on http://localhost:5001

The web interface for the project.

cd frontend
# Install dependencies
npm install
# Run the dev server
npm run dev

App runs on http://localhost:3000

1. Open Chrome and go to chrome://extensions/.
2. Enable Developer mode (top right).
3. Click Load unpacked.
4. Select the extension folder in this repository.
5. Open a YouTube video and watch the magic happen! ✨

• /backend: Flask server, Whisper integration, and NLP logic.
• /frontend: Next.js web application.
• /extension: Chrome extension content scripts and manifest.
• /gif: Asset library for sign language visualizations.
• /animation: 3D animation assets.