An experimental browser-based voice interface that explores speech recognition, AI reasoning, voice synthesis, and music playback using modern web technologies: https://voez.ai

Voez AI is an experimental project that explores how voice can become a primary interaction method for music playback within a web browser.

Rather than focusing on traditional menus, search boxes, and graphical controls, the application allows users to interact through natural speech to request songs, receive recommendations, and control playback.

The project was built to investigate the capabilities and limitations of browser-based voice experiences by combining speech recognition, AI processing, voice synthesis, and audio playback into a single conversational workflow.

Voez AI is not intended to replace platform-level voice assistants such as Siri or Alexa. Instead, it serves as a technical exploration of what is possible when modern browser APIs are used to create voice-first interactions on the web.

• Request songs using natural voice prompts
• Receive spoken responses generated by AI
• Discover music through conversational recommendations
• Interact with music playback without relying heavily on traditional UI controls
• Experience a voice-driven workflow directly within the browser

This project served as a hands-on exploration of:

• Browser-based speech recognition and synthesis
• Speech-to-text (STT) and text-to-speech (TTS) workflows
• AI-assisted music recommendations
• Event-driven audio interactions
• Frontend architecture and state management
• API orchestration across multiple services
• Graceful fallback and failure handling
• Real-world browser compatibility challenges

User Voice
    ↓
Web Speech API (STT)
    ↓
OpenAI
    ↓
ElevenLabs (TTS)
    ↓
Audio Playback UI

Uses the Web Speech API for speech recognition and a custom ElevenLabs voice for AI-generated spoken responses.

OpenAI analyzes the user's voice prompts and conversational context to infer mood, intent, and listening preferences, enabling personalized song recommendations and more natural music discovery.

Users can request music through natural language and receive spoken feedback while controlling playback through conversation.

Audio playback is coordinated through state-based and event-driven interactions to create a seamless listening experience.

The interface dynamically adjusts throughout the listening journey, minimizing unnecessary visual distractions and keeping the experience focused on audio.

When AI analysis or recommendations fail, the application falls back to predefined behavior to maintain continuity rather than exposing technical errors to users.

Redis-backed rate limiting helps prevent abuse, excessive requests, and accidental spam.

Voez AI is an experimental voice-first application and is currently optimized for the latest version of Google Chrome.

Apple Safari is supported but requires these setups:

• Enable Siri/Dictation on the device.
• Allow microphone access in Safari.
• Grant microphone permission when prompted.
• Refresh the page after changing permissions.

Support for other browsers has not been fully validated, and some features may not function as expected. Please ensure your microphone and speakers are enabled with proper settings for Google Chrome and Apple Safari.

• Next.js
• React
• React Icons
• TypeScript
• Tailwind CSS
• Web Speech API

• Next.js API Routes
• Redis / Upstash Redis
• OpenAI API
• ElevenLabs API

• Vercel

git clone git@github.com:zcdev/voez.git
cd voez

npm install

Create a .env.local file.

Example:

OPENAI_API_KEY=
ELEVENLABS_API_KEY=
UPSTASH_REDIS_REST_URL=
UPSTASH_REDIS_REST_TOKEN=

npm run dev

Open http://localhost:3000

Before deployment:

npm run verify

This workflow helped catch deployment issues early by combining linting, type checking, and build verification.

The project is configured for deployment on Vercel.

npm run build

Production deployment includes:

• Environment variable validation
• Redis-backed rate limit protection
• Build verification checks
• Responsive preview support

Key lessons from development included:

• Designing conversational flows that remain intuitive across multiple user paths
• Building reliable fallback behavior for AI-driven experiences
• Managing asynchronous workflows across multiple APIs
• Coordinating speech recognition, speech synthesis, and audio playback
• Keeping the interface simple while handling complex state transitions

Potential future enhancements include:

• Expanded conversational controls
• Playlist and queue management
• Personalized recommendations
• User accounts and listening history
• Additional voice customization options
• Improved browser compatibility
• Accessibility-focused voice interactions

This project demonstrates:

• Production-focused frontend engineering
• Multi-API integration
• Voice-first UX experimentation
• Type-safe development practices
• Event-driven application design
• Resilient error and fallback handling

MIT