WhisperSubs is a Jellyfin plugin that generates subtitles for your media library with local AI models. Transcription runs on hardware you control: this Jellyfin server by default, plus any transcription workers you choose to add. Your media never reaches a third-party cloud unless you deliberately configure one.

Everything past this page lives at geiserx.github.io/whisper-subs.

• Self-hosted processing. Audio is transcribed by whisper.cpp on this server by default, or across a pool of your own workers.
• Built-in engine setup. The settings page downloads the whisper-cli binary and a model. Binary downloads are Linux only: on macOS and Windows the variant dropdown is empty by design, and you install whisper-cli yourself and set Whisper Binary Path. Model downloads work on every platform.
• Automatic language detection. Reads each audio stream's language tag, falling back to whisper's own detection when tags are absent. A multi-language file gets one subtitle per audio language.
• Forced subtitles. Transcribe only foreign-language dialogue, via VAD speech segmentation and per-chunk language detection.
• English translation. Optionally add an English subtitle to a title that has none. English is the only language whisper can translate into.
• Lyrics (experimental). .lrc files for music libraries, picked up by Jellyfin automatically.
• Vocal separation (optional). Isolate vocals with BSRoformer.cpp before transcription for noisy content. Falls back to the original audio when unavailable.
• GPU acceleration. CUDA (NVIDIA), Vulkan (Intel / AMD / NVIDIA) and ROCm (AMD).
• Distributed transcription. Pool extra machines, GPUs, a NAS or a hosted endpoint and transcribe in parallel. Off by default, and free local workers are always used before bursting to a paid one.
• Priority queue with user requests. Manual requests outrank the background sweep. Non-admin users can request subtitles from the item menu once you enable it.
• Real-time progress. A live banner shows the current item, the phase, per-file progress and queue depth.
• Subtitle resume. An interrupted transcription continues from its last timestamp instead of starting over.
• Admin dashboard. Browse libraries, see which items have subtitles, and generate them from the Jellyfin admin panel or from an in-page button on the item page.
• Scheduled task with a skip cache. Daily at 2:00 AM and on startup by default. A persistent skip cache means repeat runs do not re-probe unchanged items.
• Configurable output filenames. The label and filename template are yours to change.

1. In Jellyfin, go to Dashboard > Plugins > Repositories.
2. Add a repository with this URL:

   https://geiserx.github.io/whisper-subs/manifest.json
   

3. Go to Catalog, find WhisperSubs, and click Install.
4. Restart Jellyfin.

Then open Dashboard > Plugins > WhisperSubs and follow the setup guide to install the engine and a model.

dotnet build --configuration Release
# copy WhisperSubs.dll to /var/lib/jellyfin/plugins/WhisperSubs/, then restart Jellyfin

Subtitles are written next to the media and picked up by Jellyfin on the metadata refresh that follows. The default filename template is {name}.{lang}.{label}{.type} with the label WhisperSubs, so a Spanish film produces:

The label is both the title shown in Jellyfin's subtitle picker and the marker the plugin uses to recognise its own files, so pick something distinctive if you change it. Files written by older versions with the .generated. anchor are still recognised. Both Subtitle label and Filename template are on the settings page.

The scheduled task skips media that already has a usable subtitle in the needed language. Forced tracks do not satisfy that need while Ignore forced subtitles when skipping is on (the default), and image-based tracks do not unless you turn on Count image-based subtitles as present. They are two independent settings. A manual Generate on a single item always transcribes, bypassing the skip.

Off by default. Turn on Allow user requests in the User Requests panel and non-admin users get a Request Subtitles entry on the item page. Requests land as Pending and cost no CPU until an admin approves them, unless you also turn on Auto-approve. Approved requests enter the queue below admin requests and above the background sweep. Per-user daily quota (5), active cap (3), per-request item cap (200) and a global cap (500) are all configurable, and 0 means unlimited.

Nine models are offered on the settings page. The default is Large V3 Turbo (Q5), 574 MB.

The two turbo models cannot translate. They were fine-tuned without the translate task, so enabling translation with one of them writes the source language into an English-named .translated.srt file. The plugin logs a warning and runs the job anyway; nothing in the filename or the subtitle picker reveals it. Download Large V3 (Q5) or Medium (Q5) before turning translation on. Limitations has the detail.

By default everything runs on this Jellyfin server, one job at a time. Expand Worker Pool (Optional / Advanced) to add OpenAI-compatible endpoints: another box with a GPU, a NAS, or a hosted API. The plugin extracts audio locally and sends it over HTTP, always preferring workers with a cost weight of 0 and bursting to a paid one only when the free ones are saturated. worker/ is a ready-to-run whisper.cpp + Vulkan worker image. Setup, upload-size caps and provider quirks are covered in Remote workers.

Every setting on Dashboard > Plugins > WhisperSubs carries its own inline help. Six tunables are deliberately not on that page and can only be changed in the plugin's XML configuration file, followed by a Jellyfin restart:

Four more fields (WhisperBinaryVariant, VocalSeparationBinaryVariant, VocalSeparationModelQuant, VadModelPath) record what the plugin downloaded or resolved. They are not meant to be edited by hand.

The plugin injects a script tag into Jellyfin's index.html to add a Generate Subtitles entry to the item page. Direct on-disk injection is the default and needs a writable web root, which containers often do not have. If yours is read-only, install the File Transformation plugin from https://www.iamparadox.dev/jellyfin/plugins/manifest.json and WhisperSubs registers a serve-time injection instead, with no permission changes. Both mechanisms coexist safely. The status panel at the top of the settings page shows which one is active, and Setup/InjectionStatus reports the same in JSON.

36 endpoints live under /Plugins/WhisperSubs/. All of them require authentication. Everything except the four user-request endpoints (Requests/Capabilities, Items/{id}/Request, Requests/Mine, Items/{id}/RequestStatus) requires a Jellyfin admin. Diagnostics shows how to call Setup/Status, Queue and Setup/InjectionStatus, which are the three worth capturing when something goes wrong.

See ROADMAP.md for planned features and design details.

• smart-covers: cover extraction for books, audiobooks, comics, magazines and music libraries, with online fallback
• quality-gate: restrict users to specific media versions based on configurable path-based policies
• jellyfin-encoder: automatic 720p HEVC/AV1 transcoding service with hardware acceleration
• jellyfin-telegram-channel-sync: sync Jellyfin access with Telegram channel membership

This project is made possible by generous supporters: yskaa001

This project is licensed under the GNU General Public License v3.0. See the LICENSE file for the full text.