Meloie — a realtime RVC voice changer (v2)

Turn the voice you speak into a trained target voice (RVC) in real time and send it to Discord / OBS / a recorder. There is exactly one route:

系统默认麦克风 (Windows default recording device)
  -> Python pipeline (StreamingRvcEngine: persistent buffer + F0 continuity + RVC inference)
  -> CABLE Input   (the app renders here)
  -> CABLE Output  (Discord / OBS / recorder select this as their "microphone")

This is a v2-only build: it runs v2-series (768-dim) RVC models such as the default model A (models/A.pth + models/V2.index). A v1 / 256-dim model is rejected at load with a clear message. The single realtime engine is the direct Applio persistent-buffer engine, which runs in .venv-applio.

Meloie is a real-time RVC voice changer (real-time Retrieval-based Voice Conversion) for Windows, built to run a trained RVC model faithfully. It reuses the environment and inference-core lineage of Applio as a starting point, then improves on the realtime RVC ecosystem in three deliberate ways — a distribution-level (CDF / quantile) F0 mapping for accurate male→female pitch conversion (not found in the other realtime RVC changers benchmarked), a structurally faithful output (the runtime never reshapes the model's voice), and an identity-first realtime engine (inference off the audio thread, with auto-fallback to your own voice on any failure). See How Meloie compares for the code-level breakdown.

The trained model is the voice. A model bundle is a .pth + its .index, plus the inference parameters it was trained against (in the model profile). The runtime's whole job is to play that bundle through the realtime audio chain faithfully, stably, and safely.

The contract distinguishes output from input:

• OUTPUT stays faithful. Between the model's samples and CABLE Input there is no EQ, no limiter, no normalize, no gain shaping, no time-stretch, and no output pitch-shift — only a structural resample (model SR → stream SR), a sample-accurate slice, and a NaN/Inf scrub. The one sanctioned blend is a short sin² seam crossfade at block boundaries (plus SOLA alignment): a seam-only join of two renders of the same audio — it never changes pitch or timbre.
• INPUT (the carrier) may be conditioned before conversion. This does not vary the model's voice, only what speech it faithfully converts:
  • pitch_shift (变调) — transposes the input F0 (a female model driven by a male voice typically needs about +12). Model A defaults to +12 (set in its profile, config/model_profiles/A.json).
  • real-audio context warm-up + look-ahead for continuity, and
  • optional input noise reduction (--direct-denoise) so ambient noise is not converted into warbly voice. Default off, so a clean mic / soft speech is never silently degraded.

Voice-identity parameters (f0_method, index_rate, protect, pitch_shift) live in the model profile — they are properties of the trained model.

The simplest path — just double-click run_A_direct.bat (it bakes --direct-f0 fcpe for model A and asks at launch whether to enable input denoise, default off). Or, by hand:

# 1. dot-source the env script (caches/temp -> RVC\, UTF-8 console, .venv-applio on)
. .\setup_env_applio.ps1

# 2. see your devices — the system default mic is marked "I", outputs "O"
python -m meloie.main --list-devices

# 3. (optional) confirm the cable carries audio: tone -> CABLE Input -> CABLE Output
python -m tools.verify_cable_route --duration-seconds 2

# 4. run: system default mic -> model A (v2) -> CABLE Input
python -m meloie.main `
    --config config/runtime.example.json `
    --model-profile config/model_profiles/A.json `
    --device cuda --direct-f0 fcpe --direct-denoise

In Discord / OBS / your recorder, select CABLE Output (VB-Audio Virtual Cable) as the microphone. Speak — you are heard as the model's voice.

Ctrl+C stops the run; a final metrics summary is printed.

By default the runtime captures the Windows default recording device (your "系统默认 mic") — change it in Windows sound settings and the runtime follows. To pin a specific mic instead, set input_device_substring in the config or pass --input-device "Realtek" (a name fragment). The runtime refuses CABLE Output as input (that would feed the cable back into itself).

• The app renders to CABLE Input (the virtual cable's render side).
• Downstream apps select CABLE Output as their microphone.
• The app's input is a physical microphone — CABLE Output is refused as input unless the diagnostic --allow-virtual-cable-input flag is set.

Voice identity is a JSON file under config/model_profiles/ (A.json):

{
  "name": "A",
  "model_path":  "models/A.pth",
  "index_path":  "models/V2.index",
  "f0_method": "rmvpe", "index_rate": 0.08, "protect": 0.33,
  "pitch_shift": 12, "target_f0_median": 200, "formant_timbre": 1.19
}

(See A.json for the live values — the file is the source of truth.) Paths are relative to the project root. Place model assets under models/ (gitignored); the F0 predictors and the contentvec embedder live in models/predictors/ and models/embedders/. The index is loaded only when index_rate > 0 — A uses a light 0.08 blend; raise it for a stronger timbre lock, lower/zero it if it averages the identity away. pitch_shift=12 is A's default (a high/female-range model needs the carrier F0 lifted into its trained range); override per run with --pitch SEMITONES (it conditions the model's input pitch — not an output pitch-shift).

In the GUI, the model dropdown lists the .pth files in models/. Tune the carrier knobs live, then click 💾 记住当前 to save the current knobs as that model's defaults (written to this profile) so it loads that way next time — set a model's pitch once and it sticks.

Blend two or more same-architecture v2 models you own into a new "in-between" voice — an offline, contract-safe way to sculpt voices without training anything. The result is a normal .pth the engine loads as usual.

In the GUI (run_gui.bat): the model dropdown lists the .pth files in your models/ folder by name. Open the Creative card's 🧬 融合模式, check the models to blend into the current (base) model, set each weight, and click 融合并 加载 — the merged voice is saved to models/, appears in the dropdown, and is auto-selected. Or by hand with the CLI:

. .\setup_env_applio.ps1
python -m tools.merge_models `
    --models models/A.pth models/C.pth --weights 0.6 0.4 `
    --output models/AC_mix.pth --name "A+C" --write-profile --profile-pitch-shift 12

• Only models with the same sampling rate / architecture merge (A + C are both 40 kHz; mixing a 40 kHz with a 48 kHz model is refused with a clear message).
• --weights are normalized to sum 1 (default: equal). --write-profile drops a config/model_profiles/<name>.json so the GUI lists it; it sets index_rate 0 (a merged voice has no shared index) — remember to tune pitch_shift (10–14 for high/female voices) or it may sound neutral / 电音.
• The output must live under RVC/ (no C: writes). Add --verify-load to confirm the merged model passes the v2 guard before you use it.

Faithful-carrier still holds: the merged model defines the voice; the runtime plays it without reshaping.

Latency budget ≈ direct-block-ms (accumulation) + prebuffer-ms (standing output) + per-block inference (~30–45 ms on an RTX 4080) + 40 ms device. There is also a one-time startup warm-up (direct-context-ms) while the engine's context buffer fills (it emits silence, not glitches). Lower --direct-context-ms to shorten that startup; none of these touch the voice — see the design stance above.

The direct engine owns streaming state the old per-chunk path never had — a persistent 16 kHz buffer plus F0 caches — so the three structural quirks of chunked RVC are handled faithfully (distilled from a study of mature RVC projects and the Applio realtime core — see docs/realtime_study_notes.md):

• F0 continuity. F0 is recomputed only on the newest trailing window and shifted into a persistent cache, not re-estimated from scratch per block — the main perceptual lever (this is what fcpe smooths).
• Real context, not a mirror. Every block is conditioned on real past audio (--direct-context-ms); the generator decodes only the new region.
• Seam phase match ("电音"). Two renders meeting at a hard cut differ in phase → a comb-filter artifact. SOLA cross-correlates each block's seam against the previously emitted tail and chooses the phase-matched cut offset, then a short sin² crossfade joins them — the one sanctioned seam blend.

On any engine error (CUDA OOM, NaN, model fault) the worker emits the block's own audio (the user's own voice) for that block instead of silence or a crash. The link stays alive; rvc_fallback_count makes it observable.

• python -m tools.offline_infer --input-wav test.wav --output-wav out.wav --model-profile config/model_profiles/A.json --device cuda --f0-method fcpe — proves the model + inference are healthy on a file (no audio devices); it runs the same direct engine, so an offline render predicts the live result.
• python -m tools.verify_cable_route --duration-seconds 2 — proves the VB-CABLE transport (CABLE Input → CABLE Output) carries audio.

If offline inference works and the cable carries a tone, but downstream is silent, the problem is device routing (wrong/silent input device, or the downstream app not listening on CABLE Output).

A per-second line and a final summary report: frames in/out, queue depths and drops, output underruns (split startup vs steady-state), input/output peak/RMS dBFS, NaN scrubs, blocks processed, inference last/mean/max ms, fallback count, stale-block drops, SOLA offset.

. .\setup_env_applio.ps1
python -m pytest -q

Pure tests only — no audio hardware, no GPU, no internet, no model files. The RVC stack is faked via duck-typed engines so the suite runs anywhere with numpy + pytest (the GUI-backend tests additionally need PySide6 and skip themselves cleanly without it). The runtime itself needs .venv-applio.

requirements.txt lists only numpy, sounddevice, pytest. The v2 inference core is first-party code under meloie/core/ (internalized from Applio, MIT — see meloie/core/NOTICE.md); its heavy dependencies (torch+CUDA, torchaudio, transformers, faiss-cpu, torchfcpe, noisereduce, librosa, scipy) are installed once into .venv-applio (the runtime venv). All caches/temp are redirected into RVC\.cache and RVC\.tmp by setup_env_applio.ps1 — nothing is written to the C: drive.

Meloie began as a faithful-RVC runtime on top of Applio's inference core, and was shaped by a code-level cross-audit of the major open-source voice-conversion projects (state as of 2026-06): Applio, w-okada/voice-changer, codename-rvc-fork, seed-vc, RT-VC, and the BigVGAN vocoder. It keeps what they get right — the SOLA + sin² seam-crossfade chunk-streaming lineage — and tightens three things they don't:

1. Distribution-level F0 mapping (the headline feature). Every other RVC changer maps pitch with a single scalar transpose (f0 *= 2**(key/12)), at best plus a median-center or autotune-to-scale. A scalar shift moves the median but cannot fix the spread and shape of the pitch distribution — a core reason male→female conversion sounds off. Meloie instead builds the empirical CDF of your own voice and of the target model's voice, then quantile-maps your whole F0 distribution onto the target's in the log2(Hz) domain (meloie/engine/f0_map.py). Across every project audited, none implement distribution / quantile F0 matching — the closest (Applio, seed-vc, RT-VC) only align the median.
2. A structurally faithful output. Realtime RVC changers routinely reshape the output: w-okada multiplies the model's audio by the input's loudness (* sqrt(vol)); Applio scales the output by the input RMS and applies change_rms / peak-normalize. Meloie forbids all output-side shaping at the API level — no change_rms, no normalize, no EQ / limiter / denoise / FX (volume_envelope=1, reduced_noise=None, board=None). The model defines the voice; the runtime only carries it. Input-side conditioning (transpose, the CDF map, optional denoise) is allowed because it changes what speech is converted, not the model's voice.
3. An identity-first realtime engine. Several changers run heavy GPU inference inside the audio callback (w-okada, seed-vc) and emit silence — or tear down the stream — when inference fails. Meloie runs inference on a worker off the audio thread (mic → in-queue → worker → out-queue → output) and, on any failure (CUDA OOM, NaN, model fault), falls back to passing through your own voice for that block. The link never goes silent or crashes.

Credit where due — what the others do better. Meloie is deliberately narrow (quality-first, latency-tolerant, single-model, v2-only, realtime). It does not try to match Applio and the RVC toolkits' breadth (training, UVR separation, ONNX, multi-backend F0, v1+v2); w-okada's model-merging and many SVC backends; seed-vc and RT-VC's zero-shot cloning (no per-voice model needed); RT-VC's ultra-low latency; or BigVGAN's vocoder fidelity. For its focused niche — a faithful realtime RVC changer with accurate male→female pitch — it aims to be the most correct option, not the most feature-rich.

_{Comparison based on a code-level reading of each project's then-current source (2026-06); "RVC" = Retrieval-based Voice Conversion. Corrections welcome via an issue.}

• Faithful. The model defines the voice; the runtime is a clean carrier of its output. Input conditioning (transpose, optional denoise) is allowed; output reshaping is not.
• v2-only. This build runs v2-series models; a v1 model is rejected at load.
• Stable first. Quality-first defaults trade latency for continuity; the link should run without glitching, draining, or drifting.
• Identity-first safety. Inference failure falls back to the user's own voice, never silence or a crash.
• No system mutation. Never changes the Windows default device, app settings, registry, PATH, or drivers.

## License

First-party Meloie code is released under the **MIT License** (see `LICENSE`).
The inference core under `meloie/core/` is derived from third-party projects
under their own permissive licenses — **Applio** (MIT), **HiFi-GAN** (MIT),
**ContentVec** (MIT) and **RMVPE** (Apache-2.0); full attribution is in
`meloie/core/NOTICE.md`. Trained model weights (`*.pth` / `*.index` / `*.pt`)
are **not** part of this repository and are not covered by these licenses.