Skip to content

Tune: mark each cycle by waveform match to remove grit in tuned voices - #45

Open
mparrett wants to merge 2 commits into
nooga:masterfrom
mparrett:tune-marks
Open

mparrett wants to merge 2 commits into
nooga:masterfrom
mparrett:tune-marks

Conversation

@mparrett

@mparrett mparrett commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Tune leaves speech with a gritty, bad-microphone edge. On a spectrogram, the harmonics above about 1.5 kHz break into noise that the take doesn't have. On six monotone TTS takes, Tune at SPEED 0 lowered the median harmonics-to-noise ratio by 1.2 to 2.6 dB compared with the source.

Listen. One line of a monotone TTS take (Gemini, Aoede voice), three times: the source, Tune on master, and Tune with this PR, all at SPEED 0, chromatic.

pr45-tune-grit.mp4
Spectrograms of source, Tune on master and Tune with this PR, 2.0-3.0 s of the same line

The still zooms to one second of each, where master's upper harmonics smear into hash and this PR's run as clean lines like the source's. Its HNR figures are over the 7 s excerpt, so they differ from the whole-take table below.

How these were rendered

The first 7.1 s of aoede-cli-directed.wav (24 kHz mono) on a sampler track, audio(…, tune="A", scale="chromatic", speed=0, humanize=0) at 120 BPM, saved by slabkit and rendered with slab --render from a ReleaseFast build of each commit: master at 472e5b4, this PR at 74030d2. The source row is the same clip with no tune. The video is the three renders joined by 0.6 s gaps.

Cause. Each pitch mark went on the highest peak within a quarter period of where the last one predicted. A voice's crest often carries near-equal ripples from a formant, and the highest one moves from cycle to cycle. A grain centred a few percent of a period off has its upper harmonics out of phase with its neighbours, so they no longer add up.

Change (tune.zig, worker side only):

  • Each mark after a voiced run's first now goes where the waveform best matches the period just marked (normalized cross-correlation).
  • It's refined between samples by a parabola through the match scores.
  • It carries the previous mark's fraction forward. Without that, a 218.3-sample period rounded to 218 every step and put a tuned tone 2 cents flat.
  • The search tries every fourth candidate, then each one around the three best. A strong formant adds correlation lobes a few samples apart, and a coarse step can land beside the true one.

Grains are still spaced by the frame's pitch. I tried spacing them by the marks, but that let the voice's own vibrato through SPEED 0.

Results (median harmonics-to-noise ratio, dB):

take source Tune before Tune after
cli-directed 14.8 12.4 15.1
cli-world-flat 18.1 15.6 18.1
agents-directed 15.6 13.1 15.9
agents-world-flat 16.9 15.7 17.7
face-directed 15.7 13.1 15.6
face-world-flat 18.0 16.4 18.5

These are headless renders (slabkit) at SPEED 0, chromatic. The upper harmonics are clean lines up to about 4.5 kHz again, and by ear the grit is gone. cli- and face-directed were re-measured on the PR head.

Tuning itself is unchanged, measured old against new on synthetic sung material:

  • A tone 40 cents flat comes out at 0.00 cents before and −0.01 after, at the same level, with the same noise between harmonics (−34.2 vs −34.1 dB).
  • On a vibrato line, SPEED 0 leaves the same ±0.4–0.6 cents.
  • SPEED 120 with HUMAN 60% matches note for note.

Cost. Marks for a 3-minute take take about 390 ms on top of the 1.15 s pitch track, up from about 10 ms. This runs once per source on the worker and is cached.

Tests. zig build test passes (589 of 590, 1 skipped). Two new marks tests:

  • Near-equal ripples: a 218.3-sample period whose crest carries swaying ripples that are locked to the pulse. Every mark must stay within a sample of the first one's phase, and the average spacing must match the period. It fails on the old peak marks (they jump by 3 samples) and on marks without the carried fraction (they drift).
  • Bright vowel: F2 stronger than the fundamental. Marks must not slip a lobe.

docs/30 §Tune and the CHANGELOG are updated.

Limitation. On an extreme spectrum (a weak fundamental under a dominant formant near 4.4 kHz), the pitch tracker itself reads about 5% sharp. The first prediction is then wrong, and the marks settle one lobe off. That's the pitch tracker's limitation rather than the marks', and it's out of scope here.

Independent of #43 (FORM), which builds on the same grains. Merged together, FORM +5 gains 2.3–2.6 dB too.

🤖 Generated with Claude Code

mparrett and others added 2 commits November 7, 2026 11:46
Tune left speech gritty above ~1.5 kHz: harmonics-to-noise 1.2-2.6 dB
under the take. Marks on the highest peak jump between a voice's
near-equal ripples on the crest, and a grain a few percent of a period
off puts the upper harmonics out of phase with its neighbours.

Each mark after a run's first now goes where the waveform best matches
the period just marked (normalized cross-correlation, every fourth
candidate then each one around the best, a parabola through the
scores), carrying the previous mark's fraction so the chain doesn't
drift. Grains are still spaced by the frame's pitch: spacing them by the
marks let the voice's own vibrato through SPEED 0.

On six monotone TTS takes the tuned voice is at or above the source's
harmonics-to-noise. A 40-cent-flat tone and a vibrato line tune exactly
as before (0.0 cents; ±0.4-0.6 cents of vibrato left at SPEED 0).
Marking a 3-minute take costs ~290 ms on top of the ~1.2 s pitch track
(was ~10 ms).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nd no lobe slips

Review fixes. A strong formant gives the match lobes a few samples
apart, and stepping by four can land beside the true one; refining
around the three best coarse candidates finds it (390 ms to mark a
3-minute take, from 290). A new test on a bright vowel (F2 over the
fundamental) holds every mark within a sample of the first one's phase,
and the near-equal-ripple test now does too: chained matches don't
wander off the pulse. The module comment no longer says marks sit on
peaks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant