Skip to content

Latest commit

 

History

History

README.md

EffeTune DSP Library and Core

The DSP core is published as the EffeTune DSP Library for Python, JavaScript, and browser AudioWorklets. Start with the DSP Library documentation for the complete guides, effect catalog, API reference, live demo, and details about streaming, assets, and deterministic processing.

The library can process audio independently of the EffeTune app. Python and JavaScript use the same C++20 DSP core and the same semantic Chain JSON format, so a chain can be shared between the two packages. The app can optionally be used as a visual preset editor.

Choose the section that matches your goal:

Python Quick Start

Install the package from PyPI:

pip install effetune

Pass a C-contiguous float32 NumPy array with shape (channels, frames) to a chain. The library does not resample audio.

import numpy as np
import effetune as et

frames = 512
phase = np.arange(frames, dtype=np.float32)
mono = (0.5 * np.sin(2 * np.pi * phase / 97)).astype(np.float32)
audio = np.ascontiguousarray(np.stack((mono, mono)))

chain = et.Chain([et.Volume(volume=-6)])
output = chain.process(audio, sample_rate=48_000)
print(output.shape)

Generated effect constructors use Python snake_case parameter names. Use Chain.stream() instead of Chain.process() when filter history, delay or reverb tails, or seeded random state must continue across blocks. See the Python guide for audio-file processing and the full API.

JavaScript Quick Start

Install the ESM-only package from npm:

npm install @effetune/dsp

Pass one equal-length Float32Array per channel. The package does not decode, encode, or resample audio.

import { createChain } from '@effetune/dsp';

const frames = 512;
const mono = Float32Array.from(
  { length: frames },
  (_, frame) => 0.5 * Math.sin(2 * Math.PI * frame / 97)
);
const input = [mono.slice(), mono.slice()];

const chain = await createChain({
  version: 1,
  chain: [{
    id: 'volume',
    type: 'Volume',
    parameters: { volume: -6 }
  }]
});
const output = await chain.process(input, { sampleRate: 48000 });
console.log(output.length, output[0].length);
chain.close();

Node.js 18 or newer is required. In a browser, use a bundler or import map and serve the package's JavaScript, WebAssembly, and metadata assets from the same secure origin. For real-time browser audio, use the package-owned AudioWorklet. See the JavaScript guide and AudioWorklet guide for complete setup.

Chains and Effects

A Chain is an ordered serial effect pipeline. Each effect has a semantic type, an ID, and parameters; the same Chain JSON is accepted by both language bindings. The generated effect reference lists every available effect and its parameter names, defaults, ranges, channel behavior, latency, and asset requirements. Offline processing starts with fresh DSP state, while streaming preserves state across blocks.

For file and batch processing without writing an application, see the CLI guide.

Graph v1

Graph v1 is an experimental, opt-in API for static directed acyclic processing graphs. It keeps the same Effect objects as Chain v1 while adding fan-out, deterministic additive fan-in, edge gain/mute/stereo-pan/solo controls, wet/dry and send/return routing, and automatic delay compensation. Chain remains the default for serial processing, and existing Chain documents and behavior are unchanged.

A Graph is fully validated, prepared, and allocated before its immutable plan is installed. The audio callback does not validate or traverse the document, allocate memory, activate assets, or change the graph structure. See the Graph v1 guide for the document, JavaScript and Python APIs, recipes, snapshots, errors, and deliberate first-version limitations.

The generated Graph contract is the capacity authority for the schema, core, bindings, and public guide. Delay-compensation storage shares its published workspace limit rather than a separate delay quota.

DSP Core Development

This directory contains the host-neutral C++20 DSP core. It has no browser or WebAudio API dependencies. The build produces:

  • A native static library used by tests
  • A baseline standalone WebAssembly module
  • A SIMD128 standalone WebAssembly module

The WebAssembly modules are used by the web, PWA, and Electron hosts.

Prerequisites

  • CMake 3.24 or newer
  • Ninja
  • Python 3.10 or newer (standard library only, for binary model embedding)
  • A C++20 compiler for native tests
  • Emscripten SDK 6.0.11 for WebAssembly builds

On Windows, install and activate the version recorded in EMSDK_VERSION, then set EMSDK to the activated SDK root. The build script checks emcc --version and rejects other SDK versions.

Verification Workflow

Run these commands in order:

npm run gen:dsp
npm run test:dsp:warnings
npm run test:dsp
node tools/dsp-parity/run.mjs --native
npm run build:dsp
node tools/dsp-parity/run.mjs --wasm
node tools/dsp-parity/run.mjs --wasm --simd

The commands perform these checks:

  1. npm run gen:dsp validates every dsp/plugins/**/params.json and deterministically updates the C++ headers and runtime JavaScript packers in js/audio/dsp-params.generated.js. Add -- --check to verify freshness without writing.
  2. npm run test:dsp:warnings uses the pinned Emscripten Clang frontend to compile every registered native test source with warnings as errors, including -Wunused-but-set-variable. It does not link or run the tests.
  3. npm run test:dsp configures a native build and runs CTest.
  4. node tools/dsp-parity/run.mjs --native runs every golden through the native runner.
  5. npm run build:dsp repeats the warning check, builds the baseline and SIMD128 modules, copies them to plugins/dsp/, smoke-instantiates both artifacts, and writes deterministic metadata.
  6. The final two parity commands verify baseline WASM and baseline-plus-SIMD processing.

The native test and parity steps do not require emsdk; the warning check does. Build directories are created below the repository-root out/dsp/ directory.

ABI and Real-Time Memory

Native pipeline latency updates

Native hosts using core/engine.h can update compensation for an existing pipeline without rebuilding its topology or stopping processing. The C ABI and language bindings are unchanged. The Engine remains owned by one thread; these methods do not provide a mailbox or make other Engine operations thread-safe.

  1. On the Engine's owning audio thread, between blocks, stage parameters using the existing instance API and call capturePipelineLatencySnapshot(snapshot). The fixed-size snapshot records the configured topology and the kernels' latest reported latencies. Transfer its value to a control thread using the host's synchronized handoff.
  2. Off the audio thread, call the static Engine::preparePipelineLatencyUpdate(snapshot, update). It reads only the snapshot, calculates the plan and allocates compensation storage. It never reads or modifies the live Engine. update.plannedLatency() is a proposed value, not the applied latency.
  3. Transfer exclusive access to the prepared update to the audio thread. After staging that block's parameters and before processPipeline, call applyPipelineLatencyUpdate(update). It verifies the Engine, configuration revision and latest kernel latencies, then installs the compensation without allocation, deallocation, locks or I/O. On success, read pipelineLatency() on that thread and publish that applied value to the host.
  4. Transfer the update back to the control thread before reusing or destroying it. After a successful application it can own retired buffers. Snapshots and updates must not outlive their originating Engine. Never prepare, destroy or access an update concurrently with its application.

ET_ERR_STATE from capture means no pipeline is configured. Preparation returns ET_ERR_OOM when storage cannot be allocated; any failure makes that update unusable. Application returns ET_ERR_STATE for an unprepared, consumed, wrong Engine or stale update. All these failures leave the active compensation and its reported latency unchanged. Discard stale results on the control thread, capture the latest state and prepare again. A successful application consumes its update and invalidates other results captured against the previous plan.

Parameter staging and compensation preparation are separate. Kernels can report staged latency before their next processing call; latency changes also may arise from asset activation during processing. While preparation is outstanding, processing continues using the existing compensation. pipelineLatency() still reports that applied plan, not a promise that newly changed kernels already align with it. The host must request another snapshot when a kernel changes, apply a matching result at a block boundary, and update its bypass delay and host notification only after that application succeeds. This API does not provide an atomic transaction covering parameter changes, host bypass and host notifications, or guarantee click-free retiming.

Each processor receives its selected channels aligned to their longest input latency, whether it processes a stereo pair or all channels. Its selected outputs then share that latency plus the processor's own latency. This keeps channel splits and mixes aligned on every bus. A cross-bus send aligns its working copy without changing the source bus; additive bus merges and final output alignment apply the remaining delays.

Unchanged delays retain their history. Shorter delays reuse existing storage; longer delays copy all retained recent samples into prepared storage before swapping it in. History older than the previous capacity starts at zero. A merge changing which signal it delays clears only that channel's compensation history, since the old signal is not valid history for the new one. Output alignment retains history on channels even when their current delay is zero, once output delay storage exists. No effect state is reset and no processing block is replaced by silence or dry audio. The existing channel, bus, block-size, master-bypass and explicit reset contracts remain in effect.

ABI Contract

The C ABI in include/effetune/abi.h is the host ABI for the bundled WebAssembly modules. ABI version 1 is not a supported public native ABI or a native binary-compatibility promise. See the Phase 0 native ABI audit.

The main ABI rules are:

  • Exported signatures use only 32-bit handles and offsets; they contain no i64 values.
  • Each engine is independent and owns all of its DSP state.
  • et_engine_memory_required validates and preflights the engine arena.
  • Memory may grow only during et_engine_prepare or kernel setup from et_instance_create.
  • Hosts must refresh arena views after either lifecycle call and before the next audio quantum.
  • Audio processing never allocates or grows memory.

Engine Arena

The arena contains:

  • The combined buffer (bus 0) and buses 1-4
  • Four full-size scratch slabs: allChannels, mixing, stereo, and mono
  • A 4 KiB byte scratch slab
  • The telemetry ring and an equally sized telemetry staging slab

Neither engine processing nor a pipeline descriptor call allocates memory.

Deterministic Seeds

Random kernels receive deterministic 64-bit seeds through et_instance_set_seed(seedLow, seedHigh). Splitting the seed keeps i64 out of exported signatures. Parity hosts set each golden case seed explicitly; normal instance creation uses a deterministic, instance-derived default.

Kernel Capacity Decisions

The following kernels preallocate their maximum supported working storage. This keeps process allocation-free while preserving the documented parameter range.

Pitch Shifter

Allocation model

During prepare, Pitch Shifter allocates the maximum legal 500 ms window for the prepared sample rate, maximum channel count, and maximum frame count. Each channel receives:

  • One input window
  • One windowed-frame scratch area
  • One output ring with space for three windows

All three use Float32 storage. A separate Float32 final-output scratch area is sized as maxChannels * maxFrames.

Maximum supported shape

At 192 kHz and sixteen channels, the per-channel allocations contain 7,680,000 floats, or 30,720,000 bytes (30.72 MB, about 29.30 MiB). Including the small final block and index arrays, one maximal instance is budgeted at roughly 30.8 MiB of kernel heap.

Kernel heap is additional to the engine arena and shares the WebAssembly module's 256 MiB limit. The combined allocation of all effect instances must remain within that limit.

Why this capacity is retained

A smaller fixed allocation would either reject the documented combination of a 500 ms window, 192 kHz audio, and sixteen channels, or require an incompatible allocation during processing. Shape changes therefore clear only the active logical regions. Pitch and fine-tune changes retain the existing state.

Modal Resonator

Allocation model

The JavaScript implementation creates a two-second Float32 ring for every resonator and channel. The public frequency range starts at log-frequency 3.0, where exp(3) = 20.0855 Hz, so the longest legal integer delay is:

floor(sampleRate / exp(3))

The native kernel allocates that delay plus one ring slot. This is observably equivalent to the two-second ring because every read is relative to the write position, and samples older than the longest legal delay cannot be read—even after a live frequency change. Disabled resonators freeze both their position and storage.

Maximum supported shape

At 192 kHz and sixteen channels:

  • Ring length: 9,560 samples
  • Five-resonator storage: 3,059,200 bytes (about 2.92 MiB)
  • Literal two-second-ring storage avoided: 122,880,000 bytes

This keeps one maximal instance comfortably within the module's 256 MiB limit. The ring is allocated only during prepare; processing does not allocate.

A defensive clamp maps an out-of-schema delay to the largest allocated delay, preventing malformed raw parameter blocks from indexing outside the ring.

Noise Reduction

Noise Reduction reserves approximately 6 MiB of working storage at 192 kHz and sixteen channels. This allocation is made during prepare; processing does not allocate memory.

Spatial Mapper

Spatial Mapper reserves approximately 5.1 MiB at 192 kHz, sixteen channels and 48 analysis bands. The four channel timelines use 2.5 MiB, input and routed spectra use 2 MiB, and FFT scratch, band tables, covariance state and the stage schedule account for the remainder. Allocation occurs during prepare; processing and parameter changes do not allocate. The 8192-sample transform uses a 2048-sample hop and reports 10240 samples of latency at this rate.

The incremental PFFFT transforms and spatial operations share a 16-sample stage schedule. Scalar FFT steps receive a higher cost weight than SIMD steps based on their measured execution time; routing skips exact zero coefficients.

RS Reverb

Allocation model

RS Reverb preallocates every legal room-size delay during prepare. Each of its eight comb lines receives a separate capacity derived from that line's base delay:

capacity[i] = ceil(sampleRate * baseDelay[i] * 1.03 * 5 * 0.001)

The public maximum room size is 50 m. The 1.03 factor covers the largest positive delay jitter, and the room-size scale can multiply that delay by five. Channels share one line-capacity table and use fixed offsets into a single Float32 buffer.

Room-size changes select active lengths within the existing capacities. They do not reallocate memory, and neither does audio processing.

Maximum supported shape

At 192 kHz and sixteen channels:

Storage Float32 samples Bytes
Comb buffers 4,271,680 17,086,720
Complete instance 4,456,016 17,824,064

The complete instance includes the comb buffers, a pre-delay ring with ceil(sampleRate * .05) + 1 samples, and two 5 ms all-pass buffers per channel. Its total is about 17.00 MiB. Giving every comb line a uniform stride based on the longest line would waste more than 3 MiB at this shape.

State behavior

  • Sample-rate preparation recalculates line lengths.
  • Fixed delay-jitter table values are retained; this kernel has no random generator or seed.
  • An explicit reset clears delay-line history and restores the fixed initial state.

Structured Parameters

ABI version 1 supports bounded structured parameter blocks without changing the numeric float layout.

  • et_kernel_param_bytes_capacity returns zero for numeric-only kernels and the maximum accepted byte count for other kernels.
  • Hosts call et_instance_set_param_bytes after et_instance_set_params to stage a structured block.
  • Both calls use the same generated layout hash and become visible at the next process boundary.

Matrix routing uses the matrix-routes-v1 codec: a four-byte version/reserved/route-count header followed by ordered three-byte input/output/phase records. The 1,024-route limit fits the 4 KiB scratch slab and preserves duplicate route order.

Telemetry

Telemetry is emitted at 60 Hz by default. et_engine_set_telemetry_rate changes the engine-wide rate; a value of zero disables emission. et_telemetry_staging_ptr and et_telemetry_capacity expose the prepared staging slab read through et_telemetry_read.

All payloads are little-endian and four-byte aligned. Consumers must accept the exact payload size for the selected format version. The default format version is 1; TAP_SCOPE_SNAPSHOT (type 3), TAP_STEREO_FIELD (type 6), and TAP_AM_RADIO_SIMULATOR (type 17) use version 2. TAP_NOTE_SPECTROGRAM (type 24) uses version 5; TAP_RHYTHM_ANALYZER (type 28) uses version 4. TAP_SPECTRUM (type 4) and TAP_SPECTROGRAM (type 5) use version 2 for HQ output; Chroma Spiral always uses type 4 version 2.

Frame Types

  • Types 1-6 — analyzer frames. Type 2, TAP_GAIN_REDUCTION, contains one nonnegative float32 dB value and is shared by Compressor, Gate, Expander, and BrickwallLimiter.

  • Types 4 and 5 — TAP_SPECTRUM and TAP_SPECTROGRAM. Format version 2 is emitted for the HQ spectrum and spectrogram paths, including Chroma Spiral's type 4 spectrum. It has a 48-byte header with sample rate, FFT-size exponent, nominal hop, analysis generation, capture-end sample index, frame index, log-grid count and bounds, and the first valid grid index and count. Type 4 then carries current and peak-held float32 dBFS values for every grid cell; type 5 carries 256 high-to-low log-frequency uint8 display intensities. Public JavaScript and Python decoders expose these as SpectrumHqTelemetryFrame with kind spectrumHq and SpectrogramHqTelemetryFrame with kind spectrogramHq. Version 1 remains the non-HQ frame format.

  • Type 7 — TAP_LOUDNESS_LEVELS. Two float32 LUFS values.

  • Type 8 — TAP_TRANSIENT_GAIN. One signed float32 dB value.

  • Type 9 — TAP_CHANNEL_COUNT. One little-endian u32 in the range 1-16.

  • Type 10 — TAP_MULTI_CHANNEL_LEVELS. A u8 channel count, three zero bytes, then one eight-byte record per channel. Each record contains a nonnegative float32 raw window peak, a zero-or-one effective-mute byte, and three zero bytes. Payload size is 4 + 8 * channelCount bytes.

  • Type 14 — TAP_FIVE_BAND_DYNAMIC_EQ. Exactly 24 bytes: a five-band count, three zero reserved bytes, and five signed float32 gain values in band order.

  • Type 15 — TAP_VINYL_SIMULATOR. Exactly 48 bytes: eight float32 values followed by four cumulative little-endian u32 counters. The float values describe left/right contact force in N, left/right mean pressure in Pa, tip-velocity RMS in m/s, left/right tracking signal-to-error ratio in dB, and contact-centroid jitter in ns. The counters track mistracks, skips, static pops, and dust hits.

  • Type 16 — TAP_FM_RADIO_SIMULATOR. Exactly 216 bytes: five float32 values, one cumulative little-endian u32 counter, and forty-eight float32 spectrum magnitudes. The first values report RF input level in dBuV, estimated CNR in dB, pilot-lock quality from 0-1, stereo blend from 0-1, and multipath echo depth in dB. The counter tracks FM threshold clicks. The spectrum uses dBFS on a fixed logarithmic grid from 300 Hz to 60 kHz.

  • Type 17 — TAP_AM_RADIO_SIMULATOR. Format version 2 is exactly 28 bytes. Five float32 values at offsets 0, 4, 8, 12, and 16 report carrier level before AGC in dB, AGC gain in dB, modulation depth in percent, fading level in dB, and stereo blend. Cumulative u32 counters at offsets 20 and 24 track static and clipping events. Legacy version 1 is 24 bytes, omits stereo blend, and stores the counters at offsets 16 and 20; the parser accepts it for backward compatibility.

  • Type 18 — TAP_SW_RADIO_SIMULATOR. Format version 1 is exactly 24 bytes. Four float32 values at offsets 0, 4, 8, and 12 report carrier level before AGC in dB, AGC gain in dB, modulation depth in percent, and fading level in dB. Cumulative u32 counters at offsets 16 and 20 track static and clipping events. Shortwave reception is mono, so there is no stereo-blend field.

    The layout is the same in every reception mode, but several values are mode-dependent:

    • In AM, the pre-AGC IF level includes the carrier. In suppressed-carrier USB and LSB, it depends on the programme.
    • In USB and LSB, fading reports the virtual path gain at the suppressed carrier, not the attenuation or programme level of the sideband as a whole.
    • In USB and LSB, modulation depth represents transmitter sideband drive.
    • The clipping counter records only AM over-modulation and envelope-detector clipping, so it never advances in USB or LSB.
  • Type 19 — TAP_TUBE_SIMULATOR. Format version 1 is exactly 72 bytes: eighteen little-endian float32 values, first for the left channel and then for the right. Each channel contains stage 1 cathode voltage, stage 2 cathode voltage, B+ voltage, stage 1 grid-to-cathode voltage, stage 1 plate-to-cathode voltage, stage 1 plate current, stage 2 grid-to-cathode voltage, stage 2 plate-to-cathode voltage, and stage 2 plate current, in that order.

  • Type 20 — TAP_PHASE_SELECT_MAP. Format version 1 is a 16-byte header followed by pointCount 12-byte records, for an exact payload size of 16 + 12 * pointCount bytes. The header contains sample rate as float32, pointCount as u16 (maximum 512), flags as u16, FFT size as u32, and frame maximum level in dB as float32. Each record contains float32 frequency in Hz, signed L/R phase difference in degrees (-180 to +180), and level in dB relative to the frame maximum.

  • Type 24 — TAP_NOTE_SPECTROGRAM. Format version 5 is exactly 8,840 bytes: a 32-byte header followed by 440 float32 pitch-confidence levels in [0, 1], 440 float32 volume levels in dB, and 440 float32 revised pitch-confidence levels in [0, 1], followed by two revision records. Each record contains a u32 revision age and 440 float32 pitch-confidence levels in [0, 1]. The volume values include a 3 dB/octave correction above 100 Hz and use -240 dB for pitches without a measured level. The header contains float32 sample rate, observation time, and hop duration; u16 pitch count 440 and first MIDI note 21; and u32 frame index, divisions per semitone 5, non-zero analysis generation, and revision age. The revision age at offset 28 is 0 when there is no final revision (the revised confidence array is then zero-filled), or 8 when the revised confidences refer to the frame eight observations earlier in the same generation. The three arrays start at offsets 32, 1792, and 3552. The additional records start at offsets 5312 and 7076; their ages are 2 and 4 when available, or 0 with a zero-filled confidence array when unavailable. Their confidence arrays start at offsets 5316 and 7080. Public JavaScript and Python decoders expose this as NoteSpectrogramTelemetryFrame with kind noteSpectrogram; levels and volumeDb / volume_db are owned Float32Array or tuple values. Index i maps to MIDI firstMidi + (i - 2) / divisionsPerSemitone, placing five bins at -40, -20, 0, +20, and +40 cents around each piano-key center. revisionAge / revision_age exposes the revision age; revisedLevels / revised_levels is an owned confidence array or tuple when the age is 8, otherwise null / None. Its pitch indexing matches levels; the revised frame index is (frameIndex - revisionAge) modulo 2^32. revisions lists available updates in age order 2, 4, and 8 (40, 80, and 160 ms), each with age and owned levels; identify each revised frame by subtracting its age from frameIndex modulo 2^32 within the same generation.

  • Type 25 — TAP_TV_AUDIO_SIMULATOR. Format version 1 is exactly 216 bytes: five float32 values, one cumulative little-endian u32 error counter, and forty-eight float32 spectrum magnitudes. The float values report received carrier level in dBuV, estimated CNR in dB, scheme health from 0-1, selected path blend from 0-1, and multipath depth in dB. The spectrum uses dBFS on a fixed logarithmic grid. It represents recovered multiplex audio for analogue FM, detected audio for L AM, and selected output audio for NICAM.

  • Type 26 — TAP_PITCH_METER. Format version 1 is exactly 44 bytes. It contains float32 sample rate, observation time, hop duration, fundamental frequency in Hz, fractional MIDI note, cents offset, confidence, and input level in dB; u32 frame index and non-zero analysis generation; and u16 flags followed by a reserved u16. Flag bit 0 marks a voiced observation. Unvoiced observations set frequency, MIDI note, cents, and confidence to zero. Public JavaScript and Python decoders expose this as PitchMeterTelemetryFrame with kind pitch.

  • Type 27 — TAP_ANALOG_METER. Format version 1 is a 4-byte header followed by channelCount 8-byte records, plus a 24-byte program record in Loudness mode, for an exact payload size of 4 + 8 * channelCount bytes (+ 24 in Loudness). The header contains u8 mode (0 VU, 1 PPM, 2 RMS, 3 Sample Peak, 4 True Peak, 5 Loudness), u8 channel count (1-16), and u16 flags: bit 0 marks a valid Integrated value and bit 1 a valid LRA, and both are zero outside Loudness. Outside Loudness each record holds the float32 needle level in dB and the float32 maximum in dB since the last accepted frame; VU and RMS read a sine at its peak level. In Loudness each record holds that channel's unweighted, ungated Momentary and Short-term loudness in LUFS, and the program record holds float32 Momentary, Short-term, Integrated, and LRA, the maximum true peak in dBTP, and the Integrated accumulation time in seconds. Integrated and LRA read zero while their flag bits are clear. Levels are floored at -240 dB, and true peaks above 0 dBFS are not clamped. Public JavaScript and Python decoders expose this as AnalogMeterTelemetryFrame with kind analogMeter.

  • Type 28 — TAP_RHYTHM_ANALYZER. Format version 4 is exactly 1496 bytes. A 32-byte header holds float32 sample rate, u32 non-zero analysis generation, envelope hop in samples, and envelope frame count E, float32 observation time and latency in seconds, u32 count of events dropped since the previous accepted frame, and u32 current event count. A 32-byte tracker record follows: u32 flags (bit 0 marks the shown-beat gate), u32 stored grid epoch, float32 evidence-derived confidence and stored grid period in seconds, u32 stored anchor envelope frame, float32 fraction of that frame, u32 anchor beat index, and float32 best tempo candidate in BPM. The gate is independent of the stored grid: period and anchor fields may remain available with the gate closed, but are zero on digital silence or before a committed grid exists. 192 float32 tempogram values from 0-1 follow, then 16 event slots of 32 bytes, of which the first eventCount are valid: u32 envelope frame, float32 fraction, u32 lock epoch, i32 beat index, float32 beat fraction from 0-1 and period in seconds, float32 strength, u8 band (0 low, 1 mid, 2 high), u8 event kind, and a zero u16. Event kinds are 0 committed onset, 1 unlocated onset, 2 shown forward beat, 3 provisional onset, 4 hidden forward beat, and 5 committed beat. Kind 5 uses a signed i32 envelope frame. Onset strength is the band detector's hit probability in (0, 1]; beat strength is confidence in [0, 1], with band and beat fraction both zero. An onset can be delivered provisionally and later with committed coordinates at the same timestamp. At offset 1344, u32 preview count (maximum 12) and float32 preview period precede twelve 12-byte preview slots: i32 envelope frame, float32 fraction in [0, 1), and i32 beat index. The preview replaces the previous path and remains revisable independently of the gate; its period is zero when unavailable. Public JavaScript and Python decoders expose this as RhythmAnalyzerTelemetryFrame with kind rhythmAnalyzer.

  • Type 29 — TAP_TONAL_BALANCE_EQ. Format version 1 is exactly 1564 bytes, emitted at most 15 times per second. A 24-byte header holds float32 sample rate, u16 band count (41) and grid count (128), u8 state flags (bit 0 the last 400 ms block passed the -70 LKFS absolute gate, bit 1 it also passed the relative gate, bit 2 a valid integrated loudness, bit 3 a target with data), u8 target index, a zero u16, float32 integrated gated loudness in LKFS (zero while bit 2 is clear) and make-up gain in dB, and u32 gated-in analysis hop count. Six float32 arrays of 41 values follow for ERB-rate bands f_b = (10^((b+1)/21.4) - 1) * 1000 / 4.37 Hz (26 Hz to 18.6 kHz): measured level in dB on the Average SPL scale, persistence and presence from 0-1, the correction command in dB, and the active target mean and standard deviation in dB. 41 u8 band flags follow (bit 0 stationary, bit 1 noise floor, bit 2 has target, bit 3 has level, bit 4 inside Low-High), then three zero bytes and 128 float32 values of the applied response, including make-up gain, in dB at 20 * 1000^(i/127) Hz. Public JavaScript and Python decoders expose this as TonalBalanceEQTelemetryFrame with kind tonalBalance.

Latency and Pipeline Descriptors

et_instance_latency reflects staged parameters immediately. BrickwallLimiter reports:

  • At 1x oversampling: max(1, ceil(lookaheadMs * sampleRate / 1000)) samples
  • At 2x, 4x, or 8x oversampling: the same lookahead term plus ceil(62 / oversampling) samples

The routed EffeTune host reports and compensates aggregate pipeline latency. Serial library bindings report latency without trimming or padding rendered output, leaving offline placement to the host application.

The Phase-5 pipeline descriptor is validated transactionally. A malformed descriptor returns ET_ERR_DESC and leaves the previous valid descriptor active. Processing supports the existing channel-slice, section-gate, replace, and cross-bus additive semantics.

Shared DSP Primitives

Reusable real-time helpers live under include/effetune/dsp/:

  • biquad.h provides binary64 DF-I and TDF-II coefficients/state plus explicit legacy Float32 persistence-point quantization.
  • delay_line.h provides a prepare-time allocated, multichannel circular delay with integer and linearly interpolated reads.
  • smoothing.h provides one-pole, attack/release envelope, and linear smoothing state.
  • math.h provides dB/linear conversion, branch-based clamping, and denormal flushing.
  • xorshift_rng.h provides the reference-compatible xorshift64 13/7/17 sequence and 53-bit float conversion used by parity-sensitive noise and modulation kernels.

Prefer these helpers when their state and coefficient semantics match the JavaScript reference. Parity takes precedence when a legacy processor intentionally uses a different formula or persistence point.

Adding a Kernel

  1. Add dsp/plugins/<category>/<plugin>/params.json and kernel.cpp.
  2. Add one alphabetical EFFETUNE_PLUGIN entry to registry.inc.
  3. Run npm run gen:dsp and the parity generator before implementing the kernel.
  4. Derive from PluginKernel, use EFFETUNE_PARAMS, and register with EFFETUNE_REGISTER_KERNEL using the exact JavaScript constructor name.
  5. Allocate persistent state only in prepare; process must not allocate, lock, throw, perform I/O, or depend on a fixed frame count.

The shared native parity runner needs no per-plugin CMake entry. A dedicated complex native_test.cpp is not auto-discovered, so it must also be registered explicitly with add_executable and add_test in dsp/CMakeLists.txt.

Production kernels are registered in dsp/registry.inc; the committed WASM metadata records that registry and each generated parameter-layout hash. Native unit tests also add a test-only gain kernel to exercise lifecycle, parameter, routing, and telemetry contracts.

Native Spectrum Taps (C++ host API)

effetune/spectrum_tap.h provides per-effect Normal / HQ analysis in After / Compare modes. SpectrumTap averages the actual routed channels, aligns Before to After by the effect's latency, and returns spectra with peaks and timing. Normal matches the browser overlay's 4096-point Hann FFT and 1/12-octave power smoothing. HQ reuses the shared two-window analyzer and its logarithmic frequency cells. After omits input analysis; Off stops capture.

Allocate one tap per observed effect and call prepare(sampleRate, maxFrames) while its audio producer is idle. A serialized control/polling thread calls configure(mode, quality) and read(droppedBlocks). The audio callback calls capture for the matching Before/After pair from Engine::PipelineObserver, using the same exclusive block timeline, channel count and frame count. Capture only copies into a bounded SPSC queue; FFTs, delay storage and result allocation belong to read. Poll frequently enough to drain the queue. A gap, queue overflow, configuration change, latency change or explicit invalidate() starts fresh analysis with a new generation. Call invalidate() for bypass, topology changes and audio resets, even when the next block's frame number remains continuous.

Engine::pipelineTapLatency(instance, result) supplies the remaining delay from the observed input/output to final pipeline output, including routing compensation. Read it on the engine owner or inside its observer. Pass result.output as capture's tapDelayFrames, and the observer's effect latency as effectLatencyFrames. The host supplies firstFrame on its processing timeline; it need not use the DAW's seekable musical timeline.

The shared Spectrum Tap timing contract is version 1, represented by SpectrumTapTiming in C++ and SpectrumTapContract in plugins/multires-spectrum.js. Browser worklet messages carry the same object as timing; older PCM messages remain accepted. All fields use sample frames at the reported sampleRate:

Field Meaning
generation, frameIndex Monotonic capture lifetime and sequence within it; discard older generations and duplicate frames.
captureEndFrame Exclusive analysis capture end on the producer timeline. HQ already removes its 48-frame FIR group delay.
windowAgeFrames Distance from capture end to the low-frequency window center: Normal 2048, HQ 8192. HQ's short window ends at the same capture end.
completionFrames Analysis work after capture end: Normal 0; HQ 48 plus its hop rounded down to 16 frames. The hop is max(2048, ceil(sampleRate / 30)).
tapDelayFrames Remaining audio delay from the After tap to final pipeline output, excluding any additional host output delay.

The matching audible position is captureEndFrame - windowAgeFrames + tapDelayFrames + outputDelayFrames. Convert that position through the host's audio-output clock for display; polling time and wall-clock arrival time are not capture timestamps. Normal spectra use linear FFT bins, HQ spectra use 2048 log-spaced cells from 20 Hz to 40 kHz, and validCellCount excludes cells above Nyquist. Publish inputSpectrum in Compare and outputSpectrum in both modes; each contains current, peaks and validCellCount, accepted by the shared overlay.

Display synchronization also requires audio delay. Use timing.requiredOutputDelay(existingDelayFrames, deviceDelayFrames, maxFrames, deliveryBudgetFrames) (C++) or SpectrumTapContract.requiredOutputDelay (JavaScript) to reserve the deficit between analysis completion and audible output. The delivery budget covers the host's bounded polling/transport/render interval. Take the maximum requirement across active taps and apply it through the host's output delay, preserving normal effect and resampler latency. In a VST host, include this additional delay in getLatencySamples() and notify the host when it changes; the library does not call host interfaces. Schedule displays against the applied delay, and invalidate pending display frames when the output timeline changes.

Build the native tests, then run node tools/verify-spectrum-tap.mjs <path-to-effetune_dsp_spectrum_tap_tests> to compare Normal/HQ spectra, capture positions and timing against the browser at 48 and 96 kHz. This API does not add hidden pipeline nodes, alter processed audio or change the public C ABI.

Vendored Code

vendor/pffft/ contains the minimal float PFFFT v1.1.0 source used directly by the Spectrum Analyzer and Spectrogram kernels. The baseline artifact uses PFFFT's scalar path; the SIMD artifact compiles PFFFT with WebAssembly SIMD128 enabled. See vendor/pffft/LICENSE.txt and plugins/dsp/NOTICE.txt.