Skip to content

Additive raw-bytes I/O door (mm_*_raw) across all four backends#12

Merged
octetta merged 3 commits into
octetta:mainfrom
billosys:feat/raw-bytes-api
Jun 19, 2026
Merged

Additive raw-bytes I/O door (mm_*_raw) across all four backends#12
octetta merged 3 commits into
octetta:mainfrom
billosys:feat/raw-bytes-api

Conversation

@oubiwann

Copy link
Copy Markdown
Contributor

Fixes #11

This is the raw-bytes feature implemented across all four backends, strictly additive, with nothing in the existing mm_message / UMP API touched. It gives callers that want to own MIDI semantics themselves (our case: an Erlang binding, midiio) a way to deal in exact wire bytes instead of the decoded struct.

It mirrors a pattern you already shipped: alongside the struct API and the UMP door (mm_in_open_ump, mm_out_send_ump, mm_ump_callback, MM_CAP_UMP) this adds a third parallel door — a _raw sibling per entry point, its own callback type, its own capability bit.

What's added (public surface)

/* one complete message per inbound callback, exact wire bytes */
typedef void (*mm_raw_callback)(mm_device* dev,
                                const uint8_t* data, size_t len,
                                double timestamp, void* userdata);

mm_result mm_in_open_raw(mm_context* ctx, mm_device* dev, uint32_t idx,
                         mm_raw_callback cb, void* userdata);
mm_result mm_in_open_virtual_raw(mm_context* ctx, mm_device* dev,
                                 mm_raw_callback cb, void* userdata);
mm_result mm_out_send_raw(mm_device* dev, const uint8_t* data, size_t len);

#define MM_CAP_RAW (1u << 5)   /* advertised by mm_context_caps where supported */

Plus two internal device fields (raw_callback, is_raw) alongside the existing ump_callback / is_ump. Lifecycle is unchanged — raw inputs open/start/stop/ close exactly like struct inputs.

Semantics

Raw mode is a faithful byte pipe:

  • Byte-exact, no rewriting — no note-on-velocity-0 → note-off folding, no status normalization. What's on the wire is what crosses the callback.
  • One complete message per inbound callback — a channel/system message as its full bytes; a SysEx as the whole F0…F7 (reassembled across packets where the backend fragments it); and a system-real-time byte (F8FF) arriving inside another message delivered as its own single-byte callback, kept out of the surrounding SysEx.
  • Outbound is byte-exact and uncapped — large SysEx to a virtual source works.
  • Timestamp keeps its existing meaning, surfaced as a callback parameter (there's no struct to carry it in raw mode).

Per-backend status

Backend mm_in_open_raw mm_in_open_virtual_raw mm_out_send_raw Notes
CoreMIDI Read-proc gains an is_raw framing branch; output uses a packet list sized to the payload (no length cap).
ALSA Uses ALSA's canonical snd_midi_event_decode/encode; "raw" means minimidio still owns the event↔byte conversion but hands you bytes.
WinMM MM_NO_BACKEND No virtual ports on WinMM (matches mm_in_open_virtual). Output frames the byte stream into midiOutShortMsg / midiOutLongMsg.
WebMIDI MM_NO_BACKEND No virtual ports in the Web MIDI API. The backend already forwarded raw byte arrays internally, so this was mostly wiring.

MM_CAP_RAW is advertised by all four backends. The two mm_in_open_virtual_raw stubs return MM_NO_BACKEND because those platforms have no virtual-port concept — intentional, not a gap.

Strictly additive

The existing mm_message decode paths, the UMP paths, and all mm_out_send* bodies are unchanged. Each backend's raw inbound branch is a single guarded insertion (if (dev->is_raw) { … return; }) at the top of the read handler, ahead of the struct path. (Each slice was reviewed with a git diff to confirm no existing logic moved.)

A note on the related bugs

If you saw the companion issues (velocity-0 folding inconsistency, CoreMIDI real-time-in-SysEx, the CoreMIDI virtual-source SysEx cap): the raw path sidesteps the first two by construction (there's nothing to fold or absorb when you're forwarding literal bytes), and mm_out_send_raw already sizes its buffers to the payload so it has no cap. But this PR deliberately does not modify the existing struct functions — those fixes are kept as separate PRs against their own issues, so this one stays purely additive. Happy to send those next if useful.

Testing — and where we're honest about gaps

  • CoreMIDItests/raw_loopback.c runs a virtual-port loopback on macOS: byte-exact short messages, velocity-0 pass-through, a >256-byte SysEx round-trip, and a real-time byte injected mid-SysEx (asserts the clock arrives separately and the SysEx payload stays clean). All pass.
  • ALSA — the same harness runs in a Linux VM (it uses two MIDI clients, since ALSA hides a client's own ports from its enumeration). All cases pass, including velocity-0 pass-through (which ALSA's struct path folds, but the raw path doesn't).
  • WinMM — cross-compile-checked with zig cc -target x86_64-windows-gnu -lwinmm. We don't have a Windows + loopMIDI setup, so a live round-trip is not yet done — the output framing has been unit-tested in isolation, but real-hardware delivery is untested. Flagging that honestly.
  • WebMIDI — builds to wasm with emcc. A browser round-trip isn't automated here; the path is a verbatim forward over the byte-array sender the struct API already uses.

tests/raw_loopback.c (loopback) and tests/raw_compile_check.c (a tiny TU that exercises the new call sites) are included; both are dependency-free C in the style of examples/.

One caveat

Line references in the commits and companion issues are against bb705e8.

Provenance

Human-directed, AI-assisted — the same workflow your AUTHORSHIP.md describes. For more info on the process, see my comment (with links) in #11.

oubiwann and others added 3 commits June 18, 2026 16:22
Adds a byte-transparent parallel door alongside the struct and UMP APIs,
mirroring the established _ump pattern: mm_raw_callback, MM_CAP_RAW, and
is_raw/raw_callback device fields, plus three entry points
(mm_in_open_raw, mm_in_open_virtual_raw, mm_out_send_raw).

CoreMIDI implementation:
  - mm__cm_raw_dispatch frames inbound bytes one complete message per
    callback: byte-exact (no velocity-0 folding), whole SysEx reassembled
    across packets via a new cm.sysex_pos accumulator, and system real-time
    (>= 0xF8) delivered as its own 1-byte callback even mid-SysEx and
    excluded from the SysEx body.
  - mm_out_send_raw sizes the MIDIPacketList to the payload (heap), so it is
    byte-exact with no length cap (resolves the U1 virtual-source SysEx cap
    by construction on the raw path).
  - mm_context_caps advertises MM_CAP_RAW.

Strictly additive: the existing struct decode loop and mm_out_send* bodies
are untouched. The read proc gains a single `if (dev && dev->is_raw)`
dispatch line; the CoreMIDI device struct gains a leading tag (mirrors the
ALSA/WebMIDI structs) so the field lives inside the documented Verify range.

WinMM / ALSA / WebMIDI define the three raw functions as MM_NO_BACKEND
stubs (real backends land in later slices); their caps do not advertise
MM_CAP_RAW.

tests/raw_loopback.c is a self-checking virtual-loopback harness covering
T1-T6 plus an mm_in_open_virtual_raw coverage case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Swaps the ALSA raw stubs left by slice 01 for real implementations, mirroring
the established struct/UMP paths. Strictly additive: the per-type struct decode
switch in mm__alsa_recv_thread and the mm_out_send*/mm_out_send_sysex bodies are
untouched.

ALSA implementation:
  - mm__dev_alsa gains snd_midi_event_t* midi_ev (the byte<->event coder),
    allocated in the raw opens / lazily in mm_out_send_raw and freed in
    mm_in_close / mm_out_close.
  - A raw inbound branch gated on dev->is_raw sits before the struct switch,
    parallel to the is_ump branch: SysEx events accumulate whole and deliver on
    0xF7; every other event is turned back into wire bytes by
    snd_midi_event_decode. snd_midi_event_no_status(...,1) on the decoder gives
    full-status, byte-exact framing and avoids running-status compression.
    Velocity-0 note-on decodes to 90 nn 00 unfolded (U2 passthrough for free —
    the fold lives only in the struct switch).
  - mm_out_send_raw encodes the byte buffer with snd_midi_event_encode and sends
    each produced event via the existing mm__alsa_send_ev helper. Byte-exact, no
    cap (a whole F0..F7 becomes one variable SysEx event).
  - ALSA mm_context_caps advertises MM_CAP_RAW.

Harness (tests/raw_loopback.c) reworked to two contexts (sender + receiver) so
the loopback works on both backends: ALSA's port enumeration skips the caller's
own client, so the slice-01 single-context "find my own virtual source" wiring
cannot see it. Two clients make the source visible cross-client; T1-T6 intent is
unchanged. The raw input is closed before T6 because ALSA drains one event queue
per client (a second recv thread would race the struct check). Verified passing
on macOS/CoreMIDI and Linux/ALSA.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Makes the raw door real on the two remaining byte-native backends, leaving
every backend either fully raw-capable or honestly stubbed. Strictly additive:
mm__wm_in_proc's struct branches, mm__web_dispatch_raw's parse loop, and all
mm_out_send* bodies are byte-unchanged; the raw branches are pure insertions.

WebMIDI (mostly wiring — the raw plumbing already existed):
  - mm__web_dispatch_raw gains a top-of-function is_raw branch that forwards the
    event's bytes verbatim (Web MIDI delivers one complete message per event, so
    no framing).
  - mm_in_open_raw mirrors mm_in_open; mm_out_send_raw forwards to the same JS
    sender mm_out_send uses. MM_CAP_RAW advertised. virtual_raw stays a stub.

WinMM (framing on both edges):
  - mm__wm_raw_data_bytes helper (same table as CoreMIDI's). mm__wm_in_proc
    gains a top is_raw branch: MIM_DATA unpacks the packed short message into
    1 + data_bytes(status) wire bytes; MIM_LONGDATA forwards the SysEx buffer,
    then re-adds it.
  - mm_in_open_raw mirrors mm_in_open. mm_out_send_raw walks the buffer, framing
    short messages via midiOutShortMsg and a whole F0..F7 via midiOutLongMsg
    (heap buffer sized to the payload — byte-exact, no length cap). MM_CAP_RAW
    advertised. virtual_raw stays a stub (no virtual ports on WinMM).

tests/raw_compile_check.c references all three raw entry points + MM_CAP_RAW so
the cross/emcc builds type-check the new call sites. Verified: zig cc
(x86_64-windows-gnu) and emcc both exit 0 with zero new warnings vs base;
CoreMIDI and ALSA still compile. Runtime round-trips on WinMM (needs loopMIDI)
and WebMIDI (needs a browser) are deferred-with-rationale per the ledger.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@octetta octetta merged commit 0fb49e6 into octetta:main Jun 19, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: additive raw-bytes I/O (mm_in_open_raw / mm_out_send_raw) for byte-transparent transport

2 participants