fix(minimald): raise the guest vsock receive window to 8 MiB - #885

Fixes gominimal/minimal#869: workspace uploads over the guest vsock die with ENOBUFS, and the client reports copying tar stream to channel: channel closed.

Mechanism

The guest kernel (6.12.94, net/vmw_vsock/virtio_transport_common.c) rejects an incoming packet in virtio_transport_inc_rx_pkt() on either of two conditions:

1. Credit — buf_used + len > buf_alloc; the peer sent more than its advertised window.
2. Overhead — (skb_queue_len + 1) * SKB_TRUESIZE(0) > buf_alloc; the bookkeeping cost of the queued socket buffers alone exceeds the window.

Either rejection resets the connection and sets sk_err = ENOBUFS.

An instrumented guest kernel attributed 8 of 8 observed rejections to the overhead condition, never the credit one, each with 5–15 KB of window still unused. The arithmetic explains why. SKB_TRUESIZE(0) is 576 bytes on arm64 and buf_alloc defaults to VSOCK_DEFAULT_BUFFER_SIZE = 256 KiB, so the overhead ceiling sits at 455 queued skbs regardless of how much data those skbs carry. libkrun clamps each read to the credit still outstanding, so under receiver backpressure its packets shrink toward ~540 bytes — less than the 576-byte per-skb overhead they cost to queue. Past that crossover every additional packet consumes more window as overhead than it delivers as payload, and the overhead ceiling arrives before the credit ceiling ever can.

The change

Raise the receive window on the guest's vsock listener to 8 MiB. __vsock_create (af_vsock.c) copies buffer_size from the listening socket onto every socket it accepts, so one call at bind time covers every session.

Two details are load-bearing, and getting either wrong fails silently:

• SO_VM_SOCKETS_BUFFER_MAX_SIZE must be set before SO_VM_SOCKETS_BUFFER_SIZE. vsock_update_buffer_size() clamps the requested size to buffer_max_size, whose default (VSOCK_DEFAULT_BUFFER_MAX_SIZE) is also 256 KiB. Set the size alone and setsockopt returns 0 having changed nothing. This produced a convincing false negative during investigation.
• setsockopt returning 0 does not mean the value took effect. A clamped request succeeds. The effective window is read back with getsockopt and the result logged.

A failed sockopt logs a warning and the daemon carries on booting: a daemon with the default window is the pre-existing bug, not a reason to refuse to start.

Evidence

Fixture: a project whose single file compresses to exactly 12,784,144 bytes. Baseline failure rate on a stock guest is ~89% (8 of 9).

The third run is the useful one. It ran the patched binary, so the only variable that differed from the passing run was the window the kernel actually installed — which isolates the window as the cause rather than any other difference between the two builds.

This is headroom, not a fix

8 MiB does not change the ratio between the two ceilings; the overhead condition can still be reached. What it does is keep a normal upload out of the credit-starved tail where libkrun's packets degenerate below the per-skb overhead. A larger upload, a slower reader, or more aggressive backpressure could still get there.

The real fix belongs in libkrun, which should not shrink a packet below the overhead it costs to queue — a separate PR is in flight for that. This is the consumer-side mitigation, and it should stay useful regardless, since it also buys back ordinary buffering headroom.

Testing

cargo test -p minimald does not build on macOS (procfs), which is pre-existing, so verification ran under cross against the Linux target:

CROSS_CONTAINER_OPTS="--env HOME=/tmp" cross test -p minimald --target aarch64-unknown-linux-musl

• lib: 145 passed, 0 failed
• src/main.rs unit tests: 3 passed, 0 failed
• netns_root_integration: 3 ignored (require netns + gvproxy; run in the ci-linux-native netns job)

cross clippy -p minimald --target aarch64-unknown-linux-musl --all-targets -- -D warnings is clean, as is cargo fmt --check.

The runtime evidence above was collected before this PR and is not reproduced by CI — the guest window is only observable from inside a booted microVM.