Skip to content

Add Swarm foundations and the sans-I/O relay-server service #70

Description

@deepso7

Parent

Part of #61.

Canonical specification

Implement PR 2 of docs/plans/circuit-relay-v2-relay-server.md. That document and the resolved decisions in #62#68 are authoritative. Do not reopen product design. In particular, preserve the frozen cardinality, admission order, address policy, lifecycle, byte-accounting, timing, error, and low-level driver-seam decisions.

What to build

Deliver the shared Swarm mechanisms and a complete deterministic no_std + alloc relay-server service. A low-level host must be able to drive reservations and circuits entirely through caller-supplied time, Swarm events, tokenized actions, and echoed results, with no sockets, clocks, waiting, or Endpoint policy inside the service.

Land the breaking Swarm foundations and migrate all consumers atomically before adding the service. Keep relay policy out of Swarm.

Swarm foundations

  • Replace the internal single protocol registry with independent inbound, outbound, and Identify-advertised roles while preserving SwarmBuilder::protocol and SwarmRuntime::add_protocol as registration in all three roles.
  • Provide crate-internal role registration for composed services: relay-server owns inbound and advertised HOP plus outbound STOP; NAT owns outbound HOP plus trusted inbound and advertised STOP.
  • Incoming multistream-select snapshots only inbound membership, outbound opens check only outbound membership, and future Identify responses snapshot only advertised membership.
  • HOP opened over a circuit connection negotiates and is denied with PERMISSION_DENIED rather than failing negotiation.
  • Add ConnectionCloseCause::{Transport, Superseded} to SwarmEvent::ConnectionClosed.
  • The synchronous last-wins connection replacement path emits Superseded; transport loss and explicit transport closure emit Transport.
  • Atomically migrate every Swarm, NAT, portable/std Endpoint, discovery, pubsub, fixture, and test consumer of the breaking close-event shape. Do not add compatibility shims.
  • Expose exact-connection remote transport-address lookup for IP limiting while retaining the single-connection-per-peer invariant.
  • Tests cover outbound-only HOP, advertised inbound HOP, inbound/outbound/Identify snapshots, HOP-on-circuit denial, and both close causes.

Sans-I/O service contract

  • Add a dedicated relay-server crate that is no_std + alloc, uses minip2p_platform::Now, and owns reservations, circuits, admission, token buckets, limiters, deadlines, forwarding actions, byte accounting, and public lifecycle events.
  • Implement the frozen RelayServerConfig, RateLimit, defaults, typed configuration/address errors, public events/reasons/errors, opaque tokens, StreamKey, action/result API, polling API, ownership queries, timeout reporting, and idle detection exactly as specified.
  • Validate before construction: reject zero reservation duration, zero control timeout, enabled limiters with zero capacity/refill interval, and circuit durations above the wire u32::MAX bound. Never panic or silently saturate configured duration into the wire value.
  • Zero reservation/circuit/pending capacities deny new work. Zero circuit duration and bytes are unlimited and are advertised as zero.
  • Token buckets start full, consume admitted checks, refill one token per elapsed interval up to capacity, use saturating arithmetic, and remove full buckets when useful. Use an expiry-ordered due-entry sweep without full peer/IP map scans and without an artificial simultaneous-refill budget.

Address and reservation behavior

  • Validate, normalize, and deduplicate supported TCP/QUIC announce addresses; reject wildcard hosts, circuit addresses, conflicting peer IDs, and unsupported shapes with indexed actionable errors.
  • Select exactly the first non-empty usable source: explicit override, AutoNAT-confirmed public direct addresses, then concrete bound listeners. Never use raw Identify-observed addresses.
  • Runtime replacement is atomic, empty input clears the override, address changes preserve existing state, and RESERVE responses append the local peer ID exactly once.
  • Encode the longest stable source-order address prefix that fits the 8 KiB control frame; refuse new reservations/renewals if no address fits, preserving the prior reservation on failed renewal.
  • Enforce one committed reservation per peer while retaining exact connection identity. Same-connection RESERVE renews; replacement defensively supersedes before fresh admission; stale old-connection inputs cannot mutate replacement state.
  • Apply RESERVE admission in the frozen order: accepting/usable addresses, peer limiter, IP limiter, exact global replacement-aware capacity, then response acceptance and commit.
  • Every renewal consumes applicable limiter tokens but replaces its existing global slot. Equality is allowed; zero global capacity denies new work. The first IP component of the exact remote address is the IP key; no IP means that limiter is inapplicable.
  • Initial and renewal acceptance commit only after the success response is accepted by transport. Failed initial delivery creates no lifecycle; failed renewal preserves the old reservation and deadline.
  • Monotonic time alone governs lifetime. Optional Unix expiry is saturating wire/event metadata. Process due expiry before same-timestamp inputs and emit exactly one terminal reason: expired, transport-closed, superseded, or internal failure.
  • Pause rejects new RESERVE with RESERVATION_REFUSED while preserving reservations and circuits.

Circuit and forwarding behavior

  • Apply CONNECT checks in the frozen order: accepting, source-peer limiter, source-IP limiter, both endpoints' per-peer counts (count self once), global capacity, then live exact destination reservation.
  • Map paused, capacity/rate, missing reservation/connection mismatch, malformed requests, and every STOP outcome to the exact specified HOP status.
  • Reserve circuit capacity while STOP is pending and release it without lifecycle events on refusal, timeout, or pre-commit failure.
  • Enforce separate post-negotiation per-connection HOP and STOP caps. At the HOP cap own/reset the stream until terminal; at the STOP cap deny with RESOURCE_LIMIT_EXCEEDED without opening STOP. Do not add a Swarm-wide pre-negotiation cap.
  • HOP timeout spans negotiated readiness through parsed decision and accepted response. STOP timeout spans open/negotiation through accepted response. Release each worker count exactly once.
  • Commit and emit CircuitOpened only after STOP acceptance and transport acceptance of the HOP success response; then start duration timing and release pipelined payload. Pre-commit failures clean both legs without an opened/closed lifecycle pair.
  • Serialize tokenized sends per direction and require the driver to echo every open/send/close/reset result. Verify STOP opens land on the reserved exact connection and clean up successful stale opens.
  • Keep bounded in-flight state: transport queues own payload backpressure, and the host drains actions/results to quiescence after each claimed input.
  • Count accepted application bytes independently and saturatingly in each direction, including HOP/STOP pipelined payload. Equality remains open; the first successful non-empty crossing chunk is fully counted/delivered and then closes both legs with the crossing direction.
  • A failed crossing send counts no bytes and closes as ForwardFailed. Zero bytes is unlimited. Duration begins at commit, includes handshake leftovers, and zero duration is unlimited.
  • EOF propagates half-closes and ends after both directions close. Reset, send failure, connection loss, duration, byte limit, and internal failure clean the other leg and emit exactly one CircuitClosed with both totals.
  • Deadline-first ordering applies at equal timestamps. Ignore stale timer, stream, and action results after terminal state.
  • Emit each stable denial once when decided; delivery failure adds one runtime error without suppressing or duplicating denial. Committed internal failure emits both its stable close reason and diagnostic error.

Tests

  • Scripted sans-I/O tests cover defaults; invalid, zero, disabled, and wire-bound configuration; typed field/reason diagnostics; and all frozen default values.
  • Tests cover limiter refill/boundaries, admission order, renewal token use, IP extraction, disabled limiters, expiry-ordered bounded cleanup, and saturating arithmetic.
  • Tests cover reservation commit/failure, renewal replacement/failure, full-capacity renewal, supersession, defensive last-wins behavior, stale isolation, exact-deadline priority, wall-clock absence/jumps/saturation, and exactly-once closure.
  • Tests cover every address source/precedence/shape, clearing, atomic invalid replacement, deduplication, peer-ID normalization, 8 KiB truncation, empty refusal, future selection changes, and renewal preservation.
  • Tests cover both-end circuit admission, self-circuit counting, deterministic status mapping, pause/resume, circuit HOP, separate caps, control timeouts, and reserved-slot release.
  • Tests cover directional pipelining, equality, full-chunk overshoot, failed sends, zero wire values, saturation, duration, half-close, stale terminals, and exactly-once totals/reasons.
  • A memory-only NatAgent client pair reserves and exchanges bidirectional payload through RelayServerAgent.

Documentation and compatibility

  • Add crate README and crate-level documentation explaining the module boundary, caller-driven action/result loop, deadline-first order, quiescent draining requirement, single-connection-per-peer assumption, and public errors/events.
  • Every public item has rustdoc with actionable failure semantics.
  • Explicitly document deliberate rust-libp2p deviations resolved in Audit the relay-server compatibility envelope #62Approve the implementation-ready relay-server specification #68: one reservation per peer; corrected rate-first/exact-capacity admission; renewal replacement; both-end circuit limits; static HOP behavior; circuit-HOP denial; deterministic malformed handling; directional accepted-byte accounting including pipelined data; duration start/zero behavior; exactly-once typed lifecycles; and separate post-negotiation caps.
  • Preserve the pinned protocol behavior where required, including exact statuses, STOP-to-HOP mapping, and voucher: None.
  • Keep sockets, system clocks, blocking waits, Endpoint composition, ACLs, metrics, discovery/autorelay, proxy-aware IP limiting, and portable Endpoint driving out of this crate.

Completion checks

  • just fmt passes.
  • just test passes.
  • just clippy passes with warnings denied.
  • just check-nostd passes, including the new crate.
  • The full workspace is migrated atomically with no old event/protocol-registration compatibility shim.
  • Every lifecycle has tested commit and exactly-once terminal behavior, and every asynchronous operational failure reaches the diagnostic event channel.
  • No frozen behavior is left to implementation judgment outside the canonical specification.

Blocked by

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions