Skip to content

Make global QNAME uniqueness an explicit stream policy #4

Description

@sounkou-bioinfo

Goal

Make the query-name trust model explicit without forcing every production stream to retain every previously seen name.

mb_query_stream() currently performs exact repeated-QNAME detection with a caller-bounded hash table. That is the safe default, but memory grows with the number of unique templates and the stream fails when max_seen_qnames is reached. Callers that already guarantee globally unique normalized QNAMEs need a deliberate way to accept responsibility for that invariant.

Required contract

Add two named policies to stream creation:

  • validate_unique — preserve the current behavior. Track every normalized QNAME exactly, fail on a repeated name, and fail with the existing typed tracker-limit error before validation becomes incomplete.
  • declared_unique — the caller asserts that each normalized QNAME occurs in exactly one logical input group across the complete input. Do not allocate or maintain a global seen-name table.

validate_unique remains the compatibility default. Selecting declared_unique must be explicit.

Both policies must continue to enforce all local input invariants:

  • paired files are consumed in lockstep;
  • mate names match after the documented normalization;
  • /1 and /2 mate suffixes, when present, are consistent;
  • malformed/truncated FASTQ and unequal mate counts remain typed failures; and
  • each emitted group retains its deterministic input_order.

Do not add an auto mode, probabilistic detector, eviction policy, or silent transition from validation to trust. The tracker limit applies only to validate_unique; contradictory policy/limit arguments should be rejected rather than ignored.

Native API and provenance

Expose the selected policy as a versioned enum in the installed C API and in stream/header metadata so a native consumer and any output receipt can distinguish validated uniqueness from a caller assertion. Update the stream ABI deliberately if the structure change cannot be made compatibly through struct_size.

The declaration means exactly this: after the existing QNAME normalization, the name identifies one single-end record or one paired template in the whole input. If that assertion is false, global repeats are intentionally not detected by Rminibwa; documentation must state that outputs relying on complete template groups are then invalid.

Acceptance criteria

  • R and C callers can select either policy unambiguously.
  • Existing calls retain exact validation by default.
  • validate_unique still reports distinct repeated-name and tracker-limit errors.
  • declared_unique can process more templates than the validation cap without allocating the global name tracker.
  • Paired-name mismatch, mate-order, malformed FASTQ, and cancellation tests pass under both policies.
  • The installed C consumer fixture observes the selected policy through versioned metadata.
  • R documentation, the downstream C vignette, and query-stream tests describe and prove the trust model.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions