Description of the bug
Multi-sample runs fail because the shared markers.csv is inflated with duplicate marker rows. The root cause is that UPDATE_FROM_OME dedupes markers on a map that still includes per-sample exposure metadata, so any channel whose exposure time differs across samples produces one row per distinct exposure. Single-sample runs are unaffected, which is why existing test data never caught it.
MCQUANT aborts with:
Exception: The number of channels in markers.csv doesn't match the image
The image is fine — instrumenting n_channels() in-container returns the correct count, and that count matches the rows of the input markers.csv. But the UPDATE_FROM_OME generated markers.csv has 64 rows instead of 44: the 44 correct unique marker_names, ragged-duplicated (DNA8 ×4, DNA3 ×3, GFAP ×3, several ×2).
The samples were imaged with different per-channel exposure times.
modules/local/omevalidation reads <Plane ExposureTime> into each marker's metadata (exposure_time, exposure_time_unit).
subworkflows/local/update_from_ome/main.nf builds one shared markersheet across all samples and dedupes it with .unique() on maps that still carry exposure_time.
So any (cycle, channel) whose exposure differs across samples is never collapsed, yielding one row per distinct exposure.
In the triggering run, of 44 (cycle, channel) pairs, 16 have more than one distinct exposure time across the 4 samples. Those extras sum to 20, giving 44 + 20 = 64. Per-marker multiplicity matches exactly (e.g. cycle 8 / ch 29 has 4 distinct exposures → DNA8 ×4).
The guard meant to catch this ("Inconsistent marker exposure data across samples") never fires: it .maps over the single toSortedList emission as if it were one marker (entry.cycle_number on a List), so the condition is never evaluated.
Scope / other triggers
The deeper flaw is that the shared markersheet merges per-sample OME metadata and then dedupes on the whole map, so any per-sample-varying field inflates it. Today the only such fields are exposure_time / exposure_time_unit (everything else comes from the single shared markers.csv), so exposure is the practical trigger — including the sub-case where some samples have a null exposure and others don't.
A related path produces the same symptom: if samples disagree on a cycle's channel_count, CHANNEL_DELTA_INDEX/agg gets duplicate cycle keys and combine(agg, by:0) fans the markers out. The per-sample count check in fix 3 below covers all of these at once.
Suggested fixes:
Generate the markersheet per sample. Run UPDATE_FROM_OME (and the markers.csv construction) per meta.id, so each MCQUANT receives a sheet built from and validated against its own image. This treats differing per-sample acquisition metadata (e.g. exposure times) as legitimate and removes the cross-sample .unique() that causes the inflation, covering the exposure, channel_count, and any-future-field triggers in one change.
- Key the markers.csv collectFile/combine in workflows/mcmicro.nf:178 by meta.id (today it broadcasts one shared sheet to every sample).
- The cross-sample consistency guard (update_from_ome L90-99) is then unnecessary; if enforcing identical panels across a cohort is still wanted, make it an explicit opt-in check (and fix it — it currently no-ops over the toSortedList emission).
- Defense-in-depth: a PRELUDE assertion that each sample's marker count equals its image SizeC, failing with a clear message.
- Independently, make MCQUANT's error report both counts + the marker file path.
Command used and terminal output
Relevant files
No response
System information
- nf-core/mcmicro
dev @ 54a9c89
- Nextflow 25.10.6
Description of the bug
Multi-sample runs fail because the shared
markers.csvis inflated with duplicate marker rows. The root cause is thatUPDATE_FROM_OMEdedupes markers on a map that still includes per-sample exposure metadata, so any channel whose exposure time differs across samples produces one row per distinct exposure. Single-sample runs are unaffected, which is why existing test data never caught it.MCQUANT aborts with:
The image is fine — instrumenting
n_channels()in-container returns the correct count, and that count matches the rows of the inputmarkers.csv. But theUPDATE_FROM_OMEgeneratedmarkers.csvhas 64 rows instead of 44: the 44 correct uniquemarker_names, ragged-duplicated (DNA8 ×4, DNA3 ×3, GFAP ×3, several ×2).The samples were imaged with different per-channel exposure times.
modules/local/omevalidationreads<Plane ExposureTime>into each marker's metadata (exposure_time,exposure_time_unit).subworkflows/local/update_from_ome/main.nfbuilds one shared markersheet across all samples and dedupes it with.unique()on maps that still carryexposure_time.So any
(cycle, channel)whose exposure differs across samples is never collapsed, yielding one row per distinct exposure.In the triggering run, of 44
(cycle, channel)pairs, 16 have more than one distinct exposure time across the 4 samples. Those extras sum to 20, giving 44 + 20 = 64. Per-marker multiplicity matches exactly (e.g. cycle 8 / ch 29 has 4 distinct exposures → DNA8 ×4).The guard meant to catch this ("Inconsistent marker exposure data across samples") never fires: it
.maps over the singletoSortedListemission as if it were one marker (entry.cycle_numberon aList), so the condition is never evaluated.Scope / other triggers
The deeper flaw is that the shared markersheet merges per-sample OME metadata and then dedupes on the whole map, so any per-sample-varying field inflates it. Today the only such fields are
exposure_time/exposure_time_unit(everything else comes from the single sharedmarkers.csv), so exposure is the practical trigger — including the sub-case where some samples have a null exposure and others don't.A related path produces the same symptom: if samples disagree on a cycle's
channel_count,CHANNEL_DELTA_INDEX/agggets duplicate cycle keys andcombine(agg, by:0)fans the markers out. The per-sample count check in fix 3 below covers all of these at once.Suggested fixes:
Generate the markersheet per sample. Run UPDATE_FROM_OME (and the markers.csv construction) per meta.id, so each MCQUANT receives a sheet built from and validated against its own image. This treats differing per-sample acquisition metadata (e.g. exposure times) as legitimate and removes the cross-sample .unique() that causes the inflation, covering the exposure, channel_count, and any-future-field triggers in one change.
Command used and terminal output
Relevant files
No response
System information
dev@54a9c89