Description of the bug
On large stitched images, MCQUANT fails with:
Exception: The number of channels in markers.csv doesn't match the image
…even when the markers and the image genuinely have the same number of channels (in our case markers.csv = 44 rows and the image's OME SizeC = 44). Small images (e.g. the exemplars) are always fine — it only shows up on large, multi-sample runs, and each failing attempt spends ~15 min before erroring.
The cause is in n_channels() (mcquant/SingleCellDataExtraction.py): the channel count is derived from min(tifffile.TiffFile(image).series[0].shape), which requires parsing the full IFD chain of a multi-GB pyramidal OME-TIFF. On AWS Batch (no Fusion) the image is copied to local scratch before the task runs, and under concurrent multi-sample execution that copy can come back incomplete (scratch/throughput contention, no integrity check). tifffile then can't walk the full directory and returns a 2-D shape, so n_channels returns 1 → the check fails. It's a false negative caused by reading the pixel payload instead of the metadata.
Fix would be to read the channel count from the OME-XML SizeC rather than from min(series[0].shape) — it lives at the tail of the file, is cheap, and doesn't depend on a fully-staged/parseable pixel payload. (Optionally also validate the staged input, and/or let this exception be retried.)
Command used and terminal output
Relevant files
No response
System information
No response
Description of the bug
On large stitched images, MCQUANT fails with:
…even when the markers and the image genuinely have the same number of channels (in our case markers.csv = 44 rows and the image's OME SizeC = 44). Small images (e.g. the exemplars) are always fine — it only shows up on large, multi-sample runs, and each failing attempt spends ~15 min before erroring.
The cause is in
n_channels()(mcquant/SingleCellDataExtraction.py): the channel count is derived frommin(tifffile.TiffFile(image).series[0].shape), which requires parsing the full IFD chain of a multi-GB pyramidal OME-TIFF. On AWS Batch (no Fusion) the image is copied to local scratch before the task runs, and under concurrent multi-sample execution that copy can come back incomplete (scratch/throughput contention, no integrity check).tifffilethen can't walk the full directory and returns a 2-D shape, son_channelsreturns 1 → the check fails. It's a false negative caused by reading the pixel payload instead of the metadata.Fix would be to read the channel count from the OME-XML
SizeCrather than frommin(series[0].shape)— it lives at the tail of the file, is cheap, and doesn't depend on a fully-staged/parseable pixel payload. (Optionally also validate the staged input, and/or let this exception be retried.)Command used and terminal output
Relevant files
No response
System information
No response