The three PRELUDE_SUMMARY_* processes each run as separate awsbatch jobs in the pandas container, and on a real run each one takes over five minutes end-to-end despite only ~1 second of actual compute — the rest is EC2 provisioning and image pull. Since these are trivial process_single QC steps (and SUMMARY_XML fires once per cycle), the cold-start overhead dominates and adds up fast on multi-cycle samples. It would be worth either collapsing them into a single process or running them on the head node so we're not spinning up a fresh oversized instance for each one.
The three
PRELUDE_SUMMARY_*processes each run as separateawsbatchjobs in the pandas container, and on a real run each one takes over five minutes end-to-end despite only ~1 second of actual compute — the rest is EC2 provisioning and image pull. Since these are trivialprocess_singleQC steps (andSUMMARY_XMLfires once per cycle), the cold-start overhead dominates and adds up fast on multi-cycle samples. It would be worth either collapsing them into a single process or running them on the head node so we're not spinning up a fresh oversized instance for each one.