Skip to content

Integration of genetic modules and donor matching - #78

Merged
nictru merged 43 commits into
devfrom
aggregate-genetic-new
Nov 24, 2025
Merged

nictru merged 43 commits into
devfrom
aggregate-genetic-new

Conversation

@LuisHeinzlmeier

@LuisHeinzlmeier LuisHeinzlmeier commented Nov 11, 2025 •

Copy link
Copy Markdown
Collaborator

Updated dataset

  • Tested all modules with an updated test dataset (barcodes from RNA and HTO weren't matching)
  • Note: demuxem is currently not working after this dataset update.
    --> demuxem does not work #81

Summarizing gene and hash results

Donor match

  • Both genetic and hashing results are now merged into a single summary csv.
  • The merged summary files are then passed to donor match, which calculates the best pairing between hashing and genetic methods.
  • findVariant = true still needs to be implemented.
    --> Implement findVariant in donor_match #82

Thanks to @seokim for the huge contribution to aggregating the genetic modules!

@nictru nictru left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a full review but rather some notes on how to improve the ´fasta´ handling. The secret sauce here is that this does not override the params.fasta if it is already set directly. This is because in nextflow, parameter assignments directly in the pipeline code have lower priority than assignments via any type of configuration file.

I wrote these change suggestions without testing them, so they may contain typos or be incomplete, but I think you can get the idea

Comment thread nextflow.config Resolved
Comment thread nextflow_schema.json Resolved
@LuisHeinzlmeier

Copy link
Copy Markdown
Collaborator Author

The test is currently failing only because of two souporcell output files: one due to a rounding error in ambient_rna.txt and another because of a difference in the cluster_genotypes.vcf’s date. I’m a bit surprised this didn’t already fail in the souporcell PR.

-ambient RNA estimated as 26.61494510799508%
+ambient RNA estimated as 26.61503361443396%
-##bcftools_concatCommand=concat -a test1/souporcell_0_0.vcf.gz test1/souporcell_1_0.vcf.gz test1/souporcell_2_0.vcf.gz test1/souporcell_3_0.vcf.gz; Date=Sat Nov 15 13:12:13 2025
+##bcftools_concatCommand=concat -a test2/souporcell_0_0.vcf.gz test2/souporcell_1_0.vcf.gz test2/souporcell_2_0.vcf.gz test2/souporcell_3_0.vcf.gz; Date=Sat Nov 15 13:14:02 2025

I actually don’t need either of these files for the pipeline. I only included them because the documentation listed them as the main outputs. For hadge, I could simply remove them. If we want to keep them, I’d have to manually adjust the files after running souporcell_pipeline.py, since I’m only using the command-line version of souporcell.

What do you think, @nictru?

@nictru

nictru commented Nov 17, 2025

Copy link
Copy Markdown
Collaborator

In a case like this you can make use of the tests/.nftignore file which is referenced here.

The tests did not fail in the souporcell PR because of the way the tests are implemented there.

@LuisHeinzlmeier

Copy link
Copy Markdown
Collaborator Author

test1.clust1.vcf.gz from freemuxlet includes the date into the file with ##fileDate=20951117. That's why the test fails after re-running a day later. As the pipeline doesn't use this file, I would also include test1.clust1.vcf.gz in tests/.nftignore.

@LuisHeinzlmeier

Copy link
Copy Markdown
Collaborator Author

The pipeline passed all tests! ✅
Now, I would really appreciate a review before we merge it.

@nictru

nictru commented Nov 18, 2025

Copy link
Copy Markdown
Collaborator

Great! Will review in the evening

@Zethson Zethson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!
I'll leave the Nextflow bits to Nico.

Comment thread conf/test.config Resolved
Comment thread modules/local/donor_match/main.nf Resolved

@nictru nictru left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general, just a few small things and open TODO strings

Comment thread conf/test.config Resolved
Comment thread modules/local/donor_match/main.nf Resolved
Comment thread modules/local/donor_match/main.nf Resolved
Comment thread modules/local/donor_match/main.nf Resolved
Comment thread modules/local/donor_match/main.nf Resolved
Comment thread modules/local/donor_match/main.nf Resolved
Comment thread modules/local/gene_summary/main.nf Resolved
Comment thread workflows/hadge.nf Resolved
Comment thread workflows/hadge.nf Resolved
@LuisHeinzlmeier

Copy link
Copy Markdown
Collaborator Author

Thank you @nictru and @Zethson for the review! I incorporated the feedback, and all tests passed except nf-core template version. @nictru , could you perhaps update the nf-core version again?

If there’s anything else I should adjust, just let me know! Otherwise, we’re good to merge, right?

@nictru

nictru commented Nov 24, 2025

Copy link
Copy Markdown
Collaborator

I think it's fine to merge this now; I can do the template update in a separate PR later. Not sure when I will have time for this.

Reminder: The fact that we are one version behind the latest nf-core template would only be displayed as a note if we were within the nf-core organization. We only get an error because the displaying of the note fails.
I am not entirely sure what changes we would need to make for this to work, but it's not particularly important. Once we move the repo to nf-core it should be fixed automatically.

@nictru
nictru merged commit b8a7d0e into dev Nov 24, 2025
@nictru
nictru deleted the aggregate-genetic-new branch November 24, 2025 22:17
LuisHeinzlmeier added a commit that referenced this pull request Sep 28, 2026
* Skip fastqc template feature

* Fix .nf-core.yml

* Fix pipeline initialization

* Init top-level subworkflow structure

* Fix top-level structure bugs

* Strength samplesheet validation

* Add tool selection parameters

* Pass methods to subworkflows

* Update nf-core modules and subworkflows

* Add method condition blocks

* Install existing nf-core modules

* Simplify raw/filtered matrix treatment

* Improve hash/genetic channel structures

* Patch demuxem module

* Implement demuxem

* Add GMM-Demux

* Update genetic parameterization

* Add POPSCLE_DSCPILEUP

* Implement demuxlet

* Implement freemuxlet

* Implement cellsnp

* Implement vireo

* Implement BAM QC

* Use PLP for demuxlet

* Implement BAM subsetting to common variants

* Update outdir structure

* Update demuxem output definition

* Add all demuxem optional parameters

* Add GMM-demux optional params

* Add cellSNP optional parameters

* Add vireo optional params

* Add popscle-dscpileup optional parameters

* Add demuxlet/freemuxlet optional params

* Use github CI runners

* Add new hash demultiplexing modules (#66)

* hasheddrops inserted and they run in the pipeline

* insert local preprocessing module for htodemux and multiseq (not tested yet)

* adding htodemux and multiseq to the pipeline

* small improvements

* add parameters of modules also to the pipeline

* address

* run pre-commit for modules/local/preprocessing_for_htodemux_multiseq

* add htodemux visualization (not tested yet)

* visualization added to the pipeline (works locally)

* add visualization params to config files

* change strings to type boolean for htodemux visualization

* improve visualization

* change data types for preprocessing

* change data types of nf-core modules if possible

* Prettier

* Pre-commit

* Fix nf-core modules not matching remote

* Fix PULL_REQUEST_TEMPLATE issue based on this thread: https://nfcore.slack.com/archives/CQY2U5QU9/p1747475193145499

* Solve remaining prettier issues

* One more prettier fix :D

* Revert "Fix nf-core modules not matching remote"

This reverts commit ac3ed24.

* Make test profile work

* Patch modules

* Prettier

* Gitignore test intermediates

* Add pipeline test snapshot

* Make sure hash diffs are visible

* Ignore some more test snaps

* Enable new hash-based tools

* Set prefix to prevent file name collisions

* nft-ignore some outputs

---------

Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Aggregate hashing (#68)

* first steps

* current changes (not tested yet)

* save changes

* htodemux and multiseq in hash_summary.py (without mudata and anndata) and debugging of ch_hashing_summary

* solve summary input with join

* pipeline finally calls the script

* empty inputs solved

* add all inputs to main.nf (empty inputs still work)

* update htodemux in script

* script passed and produced results

* correct results

* solve incorrect merging

* add demuxem (runs but merging to the summary table is not working)

* remove the duplicate variable name

* add demuxem to summary (has more barcodes than htodemux and multiseq)

* add hasheddrops

* add gmmdemux to summary

* first steps to improve hasheddrops outputs

* fix hasheddrops output

* fix using unknown as value

* remove comments

* adding hashsolo doesn't work (cannot access the correct cell_hashing_columns)

* add bff summary function

* bff works

* join into barcodes from raw hto file to adress demuxem having more output barcodes that input barcodes

* pipeline pass with class Arguments

* move testing to Argument class

* restructure hash_summary.py

* mudata and anndata outputs are created

* hashsolo works

* replace manual str with args variable

* update hasheddrops

* update modules and improve linting

* Skip “prettier” failing – covered in #69

* clean up code 1

* clean code 2 (remove hard-coded parameters)

* clean code 3 (hash_sumary.py)

* update modules bff and hashsolo

* update schema

* remove properties created by nf-core pipelines schema build --no-prompts

* structure hashing paramters of nextflow.config

* update hasheddrops (removed one TODO in hash summary, hardcoding of mapping from id to hash)

* remove hardcoded hash_list

* remove linting error

* use left join for classification and small improvements for summary csv

* remove debugging outputs

* Fix demuxem MPLCONFIGDIR

* Add hash_summary environment variables

* Comment out arg printing in hash_summary

* Update demuxem patch file

* Prettier

* update snapshot

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Template update to 3.3.2 (#74)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Update multiQC module

* Fix vireo_cell_range issue

* Template update to 3.4.1 (#79)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Fix some linting problems

* Use github runners instead of self-hosted

Updated runner configuration to use 'ubuntu-latest' for nf-test jobs.

* Fix runner configuration in nf-test workflow

* Template update for nf-core/tools version 3.4.1

* Fix logo images

* Integration of genetic modules and donor matching (#78)

* save first steps

* join outputs

* gene_summary merges results for vireo, demuxlet and freemuxlet

* merge on Barcode

* all modules work for gene_summary but demuxlet has still empty values after merging

* prepare donor matching

* try understanding donor match

* debugging donor match

* donor_match produces something

* donor_match produces something

* donor_match seems to produce correct output files now, but it still needs to be tested with properly joined data

* pipeline is running with new input data and donor_match produces correct output

* first updates to souporcell

* update souporcell

* final versions.yml

* save anndata/mudata, output before joining and left join to rna barcodes

* couln't fix demuxEM error, opened an issue at developer's github

* gene/hash_summary: save mudata/anndata, print results overview fuction and clean code

* include all methods in donor_match and update bff() in hash_summary

* improve the readability of hadge.nf

* use fasta from iGenomes for souporcell but not fully tested (see #77)

* remove prints and linting errors

* update snapshot

* Merge origin/dev

* update souporcell

* Fix image files

* update snapshot

* update snapshot 2

* fasta updates

* update snapshot with 3 modules each (hash/gene)

* update fasta description

* update snapshot

* replace  set with list to keep the same order

* snapshot from codespace

* add souporcell outputs to .nftignore

* Add trailing newline to .nftignore

* fix end of file

* add freemuxlet/*/*.clust1.vcf.gz to .nftignore

* Incorporate review comments

* update snapshot

---------

Co-authored-by: nictru <nictru32@gmail.com>

* Add find_variants, tests, and documentation (#96)

* implementation of find_variants

* subset gt_donors in mode 'donor_match' and update nextflow schema

* clean find_variants

* clean subset_gt_donors (update env)

* clean donor_match

* fix donor_match

* add versions.yml (closes #87)

* remove own TODOs and add error message for HTO names (closes #75, closes #73)

* remove rename_genes_to_features (not used anymore)

* add an own anndata/mudata module and remove it from summary modules (closes #83)

* code and dependency cleanup of summary scripts

* code and dependency cleanup of create_anndata_mudata.py

* cleanup outdir

* add a comment

* closes #76

* add method overview output (closes #86)

* add warning method mentioned in #77

* add test for genetic mode (see #95)

* add tests for hashing and donor_match mode (see #95)

* update nf-core modules vireo and souporcell (closes #94)

* update input paths and add new tests to the github workflow (closes #95)

* remove duplicate parameter

* fix linting errors

* update snapshots for rescue and donor_match

* codespace snapshot default.nf.test

* fix create anndata/mudata issue (hto==null in genetic mode)

* add contributors

* change defaultBranch

* update contributor information

* add another contributor

* Add intro page

* add rescue and genetic mode to the documentation

* update samplesheet and add usage documentation

* restructure steps of the pipeline

* outputs for hashing, genetic and summary

* close details

* place output files below the headline

* update and describe donor_match and find_variant outputs

* push pipeline png

* add citation of the relevant tools and adjust outdir

* add seo's orcid

* move checking the params and samplesheet inputs to 'utils_nfcore_hadge_pipeline'

* move the validation of hto_names to utils_nfcore_hadge_pipeline with a function

* add ext.prefix of JOIN_RESULTS with regex

* add specific profiles to the tests and test vcf files

* add nft-vcf plugin

* update snapshot for rescue and donor match mode

* update profile name of donor_match

* update default snap with codespaces

* ingore cluster_genotypes.vcf, sort AnnData columns and mention test profiles in docs

* bash script to update all test snapshots

* update snapshots with codespaces

* update snapshot from test_genetic

* update ignore souporcell vcf

* add a seed for reproducible t-SNE and PCA

* add extract hashes env.yml

* nf-core pipelines lint --fix rocrate_readme_sync

* nf-core pipelines lint --dir .  --fix files_unchanged

* Revert "nf-core pipelines lint --dir .  --fix files_unchanged"

This reverts commit 9bdae0b.

* add comment for ignoring cluster_genotypes.vcf

* Samplesheet Input Requirements by Module

* small fix in docs

* remove rna_matrix from genetic mode as it is not needed

* simplify preprocessing channels

* clean up formatting

* remove barcodes from donor_match because it was already joined on barcodes.tsv in gene_summary.py

* simplify tuple nesting issue in ch_donor_match

* update docs

* test new snapshots

* update donor_match snapshot

* fix donor_match snapshot issue

* test new jpeg image

* remove the random seed from htodemux (snapshot differences are due to OS variations)

* update snapshot hashing

* update rescue snapshot

* update genetic snapshot

* only update *_tSNE_htodemux.jpeg snapshot and date (multiqc throws an error in codespaces)

* incorporate feedback

* nf-core pipelines lint --fix rocrate_readme_sync

* own line for each sentence

* Template update 3.5.2 (#103)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.5.2

* Update multiqc

* Test with larger nf-core runners (#101)

* add larger nf-core runners

* add zenodo and citation

* update citation

* add missing bam files to docs

* add random comment to trigger CI tests

* remove nf-core TODO's

* nf-core pipelines lint --fix rocrate_readme_sync

* use BibTeX for citations

* try another style for citations

* try another style for citations (pre-commit)

* change dropdown syntax to html

* adjust syntax

* add detailed docs for demuxlet

* update format

* update format 2

* Fix Numba caching error in container by setting NUMBA_CACHE_DIR

* setup test_full

* fix hash modules for full_test compatibility

* remove demuxem TODOs

* update tests and snapshots

* current test setup

* reduce computing/storage resources of test and test_full

* closes #107

* update  nextflow_schema.json

* update docs

* update test_full input

* nf-core pipelines lint --fix rocrate_readme_sync

* update snapshots

* remove outdir from config files

* update snapshots

* increase volume of self-hosted runners

* Remove aws.client.anonymous from global nextflow.config in concordance with template

* fine tune vireo configurations

* increase runner volume and update test snapshot with all tools

* set plp as path input

* use plp dir as input to fix aws error

* new try

* new try 2

* remove time constraint for souporcell

* increase time for souporcell

* update readme

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Important! Template update for nf-core/tools v4.0.2 (#109)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Fix nextflow linting errors

* Update multiQC module

* Fix nextflow lint warnings and remove redundant samplesheet input parameter.

Use explicit closure parameters, prefix unused bindings, and drop dead HTODEMUX channel assignments. PIPELINE_INITIALISATION reads params.input directly, so the take input is no longer needed.

* Use elvis operator for simplification

* Update container config files

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Update modules and use topic channels for version capture where applicable (#110)

* Update nf-core modules and fix bam_qc for new samtools API.

Refresh installed modules from nf-core/modules, adapt BAM_QC to the updated samtools view/sort inputs, and regenerate module patch files. Drop obsolete hasheddrops and multiseqdemux patches that no longer differ from upstream.

* Drop manual version mixing for nf-core topic outputs.

Rely on the versions topic collected before MultiQC instead of passing samtools, umitools, untar, csvtk, bff, and hasheddrops versions through subworkflows.

* Emit local module versions via topic and keep patched nf-core mixing.

Local modules publish to the versions topic so subworkflows no longer mix their outputs manually, while patched nf-core modules without topic support continue to use traditional ch_versions plumbing.

* Restore Harshil alignment on local module emit statements.

Re-align output emit columns after adding topic: versions during the topics migration.

* Fix compatibility issues

* Simplify hto null branching

* Fix hto branch logic while preserving dev hash channel behaviour.

Use a side-channel branch so ch_hto stays intact for joins, and mix is_null rows with EXTRACT_HASHES hashes rather than not_null hto paths.

* Update nf-test snapshots for topic-based version collection.

Refresh pipeline software versions YAML after local modules publish to the versions topic.

* Ignore non-deterministic tSNE plots in nf-test snapshots.

tSNE htodemux JPEGs vary across CI and local Docker runs; exclude them from stable-path md5 checks.

* Fix BFF meta map types and record updated module SHA. (#111)

Correct meta output types in bff/meta.yml after the nf-core module update and sync modules.json.

* Remove vulnerable PR-comment artifact pattern (#113)

Co-authored-by: nf-core-bot <core@nf-co.re>

* Important! Template update for nf-core/tools v4.0.3 (#112)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Template update for nf-core/tools version 4.0.3

* resolve missing merge conflicts

* fix linting errors

* update mulitqc

* nf-core pipelines lint --dir /workspaces/hadge  --fix container_configs

* Fix conda container config file linting

* Align linting workflow

* use stable_name in test

* use input instead of params.input

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>
Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Prepare Release PR (#114)

* bump-version

* update changelog

* resolve lint errors/warnings

* udpate snapshots

* update conda envs

* bumped SHA to fix similarity issue in CI

* update containers

* add missing versions

* exclude png for testing

* remove png and jpeg from shapshots

* update donor_match snapshot

* update test

* update remaining snapshots

* fix test_hashing

* fix test_genetic

* use || instead of |

* add meta.yml files to local modules using claude

* update resource usage of VIREO and SOUPORCELL

* restore apptainer in container directive of local modules

* restore apptainer in container directive of patched nf-core modules

* add stub and when to FILTER_BAM

* first fixes

* make vireo filtered variants optional for FIND_VARIANTS

* only run cellSNP when vireo or FIND_VARIANTS uses its output

* update  docs

* only allow one samplesheet row in donor_match mode

* add checkIfExists

* fine tune resources of vireo and souporcell

* add mqc plot

* update snapshots

---------

Co-authored-by: nictru <nictru32@gmail.com>
Co-authored-by: nf-core bot <infrastructure@nf-co.re>
Co-authored-by: Matthias Hörtenhuber <mashehu@users.noreply.github.com>
Co-authored-by: nf-core-bot <core@nf-co.re>
Skitionek added a commit that referenced this pull request Oct 29, 2026
* Skip fastqc template feature

* Fix .nf-core.yml

* Fix pipeline initialization

* Init top-level subworkflow structure

* Fix top-level structure bugs

* Strength samplesheet validation

* Add tool selection parameters

* Pass methods to subworkflows

* Update nf-core modules and subworkflows

* Add method condition blocks

* Install existing nf-core modules

* Simplify raw/filtered matrix treatment

* Improve hash/genetic channel structures

* Patch demuxem module

* Implement demuxem

* Add GMM-Demux

* Update genetic parameterization

* Add POPSCLE_DSCPILEUP

* Implement demuxlet

* Implement freemuxlet

* Implement cellsnp

* Implement vireo

* Implement BAM QC

* Use PLP for demuxlet

* Implement BAM subsetting to common variants

* Update outdir structure

* Update demuxem output definition

* Add all demuxem optional parameters

* Add GMM-demux optional params

* Add cellSNP optional parameters

* Add vireo optional params

* Add popscle-dscpileup optional parameters

* Add demuxlet/freemuxlet optional params

* Use github CI runners

* Add new hash demultiplexing modules (#66)

* hasheddrops inserted and they run in the pipeline

* insert local preprocessing module for htodemux and multiseq (not tested yet)

* adding htodemux and multiseq to the pipeline

* small improvements

* add parameters of modules also to the pipeline

* address

* run pre-commit for modules/local/preprocessing_for_htodemux_multiseq

* add htodemux visualization (not tested yet)

* visualization added to the pipeline (works locally)

* add visualization params to config files

* change strings to type boolean for htodemux visualization

* improve visualization

* change data types for preprocessing

* change data types of nf-core modules if possible

* Prettier

* Pre-commit

* Fix nf-core modules not matching remote

* Fix PULL_REQUEST_TEMPLATE issue based on this thread: https://nfcore.slack.com/archives/CQY2U5QU9/p1747475193145499

* Solve remaining prettier issues

* One more prettier fix :D

* Revert "Fix nf-core modules not matching remote"

This reverts commit ac3ed24.

* Make test profile work

* Patch modules

* Prettier

* Gitignore test intermediates

* Add pipeline test snapshot

* Make sure hash diffs are visible

* Ignore some more test snaps

* Enable new hash-based tools

* Set prefix to prevent file name collisions

* nft-ignore some outputs

---------

Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Aggregate hashing (#68)

* first steps

* current changes (not tested yet)

* save changes

* htodemux and multiseq in hash_summary.py (without mudata and anndata) and debugging of ch_hashing_summary

* solve summary input with join

* pipeline finally calls the script

* empty inputs solved

* add all inputs to main.nf (empty inputs still work)

* update htodemux in script

* script passed and produced results

* correct results

* solve incorrect merging

* add demuxem (runs but merging to the summary table is not working)

* remove the duplicate variable name

* add demuxem to summary (has more barcodes than htodemux and multiseq)

* add hasheddrops

* add gmmdemux to summary

* first steps to improve hasheddrops outputs

* fix hasheddrops output

* fix using unknown as value

* remove comments

* adding hashsolo doesn't work (cannot access the correct cell_hashing_columns)

* add bff summary function

* bff works

* join into barcodes from raw hto file to adress demuxem having more output barcodes that input barcodes

* pipeline pass with class Arguments

* move testing to Argument class

* restructure hash_summary.py

* mudata and anndata outputs are created

* hashsolo works

* replace manual str with args variable

* update hasheddrops

* update modules and improve linting

* Skip “prettier” failing – covered in #69

* clean up code 1

* clean code 2 (remove hard-coded parameters)

* clean code 3 (hash_sumary.py)

* update modules bff and hashsolo

* update schema

* remove properties created by nf-core pipelines schema build --no-prompts

* structure hashing paramters of nextflow.config

* update hasheddrops (removed one TODO in hash summary, hardcoding of mapping from id to hash)

* remove hardcoded hash_list

* remove linting error

* use left join for classification and small improvements for summary csv

* remove debugging outputs

* Fix demuxem MPLCONFIGDIR

* Add hash_summary environment variables

* Comment out arg printing in hash_summary

* Update demuxem patch file

* Prettier

* update snapshot

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Template update to 3.3.2 (#74)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Update multiQC module

* Fix vireo_cell_range issue

* Template update to 3.4.1 (#79)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Fix some linting problems

* Use github runners instead of self-hosted

Updated runner configuration to use 'ubuntu-latest' for nf-test jobs.

* Fix runner configuration in nf-test workflow

* Template update for nf-core/tools version 3.4.1

* Fix logo images

* Integration of genetic modules and donor matching (#78)

* save first steps

* join outputs

* gene_summary merges results for vireo, demuxlet and freemuxlet

* merge on Barcode

* all modules work for gene_summary but demuxlet has still empty values after merging

* prepare donor matching

* try understanding donor match

* debugging donor match

* donor_match produces something

* donor_match produces something

* donor_match seems to produce correct output files now, but it still needs to be tested with properly joined data

* pipeline is running with new input data and donor_match produces correct output

* first updates to souporcell

* update souporcell

* final versions.yml

* save anndata/mudata, output before joining and left join to rna barcodes

* couln't fix demuxEM error, opened an issue at developer's github

* gene/hash_summary: save mudata/anndata, print results overview fuction and clean code

* include all methods in donor_match and update bff() in hash_summary

* improve the readability of hadge.nf

* use fasta from iGenomes for souporcell but not fully tested (see #77)

* remove prints and linting errors

* update snapshot

* Merge origin/dev

* update souporcell

* Fix image files

* update snapshot

* update snapshot 2

* fasta updates

* update snapshot with 3 modules each (hash/gene)

* update fasta description

* update snapshot

* replace  set with list to keep the same order

* snapshot from codespace

* add souporcell outputs to .nftignore

* Add trailing newline to .nftignore

* fix end of file

* add freemuxlet/*/*.clust1.vcf.gz to .nftignore

* Incorporate review comments

* update snapshot

---------

Co-authored-by: nictru <nictru32@gmail.com>

* Add find_variants, tests, and documentation (#96)

* implementation of find_variants

* subset gt_donors in mode 'donor_match' and update nextflow schema

* clean find_variants

* clean subset_gt_donors (update env)

* clean donor_match

* fix donor_match

* add versions.yml (closes #87)

* remove own TODOs and add error message for HTO names (closes #75, closes #73)

* remove rename_genes_to_features (not used anymore)

* add an own anndata/mudata module and remove it from summary modules (closes #83)

* code and dependency cleanup of summary scripts

* code and dependency cleanup of create_anndata_mudata.py

* cleanup outdir

* add a comment

* closes #76

* add method overview output (closes #86)

* add warning method mentioned in #77

* add test for genetic mode (see #95)

* add tests for hashing and donor_match mode (see #95)

* update nf-core modules vireo and souporcell (closes #94)

* update input paths and add new tests to the github workflow (closes #95)

* remove duplicate parameter

* fix linting errors

* update snapshots for rescue and donor_match

* codespace snapshot default.nf.test

* fix create anndata/mudata issue (hto==null in genetic mode)

* add contributors

* change defaultBranch

* update contributor information

* add another contributor

* Add intro page

* add rescue and genetic mode to the documentation

* update samplesheet and add usage documentation

* restructure steps of the pipeline

* outputs for hashing, genetic and summary

* close details

* place output files below the headline

* update and describe donor_match and find_variant outputs

* push pipeline png

* add citation of the relevant tools and adjust outdir

* add seo's orcid

* move checking the params and samplesheet inputs to 'utils_nfcore_hadge_pipeline'

* move the validation of hto_names to utils_nfcore_hadge_pipeline with a function

* add ext.prefix of JOIN_RESULTS with regex

* add specific profiles to the tests and test vcf files

* add nft-vcf plugin

* update snapshot for rescue and donor match mode

* update profile name of donor_match

* update default snap with codespaces

* ingore cluster_genotypes.vcf, sort AnnData columns and mention test profiles in docs

* bash script to update all test snapshots

* update snapshots with codespaces

* update snapshot from test_genetic

* update ignore souporcell vcf

* add a seed for reproducible t-SNE and PCA

* add extract hashes env.yml

* nf-core pipelines lint --fix rocrate_readme_sync

* nf-core pipelines lint --dir .  --fix files_unchanged

* Revert "nf-core pipelines lint --dir .  --fix files_unchanged"

This reverts commit 9bdae0b.

* add comment for ignoring cluster_genotypes.vcf

* Samplesheet Input Requirements by Module

* small fix in docs

* remove rna_matrix from genetic mode as it is not needed

* simplify preprocessing channels

* clean up formatting

* remove barcodes from donor_match because it was already joined on barcodes.tsv in gene_summary.py

* simplify tuple nesting issue in ch_donor_match

* update docs

* test new snapshots

* update donor_match snapshot

* fix donor_match snapshot issue

* test new jpeg image

* remove the random seed from htodemux (snapshot differences are due to OS variations)

* update snapshot hashing

* update rescue snapshot

* update genetic snapshot

* only update *_tSNE_htodemux.jpeg snapshot and date (multiqc throws an error in codespaces)

* incorporate feedback

* nf-core pipelines lint --fix rocrate_readme_sync

* own line for each sentence

* Template update 3.5.2 (#103)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.5.2

* Update multiqc

* Test with larger nf-core runners (#101)

* add larger nf-core runners

* add zenodo and citation

* update citation

* add missing bam files to docs

* add random comment to trigger CI tests

* remove nf-core TODO's

* nf-core pipelines lint --fix rocrate_readme_sync

* use BibTeX for citations

* try another style for citations

* try another style for citations (pre-commit)

* change dropdown syntax to html

* adjust syntax

* add detailed docs for demuxlet

* update format

* update format 2

* Fix Numba caching error in container by setting NUMBA_CACHE_DIR

* setup test_full

* fix hash modules for full_test compatibility

* remove demuxem TODOs

* update tests and snapshots

* current test setup

* reduce computing/storage resources of test and test_full

* closes #107

* update  nextflow_schema.json

* update docs

* update test_full input

* nf-core pipelines lint --fix rocrate_readme_sync

* update snapshots

* remove outdir from config files

* update snapshots

* increase volume of self-hosted runners

* Remove aws.client.anonymous from global nextflow.config in concordance with template

* fine tune vireo configurations

* increase runner volume and update test snapshot with all tools

* set plp as path input

* use plp dir as input to fix aws error

* new try

* new try 2

* remove time constraint for souporcell

* increase time for souporcell

* update readme

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Important! Template update for nf-core/tools v4.0.2 (#109)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Fix nextflow linting errors

* Update multiQC module

* Fix nextflow lint warnings and remove redundant samplesheet input parameter.

Use explicit closure parameters, prefix unused bindings, and drop dead HTODEMUX channel assignments. PIPELINE_INITIALISATION reads params.input directly, so the take input is no longer needed.

* Use elvis operator for simplification

* Update container config files

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Update modules and use topic channels for version capture where applicable (#110)

* Update nf-core modules and fix bam_qc for new samtools API.

Refresh installed modules from nf-core/modules, adapt BAM_QC to the updated samtools view/sort inputs, and regenerate module patch files. Drop obsolete hasheddrops and multiseqdemux patches that no longer differ from upstream.

* Drop manual version mixing for nf-core topic outputs.

Rely on the versions topic collected before MultiQC instead of passing samtools, umitools, untar, csvtk, bff, and hasheddrops versions through subworkflows.

* Emit local module versions via topic and keep patched nf-core mixing.

Local modules publish to the versions topic so subworkflows no longer mix their outputs manually, while patched nf-core modules without topic support continue to use traditional ch_versions plumbing.

* Restore Harshil alignment on local module emit statements.

Re-align output emit columns after adding topic: versions during the topics migration.

* Fix compatibility issues

* Simplify hto null branching

* Fix hto branch logic while preserving dev hash channel behaviour.

Use a side-channel branch so ch_hto stays intact for joins, and mix is_null rows with EXTRACT_HASHES hashes rather than not_null hto paths.

* Update nf-test snapshots for topic-based version collection.

Refresh pipeline software versions YAML after local modules publish to the versions topic.

* Ignore non-deterministic tSNE plots in nf-test snapshots.

tSNE htodemux JPEGs vary across CI and local Docker runs; exclude them from stable-path md5 checks.

* Fix BFF meta map types and record updated module SHA. (#111)

Correct meta output types in bff/meta.yml after the nf-core module update and sync modules.json.

* Remove vulnerable PR-comment artifact pattern (#113)

Co-authored-by: nf-core-bot <core@nf-co.re>

* Important! Template update for nf-core/tools v4.0.3 (#112)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Template update for nf-core/tools version 4.0.3

* resolve missing merge conflicts

* fix linting errors

* update mulitqc

* nf-core pipelines lint --dir /workspaces/hadge  --fix container_configs

* Fix conda container config file linting

* Align linting workflow

* use stable_name in test

* use input instead of params.input

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>
Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Prepare Release PR (#114)

* bump-version

* update changelog

* resolve lint errors/warnings

* udpate snapshots

* update conda envs

* bumped SHA to fix similarity issue in CI

* update containers

* add missing versions

* exclude png for testing

* remove png and jpeg from shapshots

* update donor_match snapshot

* update test

* update remaining snapshots

* fix test_hashing

* fix test_genetic

* use || instead of |

* add meta.yml files to local modules using claude

* update resource usage of VIREO and SOUPORCELL

* restore apptainer in container directive of local modules

* restore apptainer in container directive of patched nf-core modules

* add stub and when to FILTER_BAM

* first fixes

* make vireo filtered variants optional for FIND_VARIANTS

* only run cellSNP when vireo or FIND_VARIANTS uses its output

* update  docs

* only allow one samplesheet row in donor_match mode

* add checkIfExists

* fine tune resources of vireo and souporcell

* add mqc plot

* update snapshots

---------

Co-authored-by: nictru <nictru32@gmail.com>
Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>
Co-authored-by: nf-core bot <infrastructure@nf-co.re>
Co-authored-by: Matthias Hörtenhuber <mashehu@users.noreply.github.com>
Co-authored-by: nf-core-bot <core@nf-co.re>
Skitionek added a commit that referenced this pull request Oct 30, 2026
* Skip fastqc template feature

* Fix .nf-core.yml

* Fix pipeline initialization

* Init top-level subworkflow structure

* Fix top-level structure bugs

* Strength samplesheet validation

* Add tool selection parameters

* Pass methods to subworkflows

* Update nf-core modules and subworkflows

* Add method condition blocks

* Install existing nf-core modules

* Simplify raw/filtered matrix treatment

* Improve hash/genetic channel structures

* Patch demuxem module

* Implement demuxem

* Add GMM-Demux

* Update genetic parameterization

* Add POPSCLE_DSCPILEUP

* Implement demuxlet

* Implement freemuxlet

* Implement cellsnp

* Implement vireo

* Implement BAM QC

* Use PLP for demuxlet

* Implement BAM subsetting to common variants

* Update outdir structure

* Update demuxem output definition

* Add all demuxem optional parameters

* Add GMM-demux optional params

* Add cellSNP optional parameters

* Add vireo optional params

* Add popscle-dscpileup optional parameters

* Add demuxlet/freemuxlet optional params

* Use github CI runners

* Add new hash demultiplexing modules (#66)

* hasheddrops inserted and they run in the pipeline

* insert local preprocessing module for htodemux and multiseq (not tested yet)

* adding htodemux and multiseq to the pipeline

* small improvements

* add parameters of modules also to the pipeline

* address

* run pre-commit for modules/local/preprocessing_for_htodemux_multiseq

* add htodemux visualization (not tested yet)

* visualization added to the pipeline (works locally)

* add visualization params to config files

* change strings to type boolean for htodemux visualization

* improve visualization

* change data types for preprocessing

* change data types of nf-core modules if possible

* Prettier

* Pre-commit

* Fix nf-core modules not matching remote

* Fix PULL_REQUEST_TEMPLATE issue based on this thread: https://nfcore.slack.com/archives/CQY2U5QU9/p1747475193145499

* Solve remaining prettier issues

* One more prettier fix :D

* Revert "Fix nf-core modules not matching remote"

This reverts commit ac3ed24.

* Make test profile work

* Patch modules

* Prettier

* Gitignore test intermediates

* Add pipeline test snapshot

* Make sure hash diffs are visible

* Ignore some more test snaps

* Enable new hash-based tools

* Set prefix to prevent file name collisions

* nft-ignore some outputs

---------

Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Aggregate hashing (#68)

* first steps

* current changes (not tested yet)

* save changes

* htodemux and multiseq in hash_summary.py (without mudata and anndata) and debugging of ch_hashing_summary

* solve summary input with join

* pipeline finally calls the script

* empty inputs solved

* add all inputs to main.nf (empty inputs still work)

* update htodemux in script

* script passed and produced results

* correct results

* solve incorrect merging

* add demuxem (runs but merging to the summary table is not working)

* remove the duplicate variable name

* add demuxem to summary (has more barcodes than htodemux and multiseq)

* add hasheddrops

* add gmmdemux to summary

* first steps to improve hasheddrops outputs

* fix hasheddrops output

* fix using unknown as value

* remove comments

* adding hashsolo doesn't work (cannot access the correct cell_hashing_columns)

* add bff summary function

* bff works

* join into barcodes from raw hto file to adress demuxem having more output barcodes that input barcodes

* pipeline pass with class Arguments

* move testing to Argument class

* restructure hash_summary.py

* mudata and anndata outputs are created

* hashsolo works

* replace manual str with args variable

* update hasheddrops

* update modules and improve linting

* Skip “prettier” failing – covered in #69

* clean up code 1

* clean code 2 (remove hard-coded parameters)

* clean code 3 (hash_sumary.py)

* update modules bff and hashsolo

* update schema

* remove properties created by nf-core pipelines schema build --no-prompts

* structure hashing paramters of nextflow.config

* update hasheddrops (removed one TODO in hash summary, hardcoding of mapping from id to hash)

* remove hardcoded hash_list

* remove linting error

* use left join for classification and small improvements for summary csv

* remove debugging outputs

* Fix demuxem MPLCONFIGDIR

* Add hash_summary environment variables

* Comment out arg printing in hash_summary

* Update demuxem patch file

* Prettier

* update snapshot

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Template update to 3.3.2 (#74)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Update multiQC module

* Fix vireo_cell_range issue

* Template update to 3.4.1 (#79)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Fix some linting problems

* Use github runners instead of self-hosted

Updated runner configuration to use 'ubuntu-latest' for nf-test jobs.

* Fix runner configuration in nf-test workflow

* Template update for nf-core/tools version 3.4.1

* Fix logo images

* Integration of genetic modules and donor matching (#78)

* save first steps

* join outputs

* gene_summary merges results for vireo, demuxlet and freemuxlet

* merge on Barcode

* all modules work for gene_summary but demuxlet has still empty values after merging

* prepare donor matching

* try understanding donor match

* debugging donor match

* donor_match produces something

* donor_match produces something

* donor_match seems to produce correct output files now, but it still needs to be tested with properly joined data

* pipeline is running with new input data and donor_match produces correct output

* first updates to souporcell

* update souporcell

* final versions.yml

* save anndata/mudata, output before joining and left join to rna barcodes

* couln't fix demuxEM error, opened an issue at developer's github

* gene/hash_summary: save mudata/anndata, print results overview fuction and clean code

* include all methods in donor_match and update bff() in hash_summary

* improve the readability of hadge.nf

* use fasta from iGenomes for souporcell but not fully tested (see #77)

* remove prints and linting errors

* update snapshot

* Merge origin/dev

* update souporcell

* Fix image files

* update snapshot

* update snapshot 2

* fasta updates

* update snapshot with 3 modules each (hash/gene)

* update fasta description

* update snapshot

* replace  set with list to keep the same order

* snapshot from codespace

* add souporcell outputs to .nftignore

* Add trailing newline to .nftignore

* fix end of file

* add freemuxlet/*/*.clust1.vcf.gz to .nftignore

* Incorporate review comments

* update snapshot

---------

Co-authored-by: nictru <nictru32@gmail.com>

* Add find_variants, tests, and documentation (#96)

* implementation of find_variants

* subset gt_donors in mode 'donor_match' and update nextflow schema

* clean find_variants

* clean subset_gt_donors (update env)

* clean donor_match

* fix donor_match

* add versions.yml (closes #87)

* remove own TODOs and add error message for HTO names (closes #75, closes #73)

* remove rename_genes_to_features (not used anymore)

* add an own anndata/mudata module and remove it from summary modules (closes #83)

* code and dependency cleanup of summary scripts

* code and dependency cleanup of create_anndata_mudata.py

* cleanup outdir

* add a comment

* closes #76

* add method overview output (closes #86)

* add warning method mentioned in #77

* add test for genetic mode (see #95)

* add tests for hashing and donor_match mode (see #95)

* update nf-core modules vireo and souporcell (closes #94)

* update input paths and add new tests to the github workflow (closes #95)

* remove duplicate parameter

* fix linting errors

* update snapshots for rescue and donor_match

* codespace snapshot default.nf.test

* fix create anndata/mudata issue (hto==null in genetic mode)

* add contributors

* change defaultBranch

* update contributor information

* add another contributor

* Add intro page

* add rescue and genetic mode to the documentation

* update samplesheet and add usage documentation

* restructure steps of the pipeline

* outputs for hashing, genetic and summary

* close details

* place output files below the headline

* update and describe donor_match and find_variant outputs

* push pipeline png

* add citation of the relevant tools and adjust outdir

* add seo's orcid

* move checking the params and samplesheet inputs to 'utils_nfcore_hadge_pipeline'

* move the validation of hto_names to utils_nfcore_hadge_pipeline with a function

* add ext.prefix of JOIN_RESULTS with regex

* add specific profiles to the tests and test vcf files

* add nft-vcf plugin

* update snapshot for rescue and donor match mode

* update profile name of donor_match

* update default snap with codespaces

* ingore cluster_genotypes.vcf, sort AnnData columns and mention test profiles in docs

* bash script to update all test snapshots

* update snapshots with codespaces

* update snapshot from test_genetic

* update ignore souporcell vcf

* add a seed for reproducible t-SNE and PCA

* add extract hashes env.yml

* nf-core pipelines lint --fix rocrate_readme_sync

* nf-core pipelines lint --dir .  --fix files_unchanged

* Revert "nf-core pipelines lint --dir .  --fix files_unchanged"

This reverts commit 9bdae0b.

* add comment for ignoring cluster_genotypes.vcf

* Samplesheet Input Requirements by Module

* small fix in docs

* remove rna_matrix from genetic mode as it is not needed

* simplify preprocessing channels

* clean up formatting

* remove barcodes from donor_match because it was already joined on barcodes.tsv in gene_summary.py

* simplify tuple nesting issue in ch_donor_match

* update docs

* test new snapshots

* update donor_match snapshot

* fix donor_match snapshot issue

* test new jpeg image

* remove the random seed from htodemux (snapshot differences are due to OS variations)

* update snapshot hashing

* update rescue snapshot

* update genetic snapshot

* only update *_tSNE_htodemux.jpeg snapshot and date (multiqc throws an error in codespaces)

* incorporate feedback

* nf-core pipelines lint --fix rocrate_readme_sync

* own line for each sentence

* Template update 3.5.2 (#103)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.5.2

* Update multiqc

* Test with larger nf-core runners (#101)

* add larger nf-core runners

* add zenodo and citation

* update citation

* add missing bam files to docs

* add random comment to trigger CI tests

* remove nf-core TODO's

* nf-core pipelines lint --fix rocrate_readme_sync

* use BibTeX for citations

* try another style for citations

* try another style for citations (pre-commit)

* change dropdown syntax to html

* adjust syntax

* add detailed docs for demuxlet

* update format

* update format 2

* Fix Numba caching error in container by setting NUMBA_CACHE_DIR

* setup test_full

* fix hash modules for full_test compatibility

* remove demuxem TODOs

* update tests and snapshots

* current test setup

* reduce computing/storage resources of test and test_full

* closes #107

* update  nextflow_schema.json

* update docs

* update test_full input

* nf-core pipelines lint --fix rocrate_readme_sync

* update snapshots

* remove outdir from config files

* update snapshots

* increase volume of self-hosted runners

* Remove aws.client.anonymous from global nextflow.config in concordance with template

* fine tune vireo configurations

* increase runner volume and update test snapshot with all tools

* set plp as path input

* use plp dir as input to fix aws error

* new try

* new try 2

* remove time constraint for souporcell

* increase time for souporcell

* update readme

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Important! Template update for nf-core/tools v4.0.2 (#109)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Fix nextflow linting errors

* Update multiQC module

* Fix nextflow lint warnings and remove redundant samplesheet input parameter.

Use explicit closure parameters, prefix unused bindings, and drop dead HTODEMUX channel assignments. PIPELINE_INITIALISATION reads params.input directly, so the take input is no longer needed.

* Use elvis operator for simplification

* Update container config files

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Update modules and use topic channels for version capture where applicable (#110)

* Update nf-core modules and fix bam_qc for new samtools API.

Refresh installed modules from nf-core/modules, adapt BAM_QC to the updated samtools view/sort inputs, and regenerate module patch files. Drop obsolete hasheddrops and multiseqdemux patches that no longer differ from upstream.

* Drop manual version mixing for nf-core topic outputs.

Rely on the versions topic collected before MultiQC instead of passing samtools, umitools, untar, csvtk, bff, and hasheddrops versions through subworkflows.

* Emit local module versions via topic and keep patched nf-core mixing.

Local modules publish to the versions topic so subworkflows no longer mix their outputs manually, while patched nf-core modules without topic support continue to use traditional ch_versions plumbing.

* Restore Harshil alignment on local module emit statements.

Re-align output emit columns after adding topic: versions during the topics migration.

* Fix compatibility issues

* Simplify hto null branching

* Fix hto branch logic while preserving dev hash channel behaviour.

Use a side-channel branch so ch_hto stays intact for joins, and mix is_null rows with EXTRACT_HASHES hashes rather than not_null hto paths.

* Update nf-test snapshots for topic-based version collection.

Refresh pipeline software versions YAML after local modules publish to the versions topic.

* Ignore non-deterministic tSNE plots in nf-test snapshots.

tSNE htodemux JPEGs vary across CI and local Docker runs; exclude them from stable-path md5 checks.

* Fix BFF meta map types and record updated module SHA. (#111)

Correct meta output types in bff/meta.yml after the nf-core module update and sync modules.json.

* Remove vulnerable PR-comment artifact pattern (#113)

Co-authored-by: nf-core-bot <core@nf-co.re>

* Important! Template update for nf-core/tools v4.0.3 (#112)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Template update for nf-core/tools version 4.0.3

* resolve missing merge conflicts

* fix linting errors

* update mulitqc

* nf-core pipelines lint --dir /workspaces/hadge  --fix container_configs

* Fix conda container config file linting

* Align linting workflow

* use stable_name in test

* use input instead of params.input

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>
Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Prepare Release PR (#114)

* bump-version

* update changelog

* resolve lint errors/warnings

* udpate snapshots

* update conda envs

* bumped SHA to fix similarity issue in CI

* update containers

* add missing versions

* exclude png for testing

* remove png and jpeg from shapshots

* update donor_match snapshot

* update test

* update remaining snapshots

* fix test_hashing

* fix test_genetic

* use || instead of |

* add meta.yml files to local modules using claude

* update resource usage of VIREO and SOUPORCELL

* restore apptainer in container directive of local modules

* restore apptainer in container directive of patched nf-core modules

* add stub and when to FILTER_BAM

* first fixes

* make vireo filtered variants optional for FIND_VARIANTS

* only run cellSNP when vireo or FIND_VARIANTS uses its output

* update  docs

* only allow one samplesheet row in donor_match mode

* add checkIfExists

* fine tune resources of vireo and souporcell

* add mqc plot

* update snapshots

* feat: migrate to the nf-core/tools 4.1.0 template with nf-migrate

Applies the migration chain from dev's template version to 4.1.0. The CHANGELOG.md and conf/modules.config changes did not apply and follow as separate commits.

Coding/Writing assisted with Claude Sonnet 5.5

Prompts:
- Resolve nf-core/hadge PR 116 (template 4.1.0) with nf-migrate and open two stacked draft PRs

* style: apply nf-core pipelines lint --fix

* docs(changelog): add the 4.1.0 template's unreleased entry

Coding/Writing assisted with Claude Sonnet 5.5

Prompts:
- Resolve nf-core/hadge PR 116 (template 4.1.0) with nf-migrate and open two stacked draft PRs

* fix(config): drop saveAs from the default publishDir like template 4.1.0

Coding/Writing assisted with Claude Sonnet 5.5

Prompts:
- Resolve nf-core/hadge PR 116 (template 4.1.0) with nf-migrate and open two stacked draft PRs

* chore(containers): regenerate the conda lock container configs

Coding/Writing assisted with Claude Sonnet 5.5

Prompts:
- Resolve nf-core/hadge PR 116 (template 4.1.0) with nf-migrate and open two stacked draft PRs

* chore(ro-crate): sync the RO-Crate description with the README

Coding/Writing assisted with Claude Sonnet 5.5

Prompts:
- Resolve nf-core/hadge PR 116 (template 4.1.0) with nf-migrate and open two stacked draft PRs

* fix(subworkflows): update utils_nfschema_plugin to the version the 4.1.0 template calls

The template passes ten inputs (help_full was added); the installed version takes nine, so the pipeline failed to initialise.

Coding/Writing assisted with Claude Sonnet 5.5

Prompts:
- Resolve nf-core/hadge PR 116 (template 4.1.0) with nf-migrate and open two stacked draft PRs
- Fix the CI failure on the migration PR

* fix(subworkflows): pin utils_nfschema_plugin to the 4.1.0 template's version

The previous update installed a newer version that needs a newer nf-schema plugin than the one the template pins (2.5.1).

Coding/Writing assisted with Claude Sonnet 5.5

Prompts:
- Resolve nf-core/hadge PR 116 (template 4.1.0) with nf-migrate and open two stacked draft PRs
- Fix the CI failure on the migration PR

* fix(schema): point schema $id URLs at the main branch

nf-migrate reintroduced the template's `master` default branch in the `$id` of nextflow_schema.json and assets/schema_input.json; this repo's default branch is `main` (manifest.defaultBranch).

Coding/Writing assisted with Claude Opus 5.5

Prompts:
- checkout #121 and fix comment
- just push

---------

Co-authored-by: nictru <nictru32@gmail.com>
Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>
Co-authored-by: nf-core bot <infrastructure@nf-co.re>
Co-authored-by: Matthias Hörtenhuber <mashehu@users.noreply.github.com>
Co-authored-by: nf-core-bot <core@nf-co.re>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants