Skip to content

Tags

Tags

1.0.0

Toggle 1.0.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Release PR (#115)

* Skip fastqc template feature

* Fix .nf-core.yml

* Fix pipeline initialization

* Init top-level subworkflow structure

* Fix top-level structure bugs

* Strength samplesheet validation

* Add tool selection parameters

* Pass methods to subworkflows

* Update nf-core modules and subworkflows

* Add method condition blocks

* Install existing nf-core modules

* Simplify raw/filtered matrix treatment

* Improve hash/genetic channel structures

* Patch demuxem module

* Implement demuxem

* Add GMM-Demux

* Update genetic parameterization

* Add POPSCLE_DSCPILEUP

* Implement demuxlet

* Implement freemuxlet

* Implement cellsnp

* Implement vireo

* Implement BAM QC

* Use PLP for demuxlet

* Implement BAM subsetting to common variants

* Update outdir structure

* Update demuxem output definition

* Add all demuxem optional parameters

* Add GMM-demux optional params

* Add cellSNP optional parameters

* Add vireo optional params

* Add popscle-dscpileup optional parameters

* Add demuxlet/freemuxlet optional params

* Use github CI runners

* Add new hash demultiplexing modules (#66)

* hasheddrops inserted and they run in the pipeline

* insert local preprocessing module for htodemux and multiseq (not tested yet)

* adding htodemux and multiseq to the pipeline

* small improvements

* add parameters of modules also to the pipeline

* address

* run pre-commit for modules/local/preprocessing_for_htodemux_multiseq

* add htodemux visualization (not tested yet)

* visualization added to the pipeline (works locally)

* add visualization params to config files

* change strings to type boolean for htodemux visualization

* improve visualization

* change data types for preprocessing

* change data types of nf-core modules if possible

* Prettier

* Pre-commit

* Fix nf-core modules not matching remote

* Fix PULL_REQUEST_TEMPLATE issue based on this thread: https://nfcore.slack.com/archives/CQY2U5QU9/p1747475193145499

* Solve remaining prettier issues

* One more prettier fix :D

* Revert "Fix nf-core modules not matching remote"

This reverts commit ac3ed24.

* Make test profile work

* Patch modules

* Prettier

* Gitignore test intermediates

* Add pipeline test snapshot

* Make sure hash diffs are visible

* Ignore some more test snaps

* Enable new hash-based tools

* Set prefix to prevent file name collisions

* nft-ignore some outputs

---------

Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Aggregate hashing (#68)

* first steps

* current changes (not tested yet)

* save changes

* htodemux and multiseq in hash_summary.py (without mudata and anndata) and debugging of ch_hashing_summary

* solve summary input with join

* pipeline finally calls the script

* empty inputs solved

* add all inputs to main.nf (empty inputs still work)

* update htodemux in script

* script passed and produced results

* correct results

* solve incorrect merging

* add demuxem (runs but merging to the summary table is not working)

* remove the duplicate variable name

* add demuxem to summary (has more barcodes than htodemux and multiseq)

* add hasheddrops

* add gmmdemux to summary

* first steps to improve hasheddrops outputs

* fix hasheddrops output

* fix using unknown as value

* remove comments

* adding hashsolo doesn't work (cannot access the correct cell_hashing_columns)

* add bff summary function

* bff works

* join into barcodes from raw hto file to adress demuxem having more output barcodes that input barcodes

* pipeline pass with class Arguments

* move testing to Argument class

* restructure hash_summary.py

* mudata and anndata outputs are created

* hashsolo works

* replace manual str with args variable

* update hasheddrops

* update modules and improve linting

* Skip “prettier” failing – covered in #69

* clean up code 1

* clean code 2 (remove hard-coded parameters)

* clean code 3 (hash_sumary.py)

* update modules bff and hashsolo

* update schema

* remove properties created by nf-core pipelines schema build --no-prompts

* structure hashing paramters of nextflow.config

* update hasheddrops (removed one TODO in hash summary, hardcoding of mapping from id to hash)

* remove hardcoded hash_list

* remove linting error

* use left join for classification and small improvements for summary csv

* remove debugging outputs

* Fix demuxem MPLCONFIGDIR

* Add hash_summary environment variables

* Comment out arg printing in hash_summary

* Update demuxem patch file

* Prettier

* update snapshot

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Template update to 3.3.2 (#74)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Update multiQC module

* Fix vireo_cell_range issue

* Template update to 3.4.1 (#79)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Fix some linting problems

* Use github runners instead of self-hosted

Updated runner configuration to use 'ubuntu-latest' for nf-test jobs.

* Fix runner configuration in nf-test workflow

* Template update for nf-core/tools version 3.4.1

* Fix logo images

* Integration of genetic modules and donor matching (#78)

* save first steps

* join outputs

* gene_summary merges results for vireo, demuxlet and freemuxlet

* merge on Barcode

* all modules work for gene_summary but demuxlet has still empty values after merging

* prepare donor matching

* try understanding donor match

* debugging donor match

* donor_match produces something

* donor_match produces something

* donor_match seems to produce correct output files now, but it still needs to be tested with properly joined data

* pipeline is running with new input data and donor_match produces correct output

* first updates to souporcell

* update souporcell

* final versions.yml

* save anndata/mudata, output before joining and left join to rna barcodes

* couln't fix demuxEM error, opened an issue at developer's github

* gene/hash_summary: save mudata/anndata, print results overview fuction and clean code

* include all methods in donor_match and update bff() in hash_summary

* improve the readability of hadge.nf

* use fasta from iGenomes for souporcell but not fully tested (see #77)

* remove prints and linting errors

* update snapshot

* Merge origin/dev

* update souporcell

* Fix image files

* update snapshot

* update snapshot 2

* fasta updates

* update snapshot with 3 modules each (hash/gene)

* update fasta description

* update snapshot

* replace  set with list to keep the same order

* snapshot from codespace

* add souporcell outputs to .nftignore

* Add trailing newline to .nftignore

* fix end of file

* add freemuxlet/*/*.clust1.vcf.gz to .nftignore

* Incorporate review comments

* update snapshot

---------

Co-authored-by: nictru <nictru32@gmail.com>

* Add find_variants, tests, and documentation (#96)

* implementation of find_variants

* subset gt_donors in mode 'donor_match' and update nextflow schema

* clean find_variants

* clean subset_gt_donors (update env)

* clean donor_match

* fix donor_match

* add versions.yml (closes #87)

* remove own TODOs and add error message for HTO names (closes #75, closes #73)

* remove rename_genes_to_features (not used anymore)

* add an own anndata/mudata module and remove it from summary modules (closes #83)

* code and dependency cleanup of summary scripts

* code and dependency cleanup of create_anndata_mudata.py

* cleanup outdir

* add a comment

* closes #76

* add method overview output (closes #86)

* add warning method mentioned in #77

* add test for genetic mode (see #95)

* add tests for hashing and donor_match mode (see #95)

* update nf-core modules vireo and souporcell (closes #94)

* update input paths and add new tests to the github workflow (closes #95)

* remove duplicate parameter

* fix linting errors

* update snapshots for rescue and donor_match

* codespace snapshot default.nf.test

* fix create anndata/mudata issue (hto==null in genetic mode)

* add contributors

* change defaultBranch

* update contributor information

* add another contributor

* Add intro page

* add rescue and genetic mode to the documentation

* update samplesheet and add usage documentation

* restructure steps of the pipeline

* outputs for hashing, genetic and summary

* close details

* place output files below the headline

* update and describe donor_match and find_variant outputs

* push pipeline png

* add citation of the relevant tools and adjust outdir

* add seo's orcid

* move checking the params and samplesheet inputs to 'utils_nfcore_hadge_pipeline'

* move the validation of hto_names to utils_nfcore_hadge_pipeline with a function

* add ext.prefix of JOIN_RESULTS with regex

* add specific profiles to the tests and test vcf files

* add nft-vcf plugin

* update snapshot for rescue and donor match mode

* update profile name of donor_match

* update default snap with codespaces

* ingore cluster_genotypes.vcf, sort AnnData columns and mention test profiles in docs

* bash script to update all test snapshots

* update snapshots with codespaces

* update snapshot from test_genetic

* update ignore souporcell vcf

* add a seed for reproducible t-SNE and PCA

* add extract hashes env.yml

* nf-core pipelines lint --fix rocrate_readme_sync

* nf-core pipelines lint --dir .  --fix files_unchanged

* Revert "nf-core pipelines lint --dir .  --fix files_unchanged"

This reverts commit 9bdae0b.

* add comment for ignoring cluster_genotypes.vcf

* Samplesheet Input Requirements by Module

* small fix in docs

* remove rna_matrix from genetic mode as it is not needed

* simplify preprocessing channels

* clean up formatting

* remove barcodes from donor_match because it was already joined on barcodes.tsv in gene_summary.py

* simplify tuple nesting issue in ch_donor_match

* update docs

* test new snapshots

* update donor_match snapshot

* fix donor_match snapshot issue

* test new jpeg image

* remove the random seed from htodemux (snapshot differences are due to OS variations)

* update snapshot hashing

* update rescue snapshot

* update genetic snapshot

* only update *_tSNE_htodemux.jpeg snapshot and date (multiqc throws an error in codespaces)

* incorporate feedback

* nf-core pipelines lint --fix rocrate_readme_sync

* own line for each sentence

* Template update 3.5.2 (#103)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.5.2

* Update multiqc

* Test with larger nf-core runners (#101)

* add larger nf-core runners

* add zenodo and citation

* update citation

* add missing bam files to docs

* add random comment to trigger CI tests

* remove nf-core TODO's

* nf-core pipelines lint --fix rocrate_readme_sync

* use BibTeX for citations

* try another style for citations

* try another style for citations (pre-commit)

* change dropdown syntax to html

* adjust syntax

* add detailed docs for demuxlet

* update format

* update format 2

* Fix Numba caching error in container by setting NUMBA_CACHE_DIR

* setup test_full

* fix hash modules for full_test compatibility

* remove demuxem TODOs

* update tests and snapshots

* current test setup

* reduce computing/storage resources of test and test_full

* closes #107

* update  nextflow_schema.json

* update docs

* update test_full input

* nf-core pipelines lint --fix rocrate_readme_sync

* update snapshots

* remove outdir from config files

* update snapshots

* increase volume of self-hosted runners

* Remove aws.client.anonymous from global nextflow.config in concordance with template

* fine tune vireo configurations

* increase runner volume and update test snapshot with all tools

* set plp as path input

* use plp dir as input to fix aws error

* new try

* new try 2

* remove time constraint for souporcell

* increase time for souporcell

* update readme

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Important! Template update for nf-core/tools v4.0.2 (#109)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Fix nextflow linting errors

* Update multiQC module

* Fix nextflow lint warnings and remove redundant samplesheet input parameter.

Use explicit closure parameters, prefix unused bindings, and drop dead HTODEMUX channel assignments. PIPELINE_INITIALISATION reads params.input directly, so the take input is no longer needed.

* Use elvis operator for simplification

* Update container config files

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>

* Update modules and use topic channels for version capture where applicable (#110)

* Update nf-core modules and fix bam_qc for new samtools API.

Refresh installed modules from nf-core/modules, adapt BAM_QC to the updated samtools view/sort inputs, and regenerate module patch files. Drop obsolete hasheddrops and multiseqdemux patches that no longer differ from upstream.

* Drop manual version mixing for nf-core topic outputs.

Rely on the versions topic collected before MultiQC instead of passing samtools, umitools, untar, csvtk, bff, and hasheddrops versions through subworkflows.

* Emit local module versions via topic and keep patched nf-core mixing.

Local modules publish to the versions topic so subworkflows no longer mix their outputs manually, while patched nf-core modules without topic support continue to use traditional ch_versions plumbing.

* Restore Harshil alignment on local module emit statements.

Re-align output emit columns after adding topic: versions during the topics migration.

* Fix compatibility issues

* Simplify hto null branching

* Fix hto branch logic while preserving dev hash channel behaviour.

Use a side-channel branch so ch_hto stays intact for joins, and mix is_null rows with EXTRACT_HASHES hashes rather than not_null hto paths.

* Update nf-test snapshots for topic-based version collection.

Refresh pipeline software versions YAML after local modules publish to the versions topic.

* Ignore non-deterministic tSNE plots in nf-test snapshots.

tSNE htodemux JPEGs vary across CI and local Docker runs; exclude them from stable-path md5 checks.

* Fix BFF meta map types and record updated module SHA. (#111)

Correct meta output types in bff/meta.yml after the nf-core module update and sync modules.json.

* Remove vulnerable PR-comment artifact pattern (#113)

Co-authored-by: nf-core-bot <core@nf-co.re>

* Important! Template update for nf-core/tools v4.0.3 (#112)

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 3.3.2

* Template update for nf-core/tools version 4.0.0

* Template update for nf-core/tools version 4.0.2

* Template update for nf-core/tools version 4.0.3

* resolve missing merge conflicts

* fix linting errors

* update mulitqc

* nf-core pipelines lint --dir /workspaces/hadge  --fix container_configs

* Fix conda container config file linting

* Align linting workflow

* use stable_name in test

* use input instead of params.input

---------

Co-authored-by: Nico Trummer <nictru32@gmail.com>
Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>

* Prepare Release PR (#114)

* bump-version

* update changelog

* resolve lint errors/warnings

* udpate snapshots

* update conda envs

* bumped SHA to fix similarity issue in CI

* update containers

* add missing versions

* exclude png for testing

* remove png and jpeg from shapshots

* update donor_match snapshot

* update test

* update remaining snapshots

* fix test_hashing

* fix test_genetic

* use || instead of |

* add meta.yml files to local modules using claude

* update resource usage of VIREO and SOUPORCELL

* restore apptainer in container directive of local modules

* restore apptainer in container directive of patched nf-core modules

* add stub and when to FILTER_BAM

* first fixes

* make vireo filtered variants optional for FIND_VARIANTS

* only run cellSNP when vireo or FIND_VARIANTS uses its output

* update  docs

* only allow one samplesheet row in donor_match mode

* add checkIfExists

* fine tune resources of vireo and souporcell

* add mqc plot

* update snapshots

---------

Co-authored-by: nictru <nictru32@gmail.com>
Co-authored-by: nf-core bot <infrastructure@nf-co.re>
Co-authored-by: Matthias Hörtenhuber <mashehu@users.noreply.github.com>
Co-authored-by: nf-core-bot <core@nf-co.re>

0.2.0

Toggle 0.2.0's commit message
Version 0.2.0

Signed-off-by: zethson <lukas.heumos@posteo.net>

0.1.0

Toggle 0.1.0's commit message

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Merge pull request #37 from maxozo/main

Some modifications made to Hadge that were esential for running large scale processing of Onek1k data