Added

• #908 - Add nf-test snapshot for test_single_end profile (by @dialvarezs)
• #1028 - Add nf-test snapshot for test_longread_alternatives profile (by @dialvarezs)
• #1029 - Add nf-test snapshot for test_hybrid profile (by @dialvarezs)
• #1037 - Complement usage documentation with guidance on pipeline defaults, choices and alternatives (by @d4straub and @jfy133 )
• #1039 - Add nf-test snapshot for test_longread profile (started by @brovolia, finished by @dialvarezs)
• #1041 - Refined and corrected unclear section of metromap (by @jfy133)
• #1042 - Add nf-test snapshot for test_default profile (by @dialvarezs)
• #1044 - Add new --gtdbtk_place_species parameter (by @dialvarezs)
• #1047 - Add --gtdbtk_single_job to run GTDB-Tk classification for all bins in a single job (requested by @sarah-shah-bioinf, by @dialvarezs)
• #1048 - Add optional PyPOLCA polishing for long-read assemblies via --run_pypolca (by @Harshita-sriv)
• #1055 - Add dedicated page describing resource usage (by @jfy133)
• #1057 - Add skip_fastqc switch (by @jorondo1)
• #1059 - Add --filtlong_filtering_by_shortreads parameter to enable filtlong's short-read-based long read filtering (by @dialvarezs)
• #1062 - Add --bin_seqkit_stats_max_forks parameter to cap concurrent bin-stats jobs (by @dialvarezs)
• #1063 - Add new --ale_per_base_output parameter to enable ALE per-base output (by @dialvarezs)

Changed

• #1011 - Reverted CheckM2 database download workaround from #966 (by @dialvarezs)
• #1020 - Update CONCOCT subworkflow and modules (by @dialvarezs)
• #1030 - Updated to nf-core 4.0.2 template (by @dialvarezs)
• #1044 - Updated GTDB-Tk to v2.7.2 / GTDB r232 (by @dialvarezs)
• #1059 - Changed the default long read filtering tool from filtlong to chopper (by @dialvarezs)
• #1060 - Updated module tags to make them more specific (by @dialvarezs)
• #1061, #1064 - Speed up binning by not waiting for all mapping jobs to finish before starting (by @dialvarezs)
• #1062 - Remove grouping to prevent bin QC blocking by waiting for all binners (by @dialvarezs)
• #1063 - Run ALE with --metagenome and disable its per-base output by default (by @dialvarezs)
• #1065 - Improved efficiency by removing usage of per-bin GUNZIP module for bins and unbinned contigs and allowing gzip support for all modules (by @dialvarezs)
• #1066 - Improved efficiency by removing channel "locks" on Seqkit, QUAST, and GTDB-Tk (by @dialvarezs)
• #1070 - Update BUSCO nf-core module (by @dialvarezs)
• #1071 - Improved efficiency by batching DASTOOL_FASTATOCONTIG2BIN per binner instead of per bin (by @dialvarezs)
• #1074 - Exclude BUSCO output directories to reduce storage usage in output dir (by @dialvarezs)
• #1075 - Update output documentation to reflect BUSCO complete directory no longer being published (by @dialvarezs)
• #1079 - Add exit code 247 to the MEGAHIT retry error strategy (by @dialvarezs)
• #1080 - Updated to nf-core 4.0.3 TEMPLATE (by @dialvarezs)
• #1081 - Use subset database for geNomad (by @dialvarezs)
• #1084 - Disable NanoPlot static plot image files by default, since it depends on Chrome installed (by @dialvarezs)
• #1088 - Update Prokka (to v1.15.6), Prodigal and PyDamage modules, fixing conda/container output mismatches (by @dialvarezs)
• #1088 - Run GTDB-Tk with a single CPU on the test_single_end profile, so its outputs no longer depend on the machine running the test (by @dialvarezs)
• #1091 - Optimize process resource configs (by @dialvarezs)

Fixed

• #1011 - Fix issue making CheckM2 running only for one sample per run (by @dialvarezs)
• #1012 - Prevent adapter trimming with Porechop on PacBio reads (by @dialvarezs)
• #1016 - Merge input reads on assembly input to prevent repeated filenames on multi-run samples (reported by @erikrikarddaniel, fix by @dialvarezs)
• #1017 - Prevent ALE running on long read assemblies when a sample has both LR and SR data (reported by @jfy133, fix by @dialvarezs)
• #1018 - Ignore catpack/summarise errors, since it's a limitation from the tool (reported by @jfy133, fix by @dialvarezs)
• #1018 - Feed catpack/contig with merged unbinned output to prevent execution errors (reported by @Juassis, fix by @dialvarezs)
• #1021 - Prevent execution of gtdbtk/summary when no bins pass QC (reported by @jfy133, fix by @dialvarezs)
• #1031 - Fix hybrid co-assembly with SPAdes (short & long reads with --coassemble_group) (fix by @d4straub)
• #1049 - Fix publishing issue with gtdbtk/classifywf (by @dialvarezs)
• #1058 - Make create_metabinner_bins.py save gzipped bin files to prevent NFS race condition (by @dialvarezs)
• #1069 - Exclude eukaryotic bins from CheckM2, which only supports bacterial and archaeal genomes (by @dialvarezs)
• #1076 - Add missing PyPOLCA citations to README, CITATIONS.md and the pipeline citation/bibliography text (by @dialvarezs)
• #1078 - Sort bins before GTDB-Tk classification so order-sensitive outputs are reproducible across environments (by @dialvarezs)
• #1084 - Update Prokka nf-core module, which pins GNU parallel to fix Prokka failures with Conda (by @dialvarezs)
• #1086 - Rename MaxBin2 bins inside the MAXBIN2 module via a patch instead of the ADJUST_MAXBIN2_EXT local module, which emitted dangling symlinks rather than bin contents on remote filesystems (by @dialvarezs)
• #1086 - Copy instead of move staged bins in TIARA_CLASSIFY, which emitted dangling symlinks rather than bin contents on remote filesystems (by @dialvarezs)
• #1086 - Match bin filenames exactly in TIARA_CLASSIFY so that e.g. bin .1 no longer also picks up bin .10 (by @dialvarezs)
• #1087 - Use non-mutating toSorted instead of in-place sort on shared channel items, which could cause ConcurrentModificationException failures (by @dialvarezs)
• #1093 - Ignore CheckM storage/ subdirectories in nf-test snapshots, whose contents vary between runs and machines (by @dialvarezs)
• #1094 - Convert depths in a single streaming pass, so CONVERT_DEPTHS no longer writes a decompressed copy of the depth file to the work directory and re-reads it once per read set, which stalled short-read assemblies for hours on object-backed work directories (by @dialvarezs)

Dependencies

Deprecated

• #908 - Removed local quast_bins_summary in favor of csvtk/concat (by @dialvarezs)
• #1018 - Remove mag_depths_plot local module (by @dialvarezs)
• #1018 - Deprecated --gtdbtk_skip_aniscreen in favor of --gtdbtk_place_species (by @dialvarezs)
• #1067 - Deprecated --skip_metaeuk as it has no effect, MetaEuk is gated by `--metaeuk_...

Added

• #1006 - Add citations and references texts in the utils_nfcore_mag_pipeline subworkflow (by @dialvarezs)

Changed

• #1000 - Update GUNC modules and enable the tool in tests (by @dialvarezs)
• #1004 - Replace collectFile with qsv/cat on GUNC summary merging (by @dialvarezs)

Fixed

• #1001 - Include all binners in parameter validation for running DASTool (by @dialvarezs)
• #1002 - Fix BUSCO publish dir to prevent filename collision (by @dialvarezs)
• #1002 - Avoid whole-batch BUSCO failure when classification fails for a single bin. (by @dialvarezs)

Dependencies

Full Changelog: 5.4.1...5.4.2

Changed

• #993 - Tweak test_full resources to stop failures (by @jfy133)
• #995 - Update CheckM and BUSCO modules (by @dialvarezs)

Fixed

• #974 - Re-add accidently removed functionality for using metaSPAdes contigs downstream rather than scaffolds (reported by @Pranjal-Bioinfo, fix by @jfy133)
• #984 - Fix docs regarding usage of Bin QC tool scores when filtering bins for post-processing (by @dialvarezs, @claude)
• #987 - Fix several documentation issues (by @dialvarezs)
• #988 - Fix regarding validation of column long_reads_platform in the input samplesheet (#985 by @vinisalazar)
• #986 - Fix faulty ALE version YAML loading resulting in a snakeyaml error (by @jfy133)
• #992 - Fix command to get python package versions on local modules (by @dialvarezs)

Dependencies

Full Changelog: 5.4.0...5.4.1

Added

• #965 - Add support for the generation of custom contig to bin map files for all bins and refined bins (by @jfy133, @prototaxites)
• #965 - Add support for optional saving of DAS Tool inpout contig2bin map files via --refine_bins_dastool_savecontig2bin, with now singular generation process (by @jfy133, @prototaxites)
• #963 - Add support for outputting and median summarising pyDamage results at bin level when running in ancient DNA mode (by @jfy133)
• #931 - Added ALE (Assembly Likelihood Estimator) for probabilistic assembly quality control (by @PetcuBogdan, @dialvarezs)

Changed

• #966 - Replace aria2c downloading of CheckM2 database with native Nextflow downloading due to Zenodo issues (by @jfy133, @dialvarezs)

Fixed

• #962 - MetaBinner/SemiBin2 are included during bin refinement with DASTool. (by @AlexHoratio)

Dependencies

Added

• #905 - Add nf-test snapshot for test_assembly_input profile (by @dialvarezs)
• #930 - Add binner SemiBin2 (by @d4straub)
• #861 - Added --generate_bigmag_file to execute the bigmag workflow that generates the file to be used as input for BIgMAG (added by @jeffe107)

Changed

• #932 - Replaced usages of deprecated Channel() with channel() and fix other LSP warnings (by @dialvarezs)
• #937 - Updated to nf-core 3.5.1 TEMPLATE (by @dialvarezs)
• #938 - Update nf-core modules (by @dialvarezs)

Fixed

• #894 - Fix read order in metaSPAdes to allow co-assembly of paired-end data of multiple samples (reported by @maartenciers, fix by @jfy133 with contributions from @prototaxites, @d4straub and @dialvarezs)
• #927 - MetaBinner now succeeds when no contigs are too short or all are binned (reported by @MicroSeq, fix by @d4straub)
• #929 - Allow the domain_classification.R script to run with any assembler, not just Megahit or Spades (reported by @MicroSeq, fix by @prototaxites)
• #943 - Fixed concatenation of BUSCO summaries with uneven columns by changing from csvtk to qsv (reported by @jfy133 and @julianu, fix by @dialvarezs)
• #943 - Fixed creation of the Tiara report channel used for concatenation (by @dialvarezs)
• #945 - Skip mixing of GTDB-Tk MultiQC files when binning is skipped (reported by @amizeranschi, fix by @dialvarezs)
• #953 - metaSPAdes retries upon error 250 (out of memory), rather than finishing the pipeline.
• #954 - Skip GTDB-Tk when no bin QC tool is enabled and add warning messages (fix by @dialvarezs)
• #956 - Support long reads assemblers in assembly input (fix by @dialvarezs)

Dependencies

Deprecated

• #943 - Remove csvtk/concat module (by @dialvarezs).

Added

• #842 - Add support for running multiple binQC tools in one run using dedicated --run_busco, --run_checkm, and --run_checkm2 parameters (by @harper357, with contributions from @dialvarezs, @prototaxites and @jfy133)
• #881 - Add binner MetaBinner (by @d4straub, insprired by @HeshamAlmessady & @AlphaSquad)

Changed

• #842 - Change bin_summary.tsv format for improved clarity and more comprehensiveness (by @harper357, with contributions from @dialvarezs, @prototaxites and @jfy133)
  • Now will include columns from all bin QC tools executed in a given run (i.e., all/any of BUSCO, CheckM and CheckM2)
  • Adds suffixes to all columns (_<toolname>) to distinguish which column comes from which tool

Fixed

• #896 - Remove obsolete execution command from README (by @dialvarezs)
• #907 - Include refined bins from all binners in the DASTool/bins output folder (by @AlexHoratio)
• #911 - Ensure column order is consistent when generating depth summaries to prevent swapped results on merged depth summary (by @dialvarezs)
• #912 - Fix validation of multiple sequencing platforms when using binning_map_mode = "all" (reported by @mjfi2sb3, fix by @dialvarezs)
• #921 - Fix publishing of BUSCO files (reported by @joao1980, fix by @dialvarezs)

Dependencies

Deprecated

• #842 - Remove --binqc_tool (by @harper357, with contributions from @dialvarezs, @prototaxites and @jfy133)

Added

• #873 - Document usage of longread_percentidentity and shortread_percentidentity and set the value of longread_percentidentity in the test_full profile to 85 (by @prototaxites)
• #875 - Add binner COMEBin (by @d4straub)

Changed

• #878 - Refine test_full config with optimised resource usage for AWS release megatests (by @jfy133)
• #880 - Updated to nf-core 3.4.1 TEMPLATE (by @jfy133)

Fixed

• #878 - Fix METASPADES process not receiving the correct number of cpus from the fix CPUs parameter (by @jfy133)
• #885 - Fix typo in long-read assembly mode selection (reported by @feixiang1209, fix by @jfy133)
• #888 - Only error if all bins are size filtered if bins have actually been generated (reported by @hkaspersen, fix by @prototaxites)

Dependencies

Added

• #718 - Add support for independent long-read metagenomic assembly (requested by @ljmesi and many others, added by @muabnezor)
• #718 - Added metaMDBG and (meta)Flye as long read assemblers (added by @muabnezor)
• #718 - Added host removal for long reads using minimap2 as aligner (added by @muabnezor)
• #827 - Added nf-test CI testing for all test profiles (added by @jfy133)
• #829 - Add --skip_shortread_qc and --skip_longread_qc params for skipping certain default preprocessing steps (added by @erikrikarddaniel)
• #846 - Improve documentation of group samplesheet column (added by @vinisalazar)
• #855 - Add basic nf-tests for test_longreadonly, test_longreadonly_alternatives, test_hybrid and test_assembly_input (added by @dialvarezs)
• #864 - Add --gtdbtk_skip_aniscreen to disable fast classification of genomes by ANI using skani in GTDB-Tk (by @jfy133 and @prototaxites).

Changed

• #718 - Refactored all assembly steps into subworkflows (added by @muabnezor)
• #799 - Add --cat_classify_unbinned, to enable taxonomic classification of unbinned contigs using CAT (requested by @amizeranschi, added by @dialvarezs)
• #799 - Upgraded to latest version of CAT_pack modules (requested by @maxibor, added by @dialvarezs)
• #811 - Update util modules, and remove aria2 module to replace with native Nextflow downloading of CheckM database (by @dialvarezs)
• #816 - Removed all leftover references to conda 'defaults' channel (by @jfy133)
• #823 - Updated to nf-core 3.3.1TEMPLATE (by @jfy133 )
• #827 - Updated to nf-core 3.3.2TEMPLATE (by @dialvarezs)
• #841 - MultiQC config updated to support CheckM, CheckM2, and GTDB-Tk (by @harper357)
• #844 - Change loading mechanism of internal PhiX/Lambda databases to improve Dev UX when schema building (by @jfy133)
• #851 - Improve structure of local modules and subworkflows (by @dialvarezs)
• #853 - Update nf-core modules and subworkflows (by @dialvarezs)
• #856 - Update more nf-core modules (by @dialvarezs)

Fixed

• #843 - Fixed issue with large format Bowtie2 index files not being emitted from index module (reported by Nick Eckersley, fix by @jfy133)
• #847 - Allow the BBNorm process to use only 0.8 of the memory allocated to the task to stop if from oversubscribing memory (reported by and fix by @erikrikarddaniel)
• #850 - Fixed some modules of the GDTBTk subworkflow not being represented in version lists (fix by @jfy133)
• #852 - Fixed version reporting by ensure all modules are represented in final version.yml for MultiQC (by @jfy133)
• #854 - Update porechop/abi to a patched version to prevent duplicated read names (reported by @palec87, fix by @jfy133)
• #858 - Fix a single parameter validation failure reporting errors for all parameters by updated nf-schema to 2.5.1 (reported by @Pranjal-Bioinfo, fix by @nvnieuwk and @jfy133)
• #864 - Fix missing multi-threading of MetaEuk easypredict (reported by @OlivierCoen, fix by @prototaxites).

Dependencies

Deprecated

• #799 - Removed --cat_official_taxonomy in favour of --cat_allow_unofficial_lineages to control CAT's use of unofficial lineages (added by @dialvarezs)
• #825 - Removed --centrifuge_db, --kraken2_db, --krona_db and --skip_krona parameters following the removal of taxonomic profiling functionality. See nf-core/taxprofiler for replacement (added by @dialvarezs)
• #851 - Remove POOL_READ_* local modules in favor of nf-core cat/fastq (by @dialvarezs)
• #855 - Remove test_adapterremoval, test_ancient_dna, test_bbnorm, test_busco_auto, test_host_rm, test_hybrid_host_rm, test_binrefinement, test_concoct and test_longread profiles (added by @dialvarezs)
• #864 - Remove --gtdb_mash due to dropping of support by GTDBTk itself (by @prototaxites and @jfy133)

Added

• #730 - Added --busco_db_lineage to allow specifying a specific lineage for BUSCO database (added by @jfy133, @dialvarezs)
• #730 - Added a new documentation section on database setup (by @jfy133, @dialvarezs)
• #784 - Added --bin_min_size and --bin_max_size parameters to filter out bins based on size (requested by @maxibor, @alexhbnr, added by @jfy133, @prototaxites)
• #793 - Document use of a SquashFS image with --gtdb_db, useful for limited inode infrastructure (by @muniheart)
• #805 - Add support for fastp's --trim_poly_g option (by @jfy133)

Changed

• #730 - Migrate from local BUSCO module to nf-core one, updating version (by @dialvarezs)
• #730 - Use BUSCO database from nf-core test datasets (by @dialvarezs)
• #788 - Tweak method of loading GTDB database in GTDBTK_CLASSIFYWF for more stability (reported by @alexhbnr, fix by @jfy133)
• #800 - Default branch is now set to main (by @jfy133 and @mirpedrol)
• #801 - Increase CheckM memory requests to match recommended requirements (by @jfy133)

Fixed

• #789 - Improve --bowtie2_mode description to clarify default settings (reported by @IceGreb, fix by @jfy133)
• #798 - Fix overly strict database validation for --metauk_db and improve documentation (reported by @ruqse, fix by @jfy133)
• #804 - Fix broken memory specification for FASTQC (reported by @jmichaelegana, fix by @awgymer & @jfy133)

Dependencies

Deprecated

• #730 - Remove --busco_auto_lineage_prok due to update and simplified usage of BUSCO (added by @jfy133, @dialvarezs)

Added

• #745 - Added pipeline parameter spades_downstreaminput to use contigs instead of scaffolds (by @Pranjal-Bioinfo, @jfy133, @GallVp & @sateeshperi).
• #745 - Added trimmomatic as an additional pre-processing tool (by @Pranjal-Bioinfo, @jfy133, @GallVp & @sateeshperi).
• #745 - Added parameters for concoct/cut_up_fasta.py including bin_concoct_chunksize, bin_concoct_overlap and bin_concoct_donotconcatlast (by @Pranjal-Bioinfo, @jfy133, @GallVp & @sateeshperi).
• #777 - Improved input validation through additional JSON keywords and error messages (by @agusinac)

Changed

• #774 - Update CheckM2 to v1.1.0 and default database (by @dialvarezs).

Fixed

• #726 - Fix formatting errors to follow Nextflow best practice (by @dialvarezs).
• #769 - Fix megahit not emitting correct filenames due to suboptimal arguments ordering (reported and fix by @IceGreb)
• #771 - Fix misspecified checkm2 database parameter check (reported by @dpelegri and fix by @jfy133)

Dependencies