Skip to content
alejandrogziPublic

About

A Reference pipeline to annotate euKaryotes at high resolution

Resources

Stars

2 stars

Watchers

1 watching

Forks

Latest commit

 

History

788 Commits

Folders and files

Repository files navigation

Hiller Lab

ark

GitHub License

A Reference pipeline to annotate euKaryotes at high resolution
The Hiller Lab at the Senckenberg Research Institute

usage . pipeline . us


Usage

Note

Requirements: Nextflow ≥ 25.04.6, Docker or Apptainer, Java.

git clone https://github.com/hillerlab/ark.git
cd ark

Edit params.json (set relevant options/filters), then:

# Docker
nextflow run main.nf -params-file params.json -profile docker

# Apptainer / Singularity
nextflow run main.nf -params-file params.json -profile apptainer

Smoke test:

nextflow run main.nf -profile test,apptainer

Note

You can also specify these options directly in params.json.

To restart from LIMA outputs, point the input at 02_LIMA and provide the same primer FASTA used for demultiplexing:

nextflow run src/main.nf -params-file src/params.json -profile docker \
  --entrypoint refine --global_input_dir /path/to/02_LIMA \
  --global_primers /path/to/primers.fasta

The refine checkpoint reads every *.bam and its matching *.bam.pbi, creating missing indexes, and skips CCS, Skera, and LIMA. Each BAM is refined independently, preserving its primer-pair group. BAM names must end in .<5p_primer>--<3p_primer>.bam.

To restart from refined reads, point the input at 03_ISOSEQ_REFINE; no primer FASTA is needed:

nextflow run src/main.nf -params-file src/params.json -profile docker \
  --entrypoint cluster --global_input_dir /path/to/03_ISOSEQ_REFINE

The cluster checkpoint reads every *.bam, creating missing indexes, and skips CCS, Skera, LIMA, and refine. BAM names must end in _flnc.bam; the rest of the name is kept as the sample ID, so a full run's movie.IsoSeqX_bc01_5p--IsoSeqX_3p_flnc.bam stays movie.IsoSeqX_bc01_5p--IsoSeqX_3p.

cluster_mode controls clustering of those refined reads: per_sample keeps each input primer-pair group separate; multi_sample clusters all groups together under global_prefix; both produces both sets using the same refined reads. Sample IDs retain the full primer pair, for example movie.IsoSeqX_bc01_5p--IsoSeqX_3p or movie.NEB_5p--NEB_Clontech_3p. Their tissue names depend on your experimental barcode assignments; the pipeline does not infer them. Each input BAM/primer pair is treated as a separate sample, including when files come from different sequencing runs.

cluster_engine decides whether refined reads are clustered before alignment: none (default) aligns them directly, as PacBio's current Kinnex workflow does, and transcript models are built after alignment; isoseq (legacy) runs isoseq cluster2 first and needs tagged PacBio BAMs. The cluster entrypoint reads *_flnc.bam only.

For flnc, put FASTA/FASTQ files (.fasta/.fastq, optionally .gz) in global_input_dir. Public SRA/ENA runs come in every processing state, so iso-fastx inspect classifies each file from its first reads and routes it:

State Looks like What happens
ccs primers at the read ends, both orientations lima --isoseq orients and clips (kit detected, or global_primers)
mixed primer-free, some reads start with polyT iso-fastx orient flips the reversed reads
fl / flnc primer-free, oriented, with or without polyA used as is
clustered transcript/N, PB.x.y names used as is; every distinct chain is kept
subreads, ambiguous raw subreads, partial primers, long arrays the run stops and says why

--flnc_input_state forces one state for all files (ccs then needs --global_primers). cluster_mode applies as before: per_sample aligns each file separately; multi_sample concatenates all files per hq/singleton class into one pooled sample named global_prefix; both produces both sets. Do not name an input sample global_prefix: its reads would merge with the pool.

cluster_mode pooled (any entrypoint; it needs global_prefix and reconstruct_engine chain) keeps the samples apart up to fusion detection and pools them where transcript models are built: one model set named global_prefix from the reads of every sample, so ORF prediction and polishing run once. Read names get their sample as prefix (<sample>.<read>), and 07B_RECONSTRUCT/chain/<global_prefix>/*.samples.tsv lists the reads, full-chain reads and full-chain molecules of every model in each sample. With reconstruct_sample_read_fraction (default off), a novel model that falls below the read fraction of the pool is kept when it holds the fraction among the locus reads of one sample.

After polyA segmentation and fusion detection, reconstruct_engine chain (default) collapses the reads of each sample and chromosome into transcript models by intron chain (collapse chain). Only models reach ORF prediction; each keeps a real read as its representative and carries its support as #CN<n>. reconstruct_preset sets how much evidence a novel model needs: sensitive only removes redundancy (every distinct chain is kept), balanced (default) needs 2 molecules per novel chain, strict 3. Every read ends up in a model or in 07B_RECONSTRUCT/chain/<sample>/*.excluded.bed with its reason. With reconstruct_cds_pool (default off), chains that each lack support but hold the same reference CDS, held by no kept model, count their molecules together and the one with the fewest unannotated introns is kept (flag cds_pooled in *.support.tsv). reconstruct_engine none restores per-read ORF calling.

A helper sh script is provided to run the pipeline on a SLURM cluster. See details below.

Click to expand

Edit the path variables at the top of assets/hpc/ark.sh (cache dir, container image, manifest path), then submit:

sbatch --array=1-<N> ark.sh

Each array task spawns one Nextflow head job that submits all compute as child SLURM jobs.

PREDICT_ORFS run as SLURM job arrays. Partition routing, array sizes, and resource tiers are documented inline in nextflow.config — edit there to match your cluster.


Output

results/
├── 00_?/       *bed
└── pipeline_info/    timeline, trace, DAG

Where to edit

File What
params.json Genome paths, alignment settings, checkpoints — per run
nextflow.config Compute resources, profiles, container, SLURM — rarely

About

A Reference pipeline to annotate euKaryotes at high resolution

Resources

Stars

2 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages