A Reference pipeline to annotate euKaryotes at high resolution
The Hiller Lab at the Senckenberg Research Institute
usage .
pipeline .
us
Note
Requirements: Nextflow ≥ 25.04.6, Docker or Apptainer, Java.
git clone https://github.com/hillerlab/ark.git
cd arkEdit params.json (set relevant options/filters), then:
# Docker
nextflow run main.nf -params-file params.json -profile docker
# Apptainer / Singularity
nextflow run main.nf -params-file params.json -profile apptainerSmoke test:
nextflow run main.nf -profile test,apptainerNote
You can also specify these options directly in params.json.
To restart from LIMA outputs, point the input at 02_LIMA and provide the same
primer FASTA used for demultiplexing:
nextflow run src/main.nf -params-file src/params.json -profile docker \
--entrypoint refine --global_input_dir /path/to/02_LIMA \
--global_primers /path/to/primers.fastaThe refine checkpoint reads every *.bam and its matching *.bam.pbi, creating
missing indexes, and skips CCS, Skera, and LIMA. Each BAM is refined independently,
preserving its primer-pair group. BAM names must end in
.<5p_primer>--<3p_primer>.bam.
To restart from refined reads, point the input at 03_ISOSEQ_REFINE; no primer
FASTA is needed:
nextflow run src/main.nf -params-file src/params.json -profile docker \
--entrypoint cluster --global_input_dir /path/to/03_ISOSEQ_REFINEThe cluster checkpoint reads every *.bam, creating missing indexes, and skips
CCS, Skera, LIMA, and refine. BAM names must end in _flnc.bam; the rest of the
name is kept as the sample ID, so a full run's
movie.IsoSeqX_bc01_5p--IsoSeqX_3p_flnc.bam stays
movie.IsoSeqX_bc01_5p--IsoSeqX_3p.
cluster_mode controls clustering of those refined reads: per_sample
keeps each input primer-pair group separate; multi_sample clusters all groups
together under global_prefix; both produces both sets using the same refined
reads. Sample IDs retain the full primer pair, for example
movie.IsoSeqX_bc01_5p--IsoSeqX_3p or movie.NEB_5p--NEB_Clontech_3p.
Their tissue names depend on your experimental barcode assignments; the pipeline
does not infer them. Each input BAM/primer pair is treated as a separate sample,
including when files come from different sequencing runs.
cluster_engine decides whether refined reads are clustered before alignment: none
(default) aligns them directly, as PacBio's current Kinnex workflow does, and
transcript models are built after alignment; isoseq (legacy) runs
isoseq cluster2 first and needs tagged PacBio BAMs. The cluster entrypoint
reads *_flnc.bam only.
For flnc, put FASTA/FASTQ files (.fasta/.fastq, optionally .gz) in
global_input_dir. Public SRA/ENA runs come in every processing state, so
iso-fastx inspect classifies each file from its first reads and routes it:
| State | Looks like | What happens |
|---|---|---|
ccs |
primers at the read ends, both orientations | lima --isoseq orients and clips (kit detected, or global_primers) |
mixed |
primer-free, some reads start with polyT | iso-fastx orient flips the reversed reads |
fl / flnc |
primer-free, oriented, with or without polyA | used as is |
clustered |
transcript/N, PB.x.y names |
used as is; every distinct chain is kept |
subreads, ambiguous |
raw subreads, partial primers, long arrays | the run stops and says why |
--flnc_input_state forces one state for all files (ccs then needs
--global_primers). cluster_mode applies as before: per_sample aligns each
file separately; multi_sample concatenates all files per hq/singleton class
into one pooled sample named global_prefix; both produces both sets. Do not
name an input sample global_prefix: its reads would merge with the pool.
cluster_mode pooled (any entrypoint; it needs global_prefix and
reconstruct_engine chain) keeps the samples apart up to fusion detection and
pools them where transcript models are built: one model set named
global_prefix from the reads of every sample, so ORF prediction and polishing
run once. Read names get their sample as prefix (<sample>.<read>), and
07B_RECONSTRUCT/chain/<global_prefix>/*.samples.tsv lists the reads,
full-chain reads and full-chain molecules of every model in each sample. With
reconstruct_sample_read_fraction (default off), a novel model that falls
below the read fraction of the pool is kept when it holds the fraction among
the locus reads of one sample.
After polyA segmentation and fusion detection, reconstruct_engine chain
(default) collapses the reads of each sample and chromosome into transcript
models by intron chain (collapse chain). Only models reach ORF prediction;
each keeps a real read as its representative and carries its support as
#CN<n>. reconstruct_preset sets how much evidence a novel model needs:
sensitive only removes redundancy (every distinct chain is kept), balanced
(default) needs 2 molecules per novel chain, strict 3. Every read ends up in
a model or in 07B_RECONSTRUCT/chain/<sample>/*.excluded.bed with its reason.
With reconstruct_cds_pool (default off), chains that each lack support but
hold the same reference CDS, held by no kept model, count their molecules
together and the one with the fewest unannotated introns is kept (flag
cds_pooled in *.support.tsv). reconstruct_engine none restores per-read
ORF calling.
A helper sh script is provided to run the pipeline on a SLURM cluster. See details below.
Click to expand
Edit the path variables at the top of assets/hpc/ark.sh (cache dir, container image, manifest path), then submit:
sbatch --array=1-<N> ark.shEach array task spawns one Nextflow head job that submits all compute as child SLURM jobs.
PREDICT_ORFS run as SLURM job arrays. Partition routing, array sizes, and resource tiers are documented inline in nextflow.config — edit there to match your cluster.
results/
├── 00_?/ *bed
└── pipeline_info/ timeline, trace, DAG
| File | What |
|---|---|
params.json |
Genome paths, alignment settings, checkpoints — per run |
nextflow.config |
Compute resources, profiles, container, SLURM — rarely |