ad2vcf extracts allelic depth info from a SAM stream and adds it to a corresponding single-sample VCF file.
SAM input is read via stdin and the VCF input file is taken as a command-line argument. This allows expensive BAM/CRAM decoding to occur in-parallel using a pipe:
samtools view -@ 2 --input-fmt-option required_fields=0x218 \
../SRR6990379/NWD102903.b38.irc.v1.cram \
| ./ad2vcf file.vcfBoth SAM and VCF inputs must be sorted first by chromosome and then by read/call position.
The code is organized following basic object-oriented design principals, but implemented in C to minimize overhead and keep the source code accessible to scientists who don't have time to master the complexities of C++.
Structures are treated as classes, with accessor macros and mutator functions provided, so dependent applications and libraries need not access structure members directly. Since the C language cannot enforce this, it's up to application programmers to exercise self-discipline.
ad2vcf is written entirely in C and attempts to optimize CPU, memory, and disk access. It does not inhale large amounts of data into RAM, so memory use is trivial and it runs mostly from cache, making it very fast.
Processing a large CRAM file with human genome alignments against from the SRA project against dbGaP VCF data takes about 20 minutes on a modern workstation or server with ad2vcf averaging 4 to 5 MB (not GB) resident memory use. Memory use will spike briefly due to alignment buffering when processing regions where many alignments overlap multiple variant calls.
ad2vcf is intended to build cleanly in any POSIX environment on any CPU architecture. Please don't hesitate to open an issue if you encounter problems on any Unix-like system.
Primary development is done on FreeBSD with clang, but the code is frequently tested on Linux, MacOS, and NetBSD as well. MS Windows is not supported, unless using a POSIX environment such as Cygwin or Windows Subsystem for Linux.
The Makefile is designed to be friendly to package managers, such as Debian packages, FreeBSD ports, MacPorts, pkgsrc, etc. End users should install via one of these if at all possible.
I maintain a FreeBSD port and a pkgsrc package, which is sufficient to install cleanly on virtually any POSIX platform. If you would like to see a package in another package manager, please consider creating a package yourself. This will be one of the easiest packages in the collection and hence a good vehicle to learn how to create packages.
For an overview of available package managers, see the Repology website.
FreeBSD is a highly underrated platform for scientific computing, with over 1,900 scientific libraries and applications in the FreeBSD ports collection (of more than 30,000 total), modern clang compiler, fully-integrated ZFS filesystem, and renowned security, performance, and reliability. FreeBSD has a somewhat well-earned reputation for being difficult to set up and manage compared to user-friendly systems like Ubuntu. However, if you're a little bit Unix-savvy, you can very quickly set up a workstation, laptop, or VM using desktop-installer.
To install the binary package on FreeBSD:
pkg install ad2vcf
You can just as easily build and install from source. This is useful for FreeBSD ports with special build options, for building with non-portable optimizations such as -march=native, and for work-in-progress ports, for which binary packages are not yet maintained.
cd /usr/ports/biology/ad2vcf && env CFLAGS='-march=native -O2' make install
cd /usr/ports/wip/ad2vcf && make install
pkgsrc is a cross-platform package manager that works on any Unix-like platform. It is native to NetBSD and well-supported on Illumos, MacOS, RHEL/CentOS, and many other Linux distributions. Using pkgsrc does not require admin privileges. You can install a pkgsrc tree in any directory to which you have write access and easily install any of the nearly 20,000 packages in the collection. The auto-pkgsrc-setup script can assist you with basic setup.
First bootstrap pkgsrc using auto-pkgsrc-setup or any other method. Then run the following commands:
cd pkgsrc-dir/biology/ad2vcf
bmake install clean
There may also be binary packages available for your platform. If this is the case, you can install by running:
pkgin install ad2vcf
See the Joyent Cloud Services Site for available package sets.
Below are cave man install instructions for development purposes, not recommended for regular use.
ad2vcf depends on biolibc. Install biolibc before attempting to build ad2vcf.
- Clone the repository
- Run "make depend" to update Makefile.depend
- Run "make install"
The default install prefix is ../local. Clone ad2vcf, biolibc and dependent apps into sibling directories so that ../local represents a common path to all of them.
To facilitate incorporation into package managers, the Makefile respects standard make/environment variables such as CC, CFLAGS, PREFIX, etc.
The library, headers, and man pages are installed under ${DESTDIR}${PREFIX}. DESTDIR is empty by default and is primarily used by package managers to stage installations. PREFIX defaults to ../local.
Add-on libraries required for the build, such as biolibc, should be found under either ${PREFIX} or ${LOCALBASE}, which defaults to ${PREFIX}. LOCALBASE can be set independently if you want to use libraries installed by FreeBSD ports (/usr/local), MacPorts (/opt/local), pkgsrc (/usr/pkg), etc.
To install directly to /myprefix, assuming biolibc is installed there as well, using a make variable:
make PREFIX=/myprefix clean depend install
Using an environment variable:
# C-shell and derivatives
setenv PREFIX /myprefix
make clean depend install
# Bourne shell and derivatives
PREFIX=/myprefix
export PREFIX
make clean depend install
View the Makefile for full details.