Many bioinformatics pipelines are essentially made of mud and straw; quick-and-dirty single-use scripts written in various interpreted languages, with little or no meaningful error reporting and generally poor performance.
Biolibc is a collection of high-quality bricks that can be used to build efficient, robust software applications to take the place of such scripts. Using biolibc, you can develop permanent solutions that are easy to use and install, with near-optimal performance, so that no one ever need reinvent that particular wheel.
For some examples, see biolibc-tools, vcf-split, ad2vcf, vcf2hap, haploh-vcf-depths, peak-classifier, generand, and ad-matrix.
In more technical terms, biolibc is a library of fast, memory-efficient C functions for processing biological data. Like libc, it consists of numerous disparate, general-purpose functions which could be used by a wide variety of applications.
These include functions for reading and writing common file formats such as BED, GFF, FASTA, FASTQ, SAM and VCF, string functions specific to bioinformatics such as chromosome_name_cmp(), detecting feature overlaps, etc.
Biolibc, and the more generic libxtend, allow you to code in C at a higher level by providing the basic building blocks and additional software layers commonly needed in bioinformatics programming. Using C and biolibc, you can write simple, near-optimal C programs that may be orders of magnitude faster than scripting languages such as Perl, Python, and R, while requiring comparable effort. fastx-derep.c, a 1-page C program for dereplicating FASTA/FASTQ files, offers a good example of how the right libraries eliminate the need for low-level programming in C.
See the Research Computing User's Guide Compiled vs Interpreted Languages section for a comparison of language performance.
The code is organized following basic object-oriented design principals, but implemented in C to minimize overhead and keep the source code accessible to scientists who don't have time to master the complexities of C++.
Structures are treated as classes, with accessor and mutator functions (or macros) provided, so dependent applications and libraries need not access structure members directly. Since the C language cannot enforce this, it's up to application programmers to exercise self-discipline.
For detailed coding standards, see https://github.com/outpaddling/Coding-Standards/.
biolibc is intended to build cleanly in any POSIX environment on any CPU architecture. Please don't hesitate to open an issue if you encounter problems on any Unix-like system.
Primary development is done on FreeBSD with clang, but the code is frequently tested on CentOS, MacOS, and NetBSD as well. MS Windows is not supported, unless using a POSIX environment such as Cygwin or Windows Subsystem for Linux.
The Makefile is designed to be friendly to package managers, such as Debian packages, FreeBSD ports, MacPorts, pkgsrc, etc. End users should install via one of these if at all possible.
I maintain a FreeBSD port and a pkgsrc package, which is sufficient to install cleanly on virtually any POSIX platform. If you would like to see a biolibc package in another package manager, please consider creating a package yourself. This will be one of the easiest packages in the collection and hence a good vehicle to learn how to create packages.
FreeBSD is a highly underrated platform for scientific computing, with over 1,900 scientific libraries and applications in the FreeBSD ports collection (of more than 30,000 total), modern clang compiler, fully-integrated ZFS filesystem, and renowned security, performance, and reliability. FreeBSD has a somewhat well-earned reputation for being difficult to set up and manage compared to user-friendly systems like Ubuntu. However, if you're a little bit Unix-savvy, you can very quickly set up a workstation, laptop, or VM using desktop-installer. If you're new to Unix, you can also reap the benefits of FreeBSD by running GhostBSD, a FreeBSD distribution augmented with a graphical installer and management tools. GhostBSD does not offer as many options as desktop-installer, but it may be more comfortable for Unix novices.
pkg install biolibc
pkgsrc is a cross-platform package manager that works on any Unix-like platform. It is native to NetBSD and well-supported on Illumos, MacOS, RHEL/CentOS, and many other Linux distributions. Using pkgsrc does not require admin privileges. You can install a pkgsrc tree in any directory to which you have write access and easily install any of the nearly 20,000 packages in the collection. The auto-pkgsrc-setup script can assist you with basic setup.
First bootstrap pkgsrc using auto-pkgsrc-setup or any other method. Then run the following commands:
cd pkgsrc-dir/biology/biolibc
bmake install clean
There may also be binary packages available for your platform. If this is the case, you can install by running:
pkgin install biolibc
See the Joyent Cloud Services Site for available package sets.
Below are cave man install instructions for development purposes, not recommended for regular use.
biolibc depends on libxtend. Install libxtend before attempting to build biolibc.
- Clone the repository
- Run "make depend" to update Makefile.depend
- Run "make install"
The default install prefix is ../local. Clone biolibc, libxtend and dependent apps into sibling directories so that ../local represents a common path to all of them.
To facilitate incorporation into package managers, the Makefile respects standard make/environment variables such as CC, CFLAGS, PREFIX, etc.
Add-on libraries required for the build, such as libxtend, should be found
under
To install directly to /myprefix, assuming libxtend is installed there as well, using a make variable:
make LOCALBASE=/myprefix clean depend install
Using an environment variable:
# C-shell and derivatives
setenv LOCALBASE /myprefix
make clean depend install
# Bourne shell and derivatives
LOCALBASE=/myprefix
export LOCALBASE
make clean depend install
View the Makefile for full details.