Easy to use, High Performant Knowledge Distillation for LLMs

Train transformer language models with reinforcement learning.

Structured Outputs

Automatic Speech Recognition with Speaker Diarization based on OpenAI Whisper

Building a quick conversation-based search demo with Lepton AI.

Fast low-bit matmul kernels in Triton

Efficient Triton Kernels for LLM Training

Aana SDK is a powerful framework for building AI enabled multimodal applications.

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

Training LLMs with QLoRA + FSDP

Script for HQQification of mixtral from HF's shards

Run Mixtral-8x7B models in Colab or consumer desktops

Official implementation of Half-Quadratic Quantization (HQQ)

llama.cpp with BakLLaVA model describes what does it see

Streamlit Component, for a Chatbot UI

A tiny Scala library to send events and entities to Apache Flume.

Crawl popular image search engines ( google, bing, 500px , flickr ) for images given a query

A collection of routines for building web APIs on top of Tornado web server.