A Python script to programmatically generate synthetic conversations for training LLMs, chatbots, and dialogue systems.

Fine-tuning causal language models with an additional JEPA-style representation regularisation loss as well as a plain Hugging Face trainer

Fine-tunes a student LLM using teacher feedback for improved reasoning and answer quality. Implements GRPO with teacher-provided evaluations.

Normalized Variance-Aware Max-Penalized Loss

ReasonFlow is a novel framework designed to implement o1-like reasoning capabilities in large language models.

This project implements a fixed-size key-value cache for use in transformer models, specifically designed to work with the LLaMA model. The cache dynamically truncates the key and value states base…

A flexible Monte Carlo Tree Search framework with PyTorch for decision-making in language models.

Blurred-Thoughts Supervised-Finetuning (BT-SFT) is a new approach to fine-tuning language models, focusing on enhancing response diversity and creativity.

This repository contains the implementation of a self-reward head designed for language models. The self-reward head enables the model to autonomously score its generated outputs, promoting self-as…

This repo contains my custom implementation of a mixture of experts as an extension of the linear layer.