dpo

LLM alignment & hallucination reduction: RLHF, DPO, constitutional AI, grounding techniques

A modular, explainable Retrieval-Augmented Generation (RAG) pipeline for automated fact-checking on the FEVER dataset. Includes multi-model evaluation (Mistral, Qwen, GPT), PEFT & DPO fine-tuning, and token-level explainability with Captum, SHAP, and LIME

Experiments with SFT, LoRA, QLoRA, RL post-training

From-scratch RLHF pipeline: reward modeling + PPO (plus DPO & GRPO) in PyTorch — with GAE, KL control, LoRA, multi-GPU, and LLM-as-judge eval.

DPO alignment pipeline for preference data collection, training, and evaluation

SFT + DPO fine-tuning of Qwen3-1.7B for math reasoning (GSM8K/MATH), QLoRA on a single 6GB GPU

Practical implementation and comparison of SFT, Reward Modeling, PPO, DPO, ORPO, and LLM-as-a-Judge for preference alignment with Qwen2.5-0.5B-Instruct.

This is the DPO Group plugin for WooCommerce.

TrainSight: Sub-millisecond pre-flight LLM dataset profiler predicting HBM OOM crashes, padding waste, and attention entropy dispersion before GPU allocation. Built on the Two-Factor Law of LLM Compute.

Reproducible QLoRA SFT and DPO experiments for open-weight LLM reasoning, with locked selection, matched controls, and audit-grade provenance.

DPO implementation and evaluation for EPFL CS552 Modern NLP (Spring 2025), with scripts, Hugging Face models, and LightEval-ready datasets.

A React-based tool for constructing fine-tuning datasets with list and grid forms, featuring the ability to download and upload data as JSONL files. This project leverages the react-declarative library to create dynamic, interactive forms for defining user inputs, preferred outputs, and non-preferred outputs, along with associated tools

Same LoRA DPO recipe across model families (Qwen + Mistral) on UltraFeedback, measured with the correct reference-relative metric. Leads with failure analysis.

Fine-tune Llama-3.2-3B (QLoRA/Unsloth) to answer Singapore ElderShield questions closed-book: recall facts, abstain on unknowns, keep general knowledge, stay warm. SFT then DPO on a T4.

[EMNLP 2026] Official code of "Language Chain in Alignment: Cross-lingual Ranking Preference Optimization"

Real-world evaluation framework for autonomous pentesting LLMs — 12-dim safety×performance rubric, SFT/DPO/GRPO post-training comparison, MITRE ATT&CK mapping. MCS Capstone @ USyd.

This model is a security-aligned version of LLaMA-3.1-8B trained to resist Prompt Injection and Jailbreak attacks while maintaining high usability on benign and sensitive-but-legitimate queries

Manage your GDPR/RGPD processing register

🏟️ Modern RL algorithms from scratch — from Q-Learning to GRPO — with clean PyTorch code and interactive notebooks. Compare PPO vs DPO vs GRPO for LLM alignment.

Multi-domain LLM fine-tuning and serving pipeline (LoRA, QLoRA, DPO) — 70% cheaper than proprietary APIs across 5 domain models.